)]}'
{"/COMMIT_MSG":[{"author":{"_account_id":1000021,"name":"Antonio Borneo","email":"borneo.antonio@gmail.com","username":"borneoa"},"change_message_id":"a6e547c413486e48375942f5a0d9395b7c8459fe","unresolved":true,"context_lines":[{"line_number":25,"context_line":" \u003e dump_image /dev/null 0x20000000 0x40000"},{"line_number":26,"context_line":" dumped 262144 bytes in 0.601829s (425.370 KiB/s)"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"TODO: Unfortunately some adapters have bugs in the firmware"},{"line_number":29,"context_line":"and does not work reliably with multiple USB requests."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Signed-off-by: Tomas Vanek \u003cvanekt@fbl.cz\u003e"},{"line_number":32,"context_line":"Change-Id: Ic6168ea4eca4f6bd1d8ad541a07a8d70427cc509"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"9737443f_2868fa5f","line":29,"range":{"start_line":28,"start_character":1,"end_line":29,"end_character":54},"updated":"2022-11-22 22:57:45.000000000","message":"Do you have a list of some of these devices?\nWhat is the bad behavior?\nMy understanding is that the USB packets are not sent together nor reordered.\nThis API at first speeds-up the data transfer between userland and OS. Then the OS prepares the sequence in memory and the USB host controller handles them one by one with no further OS involvment.\nThe order of send and recv packets is always guaranteed; and no new packet is sent until the previous gets its answer.\nThe only problem is if the FW in the adapter needs time for internal stuff after it sends the reply and before a new request arrives. Maybe a delay could help, but I don\u0027t see libusb able to specify it.","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000021,"name":"Antonio Borneo","email":"borneo.antonio@gmail.com","username":"borneoa"},"change_message_id":"8ffc349d9de630d714eb36b82485f40aec29d102","unresolved":true,"context_lines":[{"line_number":25,"context_line":" \u003e dump_image /dev/null 0x20000000 0x40000"},{"line_number":26,"context_line":" dumped 262144 bytes in 0.601829s (425.370 KiB/s)"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"TODO: Unfortunately some adapters have bugs in the firmware"},{"line_number":29,"context_line":"and does not work reliably with multiple USB requests."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Signed-off-by: Tomas Vanek \u003cvanekt@fbl.cz\u003e"},{"line_number":32,"context_line":"Change-Id: Ic6168ea4eca4f6bd1d8ad541a07a8d70427cc509"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"fcc8cdc8_86b5563f","line":29,"range":{"start_line":28,"start_character":1,"end_line":29,"end_character":54},"in_reply_to":"95e07ea5_eb0982dd","updated":"2022-11-25 23:14:40.000000000","message":"\u003e I also observed such strange phenomenon like the target read performance went substantially up when there was traffic on the other USB port of the same internal hub. Probably relevant to polling of IN transfers by USB host or some internals of Linux USB implementation...\n\nJust guessing, as I\u0027m not expert on USB.\nBulk is the lowest priority on the USB bus. If you only have bulk, the host try to send a new packet, get NAK because adapter FW is still processing the former one, and host has to re-try after a timeout. If bInterval is too high next packet would be much later keeping bus empty. Some other higher priority traffic in parallel could introduce a small delay between bulk packets that is enough for the FW to process, but smaller that bInterval.\nThen there is optional PING packet. If host doesn\u0027t use it (possible?) then every tentative to send a packet would include the whole data, wasting BW. Again a small delay from another high prio traffic can improve the overall performance.","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"244b6086be6339ddf155f08698e71785282f6c6a","unresolved":true,"context_lines":[{"line_number":25,"context_line":" \u003e dump_image /dev/null 0x20000000 0x40000"},{"line_number":26,"context_line":" dumped 262144 bytes in 0.601829s (425.370 KiB/s)"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"TODO: Unfortunately some adapters have bugs in the firmware"},{"line_number":29,"context_line":"and does not work reliably with multiple USB requests."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Signed-off-by: Tomas Vanek \u003cvanekt@fbl.cz\u003e"},{"line_number":32,"context_line":"Change-Id: Ic6168ea4eca4f6bd1d8ad541a07a8d70427cc509"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"95e07ea5_eb0982dd","line":29,"range":{"start_line":28,"start_character":1,"end_line":29,"end_character":54},"in_reply_to":"9737443f_2868fa5f","updated":"2022-11-25 08:42:58.000000000","message":"Yes, the theory is straightforward.\n\nHowever when I finished with the first version I tested it with 3 CMSIS-DAPv2 adapters: my experimental based on public CMSIS source, Picoprobe (CMSIS source too) and Kitprog 3. Neither of them worked reliably!\n\nI found a not protected critical area in my firmware. After I fixed the bug I\u0027ve got the first working adapter.\n\nThe RP2040 Picoprobe uses USB stack implementation with a character based FIFOs which is IMO very wrong. As soon as more than one packet gets into a FIFO, packet data are concatenated together. Jonathan from RPi is working on this so I expect a new fw will work soon.\n\nI doubt the Kitprog will ever work in pipelined mode, the Cypress\u0027s firmware is a crap as usual and IMO it\u0027s useless to send them bug reports.\n\nI want to add an adapter with a vanilla Mbed CMSIS-DAP source to my test set. Didn\u0027t have time to compile and flash one.\n\nBTW: the good adapter fw uses DMA and IRQ for USB handling, so in the ideal scenario while the adapter is processing one packet, the reply to the previous one is going to the host or the next packet is being received (and the USB host controls if IN or OUT transfer goes first). Without the concurrent processing the speed is much lower...\n\nI also observed such strange phenomenon like the target read performance went substantially up when there was traffic on the other USB port of the same internal hub. Probably relevant to polling of IN transfers by USB host or some internals of Linux USB implementation...","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"e0e974351aee8dec133f1623b01f065ad8ce2c01","unresolved":true,"context_lines":[{"line_number":25,"context_line":" \u003e dump_image /dev/null 0x20000000 0x40000"},{"line_number":26,"context_line":" dumped 262144 bytes in 0.601829s (425.370 KiB/s)"},{"line_number":27,"context_line":""},{"line_number":28,"context_line":"TODO: Unfortunately some adapters have bugs in the firmware"},{"line_number":29,"context_line":"and does not work reliably with multiple USB requests."},{"line_number":30,"context_line":""},{"line_number":31,"context_line":"Signed-off-by: Tomas Vanek \u003cvanekt@fbl.cz\u003e"},{"line_number":32,"context_line":"Change-Id: Ic6168ea4eca4f6bd1d8ad541a07a8d70427cc509"}],"source_content_type":"text/x-gerrit-commit-message","patch_set":5,"id":"c9b2c33f_14d53e5a","line":29,"range":{"start_line":28,"start_character":1,"end_line":29,"end_character":54},"in_reply_to":"fcc8cdc8_86b5563f","updated":"2022-11-26 15:42:58.000000000","message":"\u003e Just guessing, as I\u0027m not expert on USB.\n\nNeither me...\n\n\u003e Bulk is the lowest priority on the USB bus. If you only have bulk, the host try to send a new packet, get NAK because adapter FW is still processing the former one, and host has to re-try after a timeout.\n\n\u003e If bInterval is too high next packet would be much later keeping bus empty.\n\nbInterval is ignored for bulk transfers.\nhttps://www.beyondlogic.org/usbnutshell/usb5.shtml\n\nEven if it were not, bInterval is too rough as expressed in frames (1 ms on USB FS!) so it would totally spoil the speed.\n\n\u003e Some other higher priority traffic in parallel could introduce a small delay between bulk packets that is enough for the FW to process, but smaller that bInterval.\n\nMaybe. The PC has USB 3.0 host, so I assume the delays resulting from mixing USB FS and HS traffic at the internal hub are nearly negligible.\n\n\u003e Then there is optional PING packet.\n\nNot implemented on USB FS, But may play role between hub and host.\n\nMaybe USB host driver employs an adaptive algorithm which changes the polling interval of devices based on bus load. And maybe the algorithm is not much smart. It might explain the difference between CMSIS-DAP write (faster and stable speed) and read (slower, fluctuating speed):\nDuring write the longer packet goes from host to the device and the response is shorter. If the host polls for a response expecting the same transfer time as the previous command took, it should get the response on the first try. During read the command is shorter than the response so the host would get NAK and should retry.\nJust a speculation... but could be easily tested:\n\nasync max 1 pending rq, command size always 64 bytes: read 189 - 197 KiB/s\n\nThe speed did not improved (we have to transfer additional dummy payload) but is now much more stable! The question is if this behaviour is typical on any USB 3.x host/hub or it is caused by a Linux USB driver...","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"}],"/PATCHSET_LEVEL":[{"author":{"_account_id":1000021,"name":"Antonio Borneo","email":"borneo.antonio@gmail.com","username":"borneoa"},"change_message_id":"a6e547c413486e48375942f5a0d9395b7c8459fe","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"5a9e1231_3736ea75","updated":"2022-11-22 22:57:45.000000000","message":"cool! this is another point in my TODO list, but I was more interested in extending the API in libusb_helper.c to have a generic libusb queue that could be used by more adapter drivers.\nMy idea is that the adapter driver sends struct libusb_transfer and at the end calls jtag_libusb_run_queue().\n\nAnyway, please go ahead with this, you already mentioned below there is some problem to be solved with some adapter.","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"e0e974351aee8dec133f1623b01f065ad8ce2c01","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"88c1dea3_7a72d491","in_reply_to":"45d632cc_f13d8481","updated":"2022-11-26 15:42:58.000000000","message":"\u003e \u003e And data read from USB IN? Delivered by a callback?\n\u003e \n\u003e yes\n\nI was unsure if the callback isn\u0027t called from other thread\u0027s context.\nFinally I found the relevant info in https://libusb.sourceforge.io/api-1.0/libusb_io.html:\n\"In the interest of being a lightweight library, libusb does not create threads and can only operate when your application is calling into it.\"\n\nPerfect for us, we don\u0027t need bother with mutexes etc...\n\n\u003e \u003e Another question is if any other USB adapter is ready for pipelined work...\n\u003e \n\u003e I don\u0027t think so!\n\u003e I\u0027m even curious to know if STLink can work properly.\n...\n\u003e While this is the best case, I expect we could still have some BW improvement without pipelining but with the adapter that correctly NAK the incoming packets that cannot be processed because of adapter only able to process one-by-one.\n\u003e The improvement is in much less context switches kernel/userland and USB host having in HW queue all the packets and dispatching them automatically.\n\nI realised I can measure it easily for CMSIS-DAP. I compiled the parent of this commit and compared it to the version with all patches from this set, throttled to send one-by-one:\n\nsync libusb calls: write 286, read 180 - 210 KiB/s\n\nasync max 1 pending rq: write 296, read 160 - 192 KiB/s\n\nNot too impressive speed up in write, unfortunately bigger slow down in read direction! I do not understand why there is such difference between write and read.\nPackets go always in both directions, they are just sized differently.","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"244b6086be6339ddf155f08698e71785282f6c6a","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"e6540d6d_03a77b56","in_reply_to":"5a9e1231_3736ea75","updated":"2022-11-25 08:42:58.000000000","message":"\u003e but I was more interested in extending the API in libusb_helper.c to have a generic libusb queue that could be used by more adapter drivers.\n\nYes, I would like a more universal solution as a long term goal but I felt that we first need a proof of concept and a sandbox for tests. The performance has improved much more than I expected. But there is also lot of drawbacks...\n\n\u003e My idea is that the adapter driver sends struct libusb_transfer and at the end calls jtag_libusb_run_queue().\n\nAnd data read from USB IN? Delivered by a callback?\n\nAnother question is if any other USB adapter is ready for pipelined work...","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000021,"name":"Antonio Borneo","email":"borneo.antonio@gmail.com","username":"borneoa"},"change_message_id":"8ffc349d9de630d714eb36b82485f40aec29d102","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":5,"id":"45d632cc_f13d8481","in_reply_to":"e6540d6d_03a77b56","updated":"2022-11-25 23:14:40.000000000","message":"\u003e And data read from USB IN? Delivered by a callback?\n\nyes\n\n\u003e Another question is if any other USB adapter is ready for pipelined work...\n\nI don\u0027t think so!\nI\u0027m even curious to know if STLink can work properly.\nTime ago was merged a patch to send asynchronously only two transfers (cmd and reply) to improve the speed of STlink with OpenOCD under VM. The developer didn\u0027t went more that that; I never asked if performance was already enough or there were other issues.\n\nHere you mention \u0027pipeline\u0027 and below you talk about adapter receiving next USB packet while still processing the current one.\nWhile this is the best case, I expect we could still have some BW improvement without pipelining but with the adapter that correctly NAK the incoming packets that cannot be processed because of adapter only able to process one-by-one.\nThe improvement is in much less context switches kernel/userland and USB host having in HW queue all the packets and dispatching them automatically.","commit_id":"e3ab2ec2d677b6bc1fc18992f28e4ef52bec321e"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"11a53d6c0b39cfda5b3c206d29a047eb3dd92933","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":7,"id":"e97916bb_52421334","updated":"2022-11-30 10:44:45.000000000","message":"Finally managed to compile DAPLink source with gcc toolchain (the project is for ARM toolchain only), flashed it to a blue-pill and tested with async transfers. It doesn\u0027t loose packets and the speed (write 267 / read 216 KiB/s) is doubled in comparison with the OpenOCD compiled from the parent commit.\n\nI\u0027m thinking how not to break support of buggy firmwares which does not work with async.\nI would keep it simple and introduce a single config command\n cmsis_dap quirk on|off\n\nand it would limit pending reqests to one, shorten the usable packet size to prevent zero length packet problems and enable ugly disconn/conn workaround before SWD sequence.","commit_id":"9c8d54066475c71ae45e05f32009d0bc77273629"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"f827e04df690697a1d103deecb60dbd3e18bb5c9","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":7,"id":"d8503b8f_d1489e52","updated":"2022-11-30 17:42:45.000000000","message":"KitProg 3 with the latest fw 2.40 works perfectly with async too! Not bad speed (write 350, read 302 KiB/s). IMO could do even better considering that it generates SWD signals for whole operation by the on-chip programmable logic. And a very positive surprise: the source code of firmware is publicly available on github!!","commit_id":"9c8d54066475c71ae45e05f32009d0bc77273629"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"9b3eee0b40caa68eb5cb671cc5ebe81bf0ac50a4","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":9,"id":"b2bd60c5_d9f3d65b","updated":"2023-10-03 16:22:50.000000000","message":"The last adapter not capable handling async usb bulk pipelining is fixed:\nhttps://github.com/raspberrypi/picoprobe/pull/47\n\nCurrently I\u0027m not aware of any CMSIS-DAP adapter not working with this patch.\nLet\u0027s proceed to merge...","commit_id":"41dd195f011abea27d3cbf87a9d1f2eda7a88d07"},{"author":{"_account_id":1000853,"name":"zapb","display_name":"Marc Schink","email":"dev@zapb.de","username":"zapb"},"change_message_id":"efc4454d11a377c4a0a74796182b78ccbe260b4d","unresolved":false,"context_lines":[],"source_content_type":"","patch_set":10,"id":"0f3e9f21_b2bf3ec0","updated":"2023-11-07 19:52:50.000000000","message":"Tested on MSPM0 Launchpad, no regression found. Also, the throughput is noticeably increased.","commit_id":"a6642b794c67b59cb7d377a9443d5138a4d8bdae"}],"src/jtag/drivers/cmsis_dap_usb_bulk.c":[{"author":{"_account_id":1000021,"name":"Antonio Borneo","email":"borneo.antonio@gmail.com","username":"borneoa"},"change_message_id":"fe052753bf0b1e5b96d114bdb6cad8279f4663f3","unresolved":true,"context_lines":[{"line_number":34,"context_line":"#include \"libusb_helper.h\""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"#if !defined(LIBUSB_API_VERSION) || (LIBUSB_API_VERSION \u003c 0x01000105) \\"},{"line_number":37,"context_line":"\t\t|| defined(_WIN32) || defined(__CYGWIN__)"},{"line_number":38,"context_line":"\t#define libusb_dev_mem_alloc(dev, sz) malloc(sz)"},{"line_number":39,"context_line":"\t#define libusb_dev_mem_free(dev, buffer, sz) free(buffer)"},{"line_number":40,"context_line":"#endif"}],"source_content_type":"text/x-csrc","patch_set":7,"id":"c5a7ff3b_072a8c18","line":37,"updated":"2022-11-24 22:16:40.000000000","message":"Are you sure you need this change?\nI don\u0027t remember any bug report and jenkins built previous version of this patch without problems\nAnd in the change in this version the function below are not used.","commit_id":"9c8d54066475c71ae45e05f32009d0bc77273629"},{"author":{"_account_id":1000687,"name":"Tomas Vanek","display_name":"Tomas Vanek","email":"vanekt@fbl.cz","username":"vanekt"},"change_message_id":"244b6086be6339ddf155f08698e71785282f6c6a","unresolved":true,"context_lines":[{"line_number":34,"context_line":"#include \"libusb_helper.h\""},{"line_number":35,"context_line":""},{"line_number":36,"context_line":"#if !defined(LIBUSB_API_VERSION) || (LIBUSB_API_VERSION \u003c 0x01000105) \\"},{"line_number":37,"context_line":"\t\t|| defined(_WIN32) || defined(__CYGWIN__)"},{"line_number":38,"context_line":"\t#define libusb_dev_mem_alloc(dev, sz) malloc(sz)"},{"line_number":39,"context_line":"\t#define libusb_dev_mem_free(dev, buffer, sz) free(buffer)"},{"line_number":40,"context_line":"#endif"}],"source_content_type":"text/x-csrc","patch_set":7,"id":"5c298214_ee1402e5","line":37,"in_reply_to":"c5a7ff3b_072a8c18","updated":"2022-11-25 08:42:58.000000000","message":"Although libusb_dev_mem_alloc() is defined for Windows, it just returns NULL.\nI want to check if libusb_dev_mem_alloc() speed gain is worth to fiddling with and otherwise go back for malloc().","commit_id":"9c8d54066475c71ae45e05f32009d0bc77273629"}]}