I have a gaming laptop and a phone, both of which have USB 3.2 Gen 2 and Gen 1 ports. I also use a USB 4 Type-C cable. Now, recently, I have downloaded music files of over 300GB. If I transfer them one by one, it takes a lot of time. Today, I compressed the folder to a single zip file, and the transfer finished in less than 20m. Why is that so?
I’ve had better luck with running an ftp server on my phone. I don’t know how they screwed up cabled transfers so profoundly badly, but I’ve experienced the same issue generation after generation.
Androids storage doesn’t show up as mass storage since quite a while ago. Instead it uses MTP which in my experience is slower and way less reliable.
Maybe that’s also part of the reason?
Yeah, I think it’s about MTP. It has always been slow and buggy in my experience.
Yeah, MTP is worthless. I always use
adb pushif I need to transfer a lot of stuff.Is adb pull an option to get data off the phone?
Yes,
adb pulltransfers from the phone to the computer.
Do you use any GUI program for this or just plain old cmd?
Adb is a command line program. To copy your music folder from home on your PC to internal storage on your phone, you would just open a terminal abd type
adb push ~/Music /sdcard. Pressing tab will autocomplete folder and file names. For some reason/sdcardon android is actually the internal storage. If you have an SD card, it will be in/storage/card_name, where card_name is the UUID of the partition on the card.You do have to enable USB debugging on your phone before you can use adb to transfer files though.
I’ll try this out. Thanks.
It has probably something to do with the large quantities of individual files.
It works through the list of files one by one. The indexing, writing and checking of individual files takes longer than one single .zip file. Thus zipping them first increases the overall speed.
Isn’t this just going to be happen when the zip is decompressed, thus not saving time? I would actually expect it to be worse, since now you’re reading and writing from the same drive instead of reading from one, and writing it to another.
No, there’s a massive difference between doing something local and doing something over a hotpluggable connection.
USB by default, especially in Windows, does a lot of extra work to make sure nothing gets corrupted in transit, and that if the cable comes unplugged, nothing gets corrupted.
When you unzip on your local system, it’s just like sending an accountant into the back to unbox something. It’s one process, going as fast as they can, with local resources ready at hand.
When transferring a ton of files over USB, it’d be more like asking someone over the phone to send the contents of the box over, one by one. So now you have someone on the far end rummaging around for stuff meant for the box, packaging it up and sending items off one by one, telling a second person at the receiving end about each in turn, and only moving on once the receiver confirmed that one item came across OK.
The difference is insane. It’s probably even more overhead than the above example implies.
This does not match my experiences. Transferring files over USB would absolutely be faster than sending a zip and unzipping it on a flash drive. I can easily do 300MB/s over USB3.2 when transferring music files.
Unzipping a large file is going to be a bunch of reads and write and the large file is going to transfer at the exact same speed as the smaller music files, which are not “small”, they’re still tens of MB. So, the zip and music files take roughly the same time except now you have to wait to unzip with one large file. It does not save time.
Transferring tens of thousands of 1kb files will slow things down, and I’d zip this, but music files are big enough.
Not zipped to a flash drive. Zipped and sent over the same USB cable as sending the bunch of files.
The actual transfer bandwidth attained does rely a ton on what connection speed gets negotiated. The overhead of how at least Windows deals with USB is very noticeable at lower speeds. 3.1 or less and I can guarantee you the zip option might start looking like a valid choice.
Of course if you get 3.2gen2+ speeds negotiated, it’s going to be ‘fast enough’ either way assuming the devices can deliver on read/write…
Understood. I’m also talking about sending a full zip over to the flash drive, then unzipping it on that same flash drive.
Music files are large enough to not get affected by overhead like sending a ton of 1kb files. I see no significant difference in transfer time sending 100 10mb files or a single 1000mb file.
This is a totally different story with actually small files (ie kilobytes). Music downloads are not small, they’re multiple megabytes.
Well if you’re unzipping on the flash drive, that’s a whole mountain worse than copying device to device. Why would you want to torture the flash drive and your patience like that?
Right… that’s what I’m saying! My entire point.
Sending a zip of music files to a phone, then decompressing that zip on the phone, seems like a really stupid idea to me. You’ve now set up a situation where you’re reading and writing to one drive rather than reading from one and writing to another.
I’d imagine so
for each file, the computer has to ask the phone about the file, wait for the phone to process the file request, and respond. Then it can start transferring the file. With a single file, it can copy everything in one go without stopping.
You’re basically being bottlenecked by your phone’s CPU, and by high latency in a single-threaded task.
No, is not the phone’s CPU. I guarantee you that can process several gigabytes a second.
It’s the overhead of all that handshaking and confirmation coming over USB, and having to be verified with the sometimes slow phone storage chips or worse on a microSD.
It’s IO overhead and waiting all over the place, not a CPU bottleneck.
Why does this overhead not exist when I’m sending files over USB to an external HDD or flash drive?
I have an external HDD array connected via USB 3.2 and it handles file transfers same as a SATA drive. There’s no handshaking beyond the initial negotiation of the USB connection, certainly not on a per-file basis.
That’s using eSATA for a protocol and not USB in all likelihood. Also a lot of the handshaking and confirmation come from the USB driver itself. You won’t see anything fancy, it’ll just be annoyingly slow.
Interesting, so eSATA protocol can be used over a USB physical link?
Only ones that support it. It has a different plug and pins, there is just a port that can take both.
If you’re plugging in with a normal USB cable, it could be doing a few other tricks to not use the normal USB way of transferring files, or just a fast enough device (eg: 3.2g2+) to not notice the overhead without a side by side.
Most of the ‘tricks’ are just usually having a driver or controller chip open up a raw data pipe (the universal serial part of USB) and exposing the drive as … anything other than removable storage. If the OS doesn’t see the drive as removable, it won’t do most of the crazy overhead stuff of confirming and validating each transfer.
Also if it’s USB4, it can expose the device as just a pcie device (I don’t remember if it’s native to 4 or has to be an overlap with Thunderbolt but eh), or a DisplayPort data stream. That’s how those newfangled docks connect to the host while keeping speed up while the dock has every plug under the sun; they’re all devices down stream of the mimic’d pcie endpoint, which gets to blast data over raw.
This external drive I have is just USB mass storage. It’s using the USB protocol and shows as removable external storage, same as a flash drive.
Removable storage is not the issue here. There is no significant overhead introduced by having removable storage with USB mass storage protocol.
OP is forced into using MTP which does suck and adds overhead. This is not the “normal” way of transferring to me, that would be USB mass storage.
Self-hosted. Open source. Your data stays on your own devices. Creates a shared folder on your laptop and your phone. Move a file into that folder on your laptop, and syncthing pushes it to your phone.
I tried it but the connection speed is limited to my internet speeds for some unknown reason. Like, both my laptop and phone support Wi-Fi 6 but speeds are always around 22 MB/s max.
Which Wi-Fi standard does your router / access point support?
Try this is as an alternative:
Same as SyncThing; transfer speeds are around 22 MB/s and less.
Iirc this should give a good reason why, regardless of OS, though he is primarily talking about the windows file transfer dialog. --> https://youtu.be/9gTLDuxmQek
Thanks for the video. I’ll watch it.
