I have been chasing my tail trying to download a very large to my desktop. The bug is strange enough that it might be specific to hardware. The issue is that I can not copy very large files accurately. The threshold value size of the file seems to be about 8589934592. Any file this size or larger seems to be uncopyable on my desktop. I had downloaded a corrupted file of 71.9GB with Ubuntu 20.04, Ubuntu 22.04, Windows 10 virtua machine running under Ubuntu 20.04, Manjaro. All of theses OS;s were running on the same hardware. I had no issues downloading the file from a raspberry pi with and external hard drive. I have included the C file I used to create the files which also has some notes opn the results of the tests. Any advice or help would be appreciated.

Thank you for reminding me about memtest86, I had a bad memory module.  I am trying to RMA it now.   It seems unlikely that the memory would go bad after about a year.  Live and learn.... Ubuntu just updated to 22.04 with the kernel 5.15.0-27-genericOn 4/28/2022 1:56 PM, Chuck Anderson wrote:

What are the kernel version(s)? What filesystem?

I suggest:

- Try memtest86+

- Try a live Fedora 35 on a USB stick since all your other tests were relying on Ubuntu, even the Windows 10 which was a VM on Ubuntu. Fedora usually has much newer kernels, so if there is a Linux bug, it might be fixed already.

On Thu, Apr 28, 2022 at 11:55:04AM -0400, Kevin Stratton via WLUG wrote:

...

I have been chasing my tail trying to download a very large to my desktop. The bug is strange enough that it might be specific to hardware.

The issue is that I can not copy very large files accurately. The threshold value size of the file seems to be about 8589934592. Any file this size or larger seems to be uncopyable on my desktop.

I had downloaded a corrupted file of 71.9GB with Ubuntu 20.04, Ubuntu 22.04, Windows 10 virtua machine running under Ubuntu 20.04, Manjaro. All of theses OS;s were running on the same hardware.

I had no issues downloading the file from a raspberry pi with and external hard drive.

I have included the C file I used to create the files which also has some notes opn the results of the tests.

Any advice or help would be appreciated.

doug> I manually added Kevin back ....full blown (Kevin Stratton style) on my reply header above. I just like seeing it. Or causing trouble. Ignore and/or X-out ifyou hit reply here..pardon the noise. THE BIG FILE debug. In general, first I think it's very cool appropriate to post an (easy-ish) C-code read/write task! I stripped comments, repasted below, pardon again: #define NUMBER 0x1FFFFFFFF // copies without md5 detected error,file len 8589934591 int main() { FILE * fp; uint32_t rand; uint64_t count; fp= fopen ("bigfile","w+"); for (count=0;count First, try the memtest86+ , reply-to-list, and i hope we swap stories on memtest later after we figure out your fun puzzle, I definitely want to hear where you ran into (+found) trouble! BUT you didn't show the shell command (syntax) that YOU RAN...was it quiet output? (what was the output?) OR... did you get error messages or stop midway on you? I await your chapter 2.... as you said it could be HW, is semi-random AND i don't get your whole story yet (close...) --doug On Thu, Apr 28, 2022 at 11:55 AM Kevin Stratton via WLUG < wlug@lists.wlug.org> wrote:

I have been chasing my tail trying to download a very large to my desktop. The bug is strange enough that it might be specific to hardware.

The issue is that I can not copy very large files accurately. The threshold value size of the file seems to be about 8589934592. Any file this size or larger seems to be uncopyable on my desktop.

I had downloaded a corrupted file of 71.9GB with Ubuntu 20.04, Ubuntu 22.04, Windows 10 virtua machine running under Ubuntu 20.04, Manjaro. All of theses OS;s were running on the same hardware.

I had no issues downloading the file from a raspberry pi with and external hard drive.

I have included the C file I used to create the files which also has some notes opn the results of the tests.

Any advice or help would be appreciated.

_______________________________________________ WLUG mailing list -- wlug@lists.wlug.org To unsubscribe send an email to wlug-leave@lists.wlug.org Create Account: https://wlug.mailman3.com/accounts/signup/ Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ Web Forum/Archive: https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/UQDQPF...

Yeah I was thinking about how to minimize resource usage, as Python has a much larger runtime compared to a compiled-compiled program. I've ported the Python program to Rust: https://gist.github.com/joshua-stone/576e5826302a720331a2686f25c957f1 When compiled with MUSL, LTO, and static-linking, then finally run through `strip`, it'll produce a 391KB binary with no external libraries, not even glibc! I'm sure plenty can be done to further optimize memory footprint, but this has been a fun exercise already. On Fri, Apr 29, 2022 at 11:05 AM Kevin Stratton via WLUG < wlug@lists.wlug.org> wrote:

Thank you everyone for your attention to this.

I was able to get a free (as in free beer) memtest86 installed on a USB drive. Memtest86 did lead me to a bad memory module. I am operating on 1/2 ram until I get a replacement ram module. Memtest86+ does not seem ready for a "secure boot" environment.

When I did the copy I was talking about, I was in the folder that the file exists: I did a simple "cp file file2 "with no options. There was no error messages of any kind, the ram issue was silent, if I recall correctly the bad copied file was always the same size as the original. The issue lead to this was that I could not install a very large software package. I would have random issues that were related to a corrupted file.

I wanted to use the minimum resources possible withe C code to eliminate confounding factors.

I am currently downloading the file and I will try to install the application today.

On 4/28/2022 4:44 PM, Doug Mildram wrote:

doug> I manually added Kevin back ....full blown (Kevin Stratton style) on my reply header above. I just like seeing it. Or causing trouble. Ignore and/or X-out ifyou hit reply here..pardon the noise.

THE BIG FILE debug. In general, first I think it's very cool appropriate to post an (easy-ish) C-code read/write task! I stripped comments, repasted below, pardon again:

#define NUMBER 0x1FFFFFFFF // copies without md5 detected error,file len 8589934591 int main() { FILE * fp; uint32_t rand; uint64_t count;

fp= fopen ("bigfile","w+"); for (count=0;count

dmildram/doug> First, try the memtest86+ , reply-to-list, and i hope we swap stories on memtest later after we figure out your fun puzzle, I definitely want to hear where you ran into (+found) trouble! BUT you didn't show the shell command (syntax) that YOU RAN...was it quiet output? (what was the output?) OR... did you get error messages or stop midway on you? I await your chapter 2.... as you said it could be HW, is semi-random AND i don't get your whole story yet (close...) --doug

On Thu, Apr 28, 2022 at 11:55 AM Kevin Stratton via WLUG < wlug@lists.wlug.org> wrote:

...

I have been chasing my tail trying to download a very large to my desktop. The bug is strange enough that it might be specific to hardware.

The issue is that I can not copy very large files accurately. The threshold value size of the file seems to be about 8589934592. Any file this size or larger seems to be uncopyable on my desktop.

I had downloaded a corrupted file of 71.9GB with Ubuntu 20.04, Ubuntu 22.04, Windows 10 virtua machine running under Ubuntu 20.04, Manjaro. All of theses OS;s were running on the same hardware.

I had no issues downloading the file from a raspberry pi with and external hard drive.

I have included the C file I used to create the files which also has some notes opn the results of the tests.

Any advice or help would be appreciated.

_______________________________________________ WLUG mailing list -- wlug@lists.wlug.org To unsubscribe send an email to wlug-leave@lists.wlug.org Create Account: https://wlug.mailman3.com/accounts/signup/ Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ Web Forum/Archive: https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/UQDQPF...

_______________________________________________ WLUG mailing list -- wlug@lists.wlug.org To unsubscribe send an email to wlug-leave@lists.wlug.org Create Account: https://wlug.mailman3.com/accounts/signup/ Change Settings: https://wlug.mailman3.com/postorius/lists/wlug.lists.wlug.org/ Web Forum/Archive: https://wlug.mailman3.com/hyperkitty/list/wlug@lists.wlug.org/message/IHIQQR...

If I understand this one liner correctly, dd is transferring 82 blocks of size 104857600.  My original testing was: sudo /dev/random > bigfile And I monitored the size of the file with ls -l in a different terminal.  Then I CTRL-C to stop the transfer. LInux is not a major part of my vocation (today?) At risk of being a "broken record", my problem has been solved by fixing my ram.  I can now work with a 71.8GB zip with no issues. On 4/29/2022 6:05 PM, Chuck Anderson wrote:

Why not just use "dd":

dd if=/dev/urandom bs=104857600 count=82 of=bigfile

On Fri, Apr 29, 2022 at 01:03:19PM -0400, Joshua Stone via WLUG wrote:

...

Yeah I was thinking about how to minimize resource usage, as Python has a much larger runtime compared to a compiled-compiled program.

I've ported the Python program to Rust:

https://gist.github.com/joshua-stone/576e5826302a720331a2686f25c957f1

When compiled with MUSL, LTO, and static-linking, then finally run through `strip`, it'll produce a 391KB binary with no external libraries, not even glibc!

I'm sure plenty can be done to further optimize memory footprint, but this has been a fun exercise already.

On Fri, Apr 29, 2022 at 11:05 AM Kevin Stratton via WLUG < wlug@lists.wlug.org> wrote:

...

Thank you everyone for your attention to this.

I was able to get a free (as in free beer) memtest86 installed on a USB drive. Memtest86 did lead me to a bad memory module. I am operating on 1/2 ram until I get a replacement ram module. Memtest86+ does not seem ready for a "secure boot" environment.

When I did the copy I was talking about, I was in the folder that the file exists: I did a simple "cp file file2 "with no options. There was no error messages of any kind, the ram issue was silent, if I recall correctly the bad copied file was always the same size as the original. The issue lead to this was that I could not install a very large software package. I would have random issues that were related to a corrupted file.

I wanted to use the minimum resources possible withe C code to eliminate confounding factors.

I am currently downloading the file and I will try to install the application today.