Write cache on local Windows 10 machine for asynchronous transfer to NAS?

I have an application that generates a stream of data as a single file. The amount of generated data is variable, but in average wouldn’t nearly saturate my network or NAS speeds. However, there can be peaks for a few seconds that totally go above what’s possible.

I don’t have control over the application’s behavior, which is just dropping data if the transfer can’t keep up. If I generate the data stream onto my local SSD or even HDD, which is way slower than my NAS, everything is fine. I guess the latter one results from Windows’ behavior of internally caching write operations in RAM. Is there some way to get this enabled for write operations on my NAS, too? Or are there other solutions available that would be able to solve this problem?

Also the whole data stream wouldn’t fit on my SSD or even HDD, so just dumping it there and copying or syncing it later/asynchronous isn’t an option for me. Ideally, if at all, I’d just want the non-synced portion of the file to reside somewhere on my local machine.

I am using Windows 10 Pro on my local machine. The NAS is a DS220+ and mounted via SMB.

Answer

Windows uses cache memory for all file writes, which also pertains to
files residing on network shares.
Cache memory is important – if it is not large enough
for containing the data generated during these few seconds of
peak activity, the best solution would be to increase
the amount of RAM in the computer until it can contain it all,
catching up with writing it out when activity calms down.

You could also increase the size of the pagefile, so the RAM could
swap out more. Insure that the pagefile is on the SSD rather than the HDD.

You could try to improve the network speed with the aim of helping
Windows to empty the RAM faster toward the NAS, therefore postponing
the time that the RAM will fill up.

The following operations might help (or not). You would need to
try them and see. A good test might be measuring the speed of a
large file-copy to the NAS (larger than the RAM).

  • Invest in a faster network adapter, if the NAS supports it

  • Configure in Windows and in the NAS the network adapters
    jumbo frames,
    receive buffers, and interrupt moderation rate.
    On Windows, this is done in Device Management, right-click the network
    adapter, Properties, Advanced tab. The available settings depend
    on the adapter and the driver.
    Receive buffers might be more important on the NAS side, and this is
    done via modprobe or ethtool, depending on the adapter.

  • Turn off the Windows firewall and disable the anti-virus.

  • Use NFS for better performance. See the article
    Network share: Performance differences between NFS & SMB.

You may find more details in the articles:

Attribution
Source : Link , Question Author : Num Lock , Answer Author : harrymc

Leave a Comment