Quantcast

How to write files > 4GB with pwrite in Mono C# in 32-bit OS?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to write files > 4GB with pwrite in Mono C# in 32-bit OS?

pedrom71
I'm trying to write a new file with size > 64 gb using mono c# and Mono.Posix
library, with the Syscall.pwrite() function. But in a 32-bit OS, I get error
EOVERFLOW when the offset reach 4GB.

This is the example code I'm using:

    public unsafe static void Main (string[] args)
    {
        int fd = Syscall.open("/dev/null", OpenFlags.O_WRONLY);
        long bLen = 4096;
        long fLen = 5368709120;
        byte[] buffer = new byte[bLen];
        long c = 0;
        long res = -1;
        if (fd == -1)
            goto Finish;

        do
        {
            fixed (byte *pb = buffer)
            {

                res = Syscall.pwrite(fd, pb, (ulong)bLen, c);
                if (res == -1)
                    break;
                c+= bLen;
            }
        }
        while (c < fLen);
        Syscall.close(fd);

        Finish:
        if (res == -1)
        {
            Console.WriteLine("Error in offset " + c.ToString());
            Console.WriteLine("Error: " + Stdlib.GetLastError());
        }
        else
        {
            Console.WriteLine("All OK");
        }
        Console.ReadLine();
    }


In C, the solution is compiling the code with gcc with parameter
-D_FILE_OFFSET_BITS=64. But in mono I haven't found the solution yet.

*Note: This error only happens in a 32-bit system. In a 64-bit OS works
correctly.



--
View this message in context: http://mono.1490590.n4.nabble.com/How-to-write-files-4GB-with-pwrite-in-Mono-C-in-32-bit-OS-tp4670286.html
Sent from the Mono - Dev mailing list archive at Nabble.com.
_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to write files > 4GB with pwrite in Mono C# in 32-bit OS?

Jonathan Pryor
On Mar 18, 2017, at 5:47 PM, pedrom71 <[hidden email]> wrote:
> I'm trying to write a new file with size > 64 gb using mono c# and Mono.Posix
> library, with the Syscall.pwrite() function. But in a 32-bit OS, I get error
> EOVERFLOW when the offset reach 4GB.

Because libMonoPosixHelper.so believes you can’t do that:

        https://github.com/mono/mono/blob/9b824c3/support/unistd.c#L60-L67

Specifically, it asserts that your `offset` value can fit within a `off_t`. On 32-bit platforms, `off_t` is usually a 32-bit value, so your 64-bit value cannot fit, so it returns EOVERFLOW.

Though I also see:

        https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html

> Macro: _FILE_OFFSET_BITS
> …
> If the macro is defined to the value 64, the large file interface replaces the old interface. I.e., the functions are not made available under different names (as they are with _LARGEFILE64_SOURCE).

This doesn’t mention what happens with `off_t`, but I also see:

        http://users.suse.com/~aj/linux_lfs.html
> Compile your programs with "gcc -D_FILE_OFFSET_BITS=64". This forces all file access calls to use the 64 bit variants. Several types change also, e.g. off_t becomes off64_t.

This strongly implies that your libMonoPosixHelper.so isn’t being compiled with `-D_FILE_OFFSET_BITS=64`. You should investigate the compilation flags that are used when building within the mono/support directory.

 - Jon

_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Loading...