increased memory usage when embedding mono under special environment

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

increased memory usage when embedding mono under special environment

Jovic, Vladimir
Hello,

I am crosscompiling mono 4.4.2 for PPC platform (couldn't compile newer version, as the configuration fails).
Also, I implemented a c++ library, which executes a C# function in DLL. This library is dynamically loaded at runtime (using dlopen) by a program with very special environment. I wouldn't be surprised to hear that this program experience both stack smash and memory overwrites at the same time, but I am not in a position to pick.

Now the problem I am having is: when I use my library in a standalone program, then it works fine. But when I use my library with the program with this special environment, than it uses 40 more megabytes to do some initializations. The parameters for both programs are the same (default, as I do not set any environment variables).

The code to set the mono library is next. As you see, it is a code that can be found in any example of how to execute a c# function from a DLL.

MonoMethod* csMethodEntryPoint = NULL;

void SetMethodEntryPoint( const std::string& ifmDllsPath )
{
    mono_config_parse( NULL );

    MonoDomain* monoDomain = mono_jit_init_version( "pdm_mono_sdk", "v4.0.30319" );

    const std::string assemblyName( ifmDllsPath + "/ifmApiLinuxWrapper.dll" );
    MonoAssembly* assembly = mono_domain_assembly_open( monoDomain, assemblyName.c_str() );
    if ( NULL == assembly )
    {
        throw std::runtime_error( "assembly is NULL" );
    }

    MonoImage* monoImage = mono_assembly_get_image( assembly );
    if ( NULL == monoImage )
    {
        throw std::runtime_error( "monoImage is NULL" );
    }

    MonoClass* entityClass = mono_class_from_name( monoImage, "ifmApiLinuxWrapper", "ifmApiLinuxWrapper" );
    if ( NULL ==  entityClass )
    {
        throw std::runtime_error( "entityClass is NULL" );
    }

    csMethodEntryPoint = mono_class_get_method_from_name( entityClass, "ApiCmdRequest", 2 );
    if ( NULL == csMethodEntryPoint )
    {
        throw std::runtime_error( "entryPoint is NULL" );
    }
}


The function that allocates extra memory is mono_jit_init_version(). In a standalone program, it allocates 1M, and with this special program 20M!
I said above that I lose 40M. So, 19M with mono_jit_init_version(), and extra 21M from a call to c# function.
I took a look into the mono source code for mono_jit_init_version() function, but the function is so huge, that I got lost quite quickly.

The question is how to debug this. Can I set some environment variable to help me track why extra memory is allocated?

Cheers,
Vladimir Jovic


_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: increased memory usage when embedding mono under special environment

Alexander Köplinger via Mono-devel-list
How do you determine said memory usage?  Is your special program somehow messing with malloc?  Note that mono uses malloc for quite some stuff, so if your special program messes with the allocation pattern that could explain it. I suggest to look at it with some profiler.

-Bernhard
________________________________________
From: Mono-devel-list <[hidden email]> on behalf of Jovic, Vladimir <[hidden email]>
Sent: Tuesday, July 4, 2017 2:39:35 PM
To: [hidden email]
Subject: [Mono-dev] increased memory usage when embedding mono under special    environment

Hello,

I am crosscompiling mono 4.4.2 for PPC platform (couldn't compile newer version, as the configuration fails).
Also, I implemented a c++ library, which executes a C# function in DLL. This library is dynamically loaded at runtime (using dlopen) by a program with very special environment. I wouldn't be surprised to hear that this program experience both stack smash and memory overwrites at the same time, but I am not in a position to pick.

Now the problem I am having is: when I use my library in a standalone program, then it works fine. But when I use my library with the program with this special environment, than it uses 40 more megabytes to do some initializations. The parameters for both programs are the same (default, as I do not set any environment variables).

The code to set the mono library is next. As you see, it is a code that can be found in any example of how to execute a c# function from a DLL.

MonoMethod* csMethodEntryPoint = NULL;

void SetMethodEntryPoint( const std::string& ifmDllsPath )
{
    mono_config_parse( NULL );

    MonoDomain* monoDomain = mono_jit_init_version( "pdm_mono_sdk", "v4.0.30319" );

    const std::string assemblyName( ifmDllsPath + "/ifmApiLinuxWrapper.dll" );
    MonoAssembly* assembly = mono_domain_assembly_open( monoDomain, assemblyName.c_str() );
    if ( NULL == assembly )
    {
        throw std::runtime_error( "assembly is NULL" );
    }

    MonoImage* monoImage = mono_assembly_get_image( assembly );
    if ( NULL == monoImage )
    {
        throw std::runtime_error( "monoImage is NULL" );
    }

    MonoClass* entityClass = mono_class_from_name( monoImage, "ifmApiLinuxWrapper", "ifmApiLinuxWrapper" );
    if ( NULL ==  entityClass )
    {
        throw std::runtime_error( "entityClass is NULL" );
    }

    csMethodEntryPoint = mono_class_get_method_from_name( entityClass, "ApiCmdRequest", 2 );
    if ( NULL == csMethodEntryPoint )
    {
        throw std::runtime_error( "entryPoint is NULL" );
    }
}


The function that allocates extra memory is mono_jit_init_version(). In a standalone program, it allocates 1M, and with this special program 20M!
I said above that I lose 40M. So, 19M with mono_jit_init_version(), and extra 21M from a call to c# function.
I took a look into the mono source code for mono_jit_init_version() function, but the function is so huge, that I got lost quite quickly.

The question is how to debug this. Can I set some environment variable to help me track why extra memory is allocated?

Cheers,
Vladimir Jovic


_______________________________________________
Mono-devel-list mailing list
[hidden email]
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.dot.net%2Fmailman%2Flistinfo%2Fmono-devel-list&data=02%7C01%7Cbeurba%40microsoft.com%7Cf316d1bef9ff4526a28e08d4c2d9c199%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636347687894932396&sdata=7STXOC7bR3PWerb3i2SWA8ZjNKY5nuxhrhNa8YdCgj4%3D&reserved=0
_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: increased memory usage when embedding mono under special environment

Jovic, Vladimir
Hello,

To see where the memory goes, I used sysinfo() :
http://man7.org/linux/man-pages/man2/sysinfo.2.html
and I printed before and after calling any mono function in the function bellow. Like that, I see free memory after every function.

As I am developing for a PPC embedded device, I am quite limited in tools. Otherwise, I would use valgrind. Although I am not sure how it would help in this case, as the memory is allocated by mono.

Could it be possible that program arguments, or environment variables, are somehow changing how much memory is allocated by mono?

Cheers,
Vladimir


> -----Ursprüngliche Nachricht-----
> Von: Bernhard Urban [mailto:[hidden email]]
> Gesendet: Dienstag, 4. Juli 2017 15:01
> An: Jovic, Vladimir <[hidden email]>; [hidden email]
> Betreff: Re: [Mono-dev] increased memory usage when embedding mono under
> special environment
>
> How do you determine said memory usage?  Is your special program somehow
> messing with malloc?  Note that mono uses malloc for quite some stuff, so if your
> special program messes with the allocation pattern that could explain it. I suggest
> to look at it with some profiler.
>
> -Bernhard
> ________________________________________
> From: Mono-devel-list <[hidden email]> on behalf of
> Jovic, Vladimir <[hidden email]>
> Sent: Tuesday, July 4, 2017 2:39:35 PM
> To: [hidden email]
> Subject: [Mono-dev] increased memory usage when embedding mono under
> special    environment
>
> Hello,
>
> I am crosscompiling mono 4.4.2 for PPC platform (couldn't compile newer version,
> as the configuration fails).
> Also, I implemented a c++ library, which executes a C# function in DLL. This library
> is dynamically loaded at runtime (using dlopen) by a program with very special
> environment. I wouldn't be surprised to hear that this program experience both
> stack smash and memory overwrites at the same time, but I am not in a position to
> pick.
>
> Now the problem I am having is: when I use my library in a standalone program,
> then it works fine. But when I use my library with the program with this special
> environment, than it uses 40 more megabytes to do some initializations. The
> parameters for both programs are the same (default, as I do not set any
> environment variables).
>
> The code to set the mono library is next. As you see, it is a code that can be
> found in any example of how to execute a c# function from a DLL.
>
> MonoMethod* csMethodEntryPoint = NULL;
>
> void SetMethodEntryPoint( const std::string& ifmDllsPath ) {
>     mono_config_parse( NULL );
>
>     MonoDomain* monoDomain = mono_jit_init_version( "pdm_mono_sdk",
> "v4.0.30319" );
>
>     const std::string assemblyName( ifmDllsPath + "/ifmApiLinuxWrapper.dll" );
>     MonoAssembly* assembly = mono_domain_assembly_open( monoDomain,
> assemblyName.c_str() );
>     if ( NULL == assembly )
>     {
>         throw std::runtime_error( "assembly is NULL" );
>     }
>
>     MonoImage* monoImage = mono_assembly_get_image( assembly );
>     if ( NULL == monoImage )
>     {
>         throw std::runtime_error( "monoImage is NULL" );
>     }
>
>     MonoClass* entityClass = mono_class_from_name( monoImage,
> "ifmApiLinuxWrapper", "ifmApiLinuxWrapper" );
>     if ( NULL ==  entityClass )
>     {
>         throw std::runtime_error( "entityClass is NULL" );
>     }
>
>     csMethodEntryPoint = mono_class_get_method_from_name( entityClass,
> "ApiCmdRequest", 2 );
>     if ( NULL == csMethodEntryPoint )
>     {
>         throw std::runtime_error( "entryPoint is NULL" );
>     }
> }
>
>
> The function that allocates extra memory is mono_jit_init_version(). In a standalone
> program, it allocates 1M, and with this special program 20M!
> I said above that I lose 40M. So, 19M with mono_jit_init_version(), and extra 21M
> from a call to c# function.
> I took a look into the mono source code for mono_jit_init_version() function, but
> the function is so huge, that I got lost quite quickly.
>
> The question is how to debug this. Can I set some environment variable to help me
> track why extra memory is allocated?
>
> Cheers,
> Vladimir Jovic
>
>
> _______________________________________________
> Mono-devel-list mailing list
> [hidden email]
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-
> 252Flists.dot.net-252Fmailman-252Flistinfo-252Fmono-2Ddevel-2Dlist-26data-
> 3D02-257C01-257Cbeurba-2540microsoft.com-
> 257Cf316d1bef9ff4526a28e08d4c2d9c199-
> 257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0-
> 257C636347687894932396-26sdata-
> 3D7STXOC7bR3PWerb3i2SWA8ZjNKY5nuxhrhNa8YdCgj4-253D-26reserved-
> 3D0&d=DwIFAg&c=riR7jviByh3sGm7GIiSlHkFN0_aSATB6A8x0nHa2EM0&r=8wfds
> MUxnRudpdLb_Cg4rnnnKI-
> gQZ4ykr3tjUrejXg&m=UXnPdEnbJm2f8D7zbepyWHCkddBX3nmBbM20hcNRWVw
> &s=HeLECPauo2viZhcAN-930TZQaRS3NLvG4xlxnV-p8sM&e=


_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: increased memory usage when embedding mono under special environment

Alexander Köplinger via Mono-devel-list
As hinted in my previous message I could image that your special program sets some environment variable that the malloc implementation of your embedded system respects, and thus changes the allocation behavior you observe. How about you dump all environment variables before calling into mono?

Also it would be useful to know what libc implementation you're using. What is the absolute memory usage before calling into mono in your special program vs. the example you posted? Maybe you trigger some threshold and malloc allocates some bigger chunk.

There are environment variables to control the heap size of mono (check the man page), but I doubt you set them by accident.

Just some thoughts, no idea what's going on.

-Bernhard

From: Jovic, Vladimir
Sent: Tuesday, July 4, 15:12
Subject: AW: [Mono-dev] increased memory usage when embedding mono under specialenvironment
To: Bernhard Urban, [hidden email]


Hello, To see where the memory goes, I used sysinfo() : https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fman7.org%2Flinux%2Fman-pages%2Fman2%2Fsysinfo.2.html&data=02%7C01%7Cbeurba%40microsoft.com%7Cea86b8b4102a4006b02d08d4c2de5d65%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636347707686498740&sdata=x9UzLkDI%2BdXcA%2BIjcedHbkcLKX1ZKgpww%2F83UvMh3fY%3D&reserved=0 and I printed before and after calling any mono function in the function bellow. Like that, I see free memory after every function. As I am developing for a PPC embedded device, I am quite limited in tools. Otherwise, I would use valgrind. Although I am not sure how it would help in this case, as the memory is allocated by mono. Could it be possible that program arguments, or environment variables, are somehow changing how much memory is allocated by mono? Cheers, Vladimir > -----Ursprüngliche Nachricht----- > Von: Bernhard Urban [mailto:[hidden email]] > Gesendet: Dienstag, 4. Juli 2017 15:01 > An: Jovic, Vladimir ; [hidden email] > Betreff: Re: [Mono-dev] increased memory usage when embedding mono under > special environment > > How do you determine said memory usage? Is your special program somehow > messing with malloc? Note that mono uses malloc for quite some stuff, so if your > special program messes with the allocation pattern that could explain it. I suggest > to look at it with some profiler. > > -Bernhard > ________________________________________ > From: Mono-devel-list on behalf of > Jovic, Vladimir > Sent: Tuesday, July 4, 2017 2:39:35 PM > To: [hidden email] > Subject: [Mono-dev] increased memory usage when embedding mono under > special environment > > Hello, > > I am crosscompiling mono 4.4.2 for PPC platform (couldn't compile newer version, > as the configuration fails). > Also, I implemented a c++ library, which executes a C# function in DLL. This library > is dynamically loaded at runtime (using dlopen) by a program with very special > environment. I wouldn't be surprised to hear that this program experience both > stack smash and memory overwrites at the same time, but I am not in a position to > pick. > > Now the problem I am having is: when I use my library in a standalone program, > then it works fine. But when I use my library with the program with this special > environment, than it uses 40 more megabytes to do some initializations. The > parameters for both programs are the same (default, as I do not set any > environment variables). > > The code to set the mono library is next. As you see, it is a code that can be > found in any example of how to execute a c# function from a DLL. > > MonoMethod* csMethodEntryPoint = NULL; > > void SetMethodEntryPoint( const std::string& ifmDllsPath ) { > mono_config_parse( NULL ); > > MonoDomain* monoDomain = mono_jit_init_version( "pdm_mono_sdk", > "v4.0.30319" ); > > const std::string assemblyName( ifmDllsPath + "/ifmApiLinuxWrapper.dll" ); > MonoAssembly* assembly = mono_domain_assembly_open( monoDomain, > assemblyName.c_str() ); > if ( NULL == assembly ) > { > throw std::runtime_error( "assembly is NULL" ); > } > > MonoImage* monoImage = mono_assembly_get_image( assembly ); > if ( NULL == monoImage ) > { > throw std::runtime_error( "monoImage is NULL" ); > } > > MonoClass* entityClass = mono_class_from_name( monoImage, > "ifmApiLinuxWrapper", "ifmApiLinuxWrapper" ); > if ( NULL == entityClass ) > { > throw std::runtime_error( "entityClass is NULL" ); > } > > csMethodEntryPoint = mono_class_get_method_from_name( entityClass, > "ApiCmdRequest", 2 ); > if ( NULL == csMethodEntryPoint ) > { > throw std::runtime_error( "entryPoint is NULL" ); > } > } > > > The function that allocates extra memory is mono_jit_init_version(). In a standalone > program, it allocates 1M, and with this special program 20M! > I said above that I lose 40M. So, 19M with mono_jit_init_version(), and extra 21M > from a call to c# function. > I took a look into the mono source code for mono_jit_init_version() function, but > the function is so huge, that I got lost quite quickly. > > The question is how to debug this. Can I set some environment variable to help me > track why extra memory is allocated? > > Cheers, > Vladimir Jovic > > > _______________________________________________ > Mono-devel-list mailing list > [hidden email] > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-&data=02%7C01%7Cbeurba%40microsoft.com%7Cea86b8b4102a4006b02d08d4c2de5d65%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636347707686498740&sdata=w72x1rfEWVGqz57A5jXS2mBqfU4CS7Prna8rxuQtlFI%3D&reserved=0 > 3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F- > 252Flists.dot.net-252Fmailman-252Flistinfo-252Fmono-2Ddevel-2Dlist-26data- > 3D02-257C01-257Cbeurba-2540microsoft.com- > 257Cf316d1bef9ff4526a28e08d4c2d9c199- > 257C72f988bf86f141af91ab2d7cd011db47-257C1-257C0- > 257C636347687894932396-26sdata- > 3D7STXOC7bR3PWerb3i2SWA8ZjNKY5nuxhrhNa8YdCgj4-253D-26reserved- > 3D0&d=DwIFAg&c=riR7jviByh3sGm7GIiSlHkFN0_aSATB6A8x0nHa2EM0&r=8wfds > MUxnRudpdLb_Cg4rnnnKI- > gQZ4ykr3tjUrejXg&m=UXnPdEnbJm2f8D7zbepyWHCkddBX3nmBbM20hcNRWVw > &s=HeLECPauo2viZhcAN-930TZQaRS3NLvG4xlxnV-p8sM&e=


_______________________________________________
Mono-devel-list mailing list
[hidden email]
http://lists.dot.net/mailman/listinfo/mono-devel-list
Loading...