String interning

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

String interning

Hayes, Peter

Does mybatis support optionally interning strings?  We sometimes load a lot of the same strings from the database and it would be memory efficient if we could intern some of these so we don’t waste so much heap with the same string value.

 

Thanks,

Peter Hayes

 

Reply | Threaded
Open this post in threaded view
|

Re: String interning

Larry Meadors
I don't see any disadvantage to doing that as the default behavior
instead of an option.

IMO, get the source, make the change and test it out - if the
improvement is there, we'll get it in.

Larry


On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]> wrote:

> Does mybatis support optionally interning strings?  We sometimes load a lot
> of the same strings from the database and it would be memory efficient if we
> could intern some of these so we don’t waste so much heap with the same
> string value.
>
>
>
> Thanks,
>
> Peter Hayes
>
>
Tim
Reply | Threaded
Open this post in threaded view
|

Re: String interning

Tim
I don't know if you know what happens when a string is interned. But
assuming that you are caching and interning, this could have some
seriously bad effects on your running program. Interning strings is
highly use case specific in my opinion.

On Thu, Feb 17, 2011 at 3:24 PM, Larry Meadors <[hidden email]> wrote:

> I don't see any disadvantage to doing that as the default behavior
> instead of an option.
>
> IMO, get the source, make the change and test it out - if the
> improvement is there, we'll get it in.
>
> Larry
>
>
> On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]> wrote:
>> Does mybatis support optionally interning strings?  We sometimes load a lot
>> of the same strings from the database and it would be memory efficient if we
>> could intern some of these so we don’t waste so much heap with the same
>> string value.
>>
>>
>>
>> Thanks,
>>
>> Peter Hayes
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: String interning

Larry Meadors
Yeah, I was reading more, and now I am thinking it's probably not a
great idea for a few reasons:

The JDBC driver is most likely doing new String(byte[]), so the string
will go on the heap anyway - so that's happening before it gets to us.

...then we'd intern it and it'll go into permgen - effectively
doubling the memory use. Snap.

IMO, intern() is a method that probably shouldn't have been exposed -
I suspect it's primary use is by the compiler.

I don't think that the memory savings are going to be there. :-/

If you wanted to try it more easily (w/o changing the mybatis code), I
think you could do it with a type handler.

Larry
Reply | Threaded
Open this post in threaded view
|

RE: String interning

Hayes, Peter
In reply to this post by Tim
Yes, I agree.  We have some very denormalized tables that we are reading from.  The strings that are candidates for interning are "code" values.  These string values are returned 100k+ times in a result set but there are only 3 - 10 variations of the string.  We could save quite a bit of memory in these cases.  We can accomplish the same by modifying our setter methods to intern the string:

public void setCode(String code) {
        this.code = code != null ? code.intern() : null;
}

I would rather have this managed by the framework though.  I'll look into a custom type handler for these use cases.

Thanks,
Pete

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim
Sent: Thursday, February 17, 2011 6:52 PM
To: [hidden email]
Cc: Larry Meadors; Hayes, Peter
Subject: Re: String interning

I don't know if you know what happens when a string is interned. But
assuming that you are caching and interning, this could have some
seriously bad effects on your running program. Interning strings is
highly use case specific in my opinion.

On Thu, Feb 17, 2011 at 3:24 PM, Larry Meadors <[hidden email]> wrote:

> I don't see any disadvantage to doing that as the default behavior
> instead of an option.
>
> IMO, get the source, make the change and test it out - if the
> improvement is there, we'll get it in.
>
> Larry
>
>
> On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]> wrote:
>> Does mybatis support optionally interning strings?  We sometimes load a lot
>> of the same strings from the database and it would be memory efficient if we
>> could intern some of these so we don't waste so much heap with the same
>> string value.
>>
>>
>>
>> Thanks,
>>
>> Peter Hayes
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: String interning

Larry Meadors
Why not make an enum with those values instead of using String?


On Fri, Feb 18, 2011 at 6:11 AM, Hayes, Peter <[hidden email]> wrote:

> Yes, I agree.  We have some very denormalized tables that we are reading from.  The strings that are candidates for interning are "code" values.  These string values are returned 100k+ times in a result set but there are only 3 - 10 variations of the string.  We could save quite a bit of memory in these cases.  We can accomplish the same by modifying our setter methods to intern the string:
>
> public void setCode(String code) {
>        this.code = code != null ? code.intern() : null;
> }
>
> I would rather have this managed by the framework though.  I'll look into a custom type handler for these use cases.
>
> Thanks,
> Pete
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim
> Sent: Thursday, February 17, 2011 6:52 PM
> To: [hidden email]
> Cc: Larry Meadors; Hayes, Peter
> Subject: Re: String interning
>
> I don't know if you know what happens when a string is interned. But
> assuming that you are caching and interning, this could have some
> seriously bad effects on your running program. Interning strings is
> highly use case specific in my opinion.
>
> On Thu, Feb 17, 2011 at 3:24 PM, Larry Meadors <[hidden email]> wrote:
>> I don't see any disadvantage to doing that as the default behavior
>> instead of an option.
>>
>> IMO, get the source, make the change and test it out - if the
>> improvement is there, we'll get it in.
>>
>> Larry
>>
>>
>> On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]> wrote:
>>> Does mybatis support optionally interning strings?  We sometimes load a lot
>>> of the same strings from the database and it would be memory efficient if we
>>> could intern some of these so we don't waste so much heap with the same
>>> string value.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Peter Hayes
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

RE: String interning

Hayes, Peter
That could work as long as we don't mind hard coding the values in the Java code.  In a lot of these types of cases though, I think we probably do.   I'll take a look at that support as well.

Thanks,
Pete

-----Original Message-----
From: Larry Meadors [mailto:[hidden email]]
Sent: Friday, February 18, 2011 8:13 AM
To: Hayes, Peter
Cc: [hidden email]
Subject: Re: String interning

Why not make an enum with those values instead of using String?


On Fri, Feb 18, 2011 at 6:11 AM, Hayes, Peter <[hidden email]> wrote:

> Yes, I agree.  We have some very denormalized tables that we are reading from.  The strings that are candidates for interning are "code" values.  These string values are returned 100k+ times in a result set but there are only 3 - 10 variations of the string.  We could save quite a bit of memory in these cases.  We can accomplish the same by modifying our setter methods to intern the string:
>
> public void setCode(String code) {
>        this.code = code != null ? code.intern() : null;
> }
>
> I would rather have this managed by the framework though.  I'll look into a custom type handler for these use cases.
>
> Thanks,
> Pete
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Tim
> Sent: Thursday, February 17, 2011 6:52 PM
> To: [hidden email]
> Cc: Larry Meadors; Hayes, Peter
> Subject: Re: String interning
>
> I don't know if you know what happens when a string is interned. But
> assuming that you are caching and interning, this could have some
> seriously bad effects on your running program. Interning strings is
> highly use case specific in my opinion.
>
> On Thu, Feb 17, 2011 at 3:24 PM, Larry Meadors <[hidden email]> wrote:
>> I don't see any disadvantage to doing that as the default behavior
>> instead of an option.
>>
>> IMO, get the source, make the change and test it out - if the
>> improvement is there, we'll get it in.
>>
>> Larry
>>
>>
>> On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]> wrote:
>>> Does mybatis support optionally interning strings?  We sometimes load a lot
>>> of the same strings from the database and it would be memory efficient if we
>>> could intern some of these so we don't waste so much heap with the same
>>> string value.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Peter Hayes
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: String interning

Clinton Begin
Administrator
You could create a string type handler replacement that implements an
'least frequently used' type cache semantic.  You may even be able to
scope it to the session, but I'd have to think about that.  Better
than string intern at least.

Clinton

On 2011-02-18, Hayes, Peter <[hidden email]> wrote:

> That could work as long as we don't mind hard coding the values in the Java
> code.  In a lot of these types of cases though, I think we probably do.
> I'll take a look at that support as well.
>
> Thanks,
> Pete
>
> -----Original Message-----
> From: Larry Meadors [mailto:[hidden email]]
> Sent: Friday, February 18, 2011 8:13 AM
> To: Hayes, Peter
> Cc: [hidden email]
> Subject: Re: String interning
>
> Why not make an enum with those values instead of using String?
>
>
> On Fri, Feb 18, 2011 at 6:11 AM, Hayes, Peter <[hidden email]> wrote:
>> Yes, I agree.  We have some very denormalized tables that we are reading
>> from.  The strings that are candidates for interning are "code" values.
>>  These string values are returned 100k+ times in a result set but there
>> are only 3 - 10 variations of the string.  We could save quite a bit of
>> memory in these cases.  We can accomplish the same by modifying our setter
>> methods to intern the string:
>>
>> public void setCode(String code) {
>>        this.code = code != null ? code.intern() : null;
>> }
>>
>> I would rather have this managed by the framework though.  I'll look into
>> a custom type handler for these use cases.
>>
>> Thanks,
>> Pete
>>
>> -----Original Message-----
>> From: [hidden email] [mailto:[hidden email]]
>> On Behalf Of Tim
>> Sent: Thursday, February 17, 2011 6:52 PM
>> To: [hidden email]
>> Cc: Larry Meadors; Hayes, Peter
>> Subject: Re: String interning
>>
>> I don't know if you know what happens when a string is interned. But
>> assuming that you are caching and interning, this could have some
>> seriously bad effects on your running program. Interning strings is
>> highly use case specific in my opinion.
>>
>> On Thu, Feb 17, 2011 at 3:24 PM, Larry Meadors <[hidden email]>
>> wrote:
>>> I don't see any disadvantage to doing that as the default behavior
>>> instead of an option.
>>>
>>> IMO, get the source, make the change and test it out - if the
>>> improvement is there, we'll get it in.
>>>
>>> Larry
>>>
>>>
>>> On Thu, Feb 17, 2011 at 2:06 PM, Hayes, Peter <[hidden email]>
>>> wrote:
>>>> Does mybatis support optionally interning strings?  We sometimes load a
>>>> lot
>>>> of the same strings from the database and it would be memory efficient
>>>> if we
>>>> could intern some of these so we don't waste so much heap with the same
>>>> string value.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Peter Hayes
>>>>
>>>>
>>>
>>
>

--
Sent from my mobile device