Question of defualtFetchSize

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Question of defualtFetchSize

yang yu
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

Iwao AVE!
Hello Yang,

Specifying defaultFetchSize=-2147483648 in JDBC URL should enable the streaming resultset feature, with the recent versions at least.
If you still have difficulty with verifying it, please show us how you test it.

Regards,
Iwao

On Mon, Oct 26, 2020 at 8:43 PM yang yu <[hidden email]> wrote:
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2R91g9QogSoFE1aNjNiA%3DU9Mr0R8xjiyHSrBqvyNEzFoA%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

yang yu
thx
A few days ago, the test found that the streaming query did not take effect (fetchsize=0). The situation is that the defaultfetchsize is not set in mybatis, but only set in jdbcurl. After deleting the defaultfetchsize of jdbcurl, and adding the defaultfetchsize to mybatis configration, it took effect, which is very strange. Version information: mybatis 3.4.4, jdbc 5.1.36

Is there a version requirement?


Iwao AVE! <[hidden email]> 于2020年10月27日周二 上午1:37写道:
Hello Yang,

Specifying defaultFetchSize=-2147483648 in JDBC URL should enable the streaming resultset feature, with the recent versions at least.
If you still have difficulty with verifying it, please show us how you test it.

Regards,
Iwao

On Mon, Oct 26, 2020 at 8:43 PM yang yu <[hidden email]> wrote:
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2R91g9QogSoFE1aNjNiA%3DU9Mr0R8xjiyHSrBqvyNEzFoA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CAKi5b%2BKm0%2BVn25rCJ4k75RqOwHs7hEH7miUxpKHUudc0Ond7Fg%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

yang yu
In reply to this post by Iwao AVE!
thx
A few days ago, the test found that the streaming query did not take effect (fetchsize=0). The situation is that the defaultfetchsize is not set in mybatis, but only set in jdbcurl. After deleting the defaultfetchsize of jdbcurl, and adding the defaultfetchsize to mybatis configration, it took effect, which is very strange. Version information: mybatis 3.4.4, jdbc 5.1.36

2020年10月27日 上午1:36,Iwao AVE! <[hidden email]> 写道:

Hello Yang,

Specifying defaultFetchSize=-2147483648 in JDBC URL should enable the streaming resultset feature, with the recent versions at least.
If you still have difficulty with verifying it, please show us how you test it.

Regards,
Iwao

On Mon, Oct 26, 2020 at 8:43 PM yang yu <[hidden email]> wrote:
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2R91g9QogSoFE1aNjNiA%3DU9Mr0R8xjiyHSrBqvyNEzFoA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/99A49D49-199D-4A0E-B89B-F2160BCE7371%40pingcap.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

Iwao AVE!
So, how do you verify if the result set is streamed or not?
Could you share your test case or demo project?


On Tue, Oct 27, 2020 at 4:00 PM yang yu <[hidden email]> wrote:
thx
A few days ago, the test found that the streaming query did not take effect (fetchsize=0). The situation is that the defaultfetchsize is not set in mybatis, but only set in jdbcurl. After deleting the defaultfetchsize of jdbcurl, and adding the defaultfetchsize to mybatis configration, it took effect, which is very strange. Version information: mybatis 3.4.4, jdbc 5.1.36

2020年10月27日 上午1:36,Iwao AVE! <[hidden email]> 写道:

Hello Yang,

Specifying defaultFetchSize=-2147483648 in JDBC URL should enable the streaming resultset feature, with the recent versions at least.
If you still have difficulty with verifying it, please show us how you test it.

Regards,
Iwao

On Mon, Oct 26, 2020 at 8:43 PM yang yu <[hidden email]> wrote:
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2R91g9QogSoFE1aNjNiA%3DU9Mr0R8xjiyHSrBqvyNEzFoA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/99A49D49-199D-4A0E-B89B-F2160BCE7371%40pingcap.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2RGyKQEOowLg666pntrKr1vFZKp2X-G-ZYFEL_Z-QL-qw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

yang yu
Because the use of streaming query still causes the client server to oom, check the hprof file and find that fetchsize=0

2020年10月28日 上午1:18,Iwao AVE! <[hidden email]> 写道:

So, how do you verify if the result set is streamed or not?
Could you share your test case or demo project?


On Tue, Oct 27, 2020 at 4:00 PM yang yu <[hidden email]> wrote:
thx
A few days ago, the test found that the streaming query did not take effect (fetchsize=0). The situation is that the defaultfetchsize is not set in mybatis, but only set in jdbcurl. After deleting the defaultfetchsize of jdbcurl, and adding the defaultfetchsize to mybatis configration, it took effect, which is very strange. Version information: mybatis 3.4.4, jdbc 5.1.36

2020年10月27日 上午1:36,Iwao AVE! <[hidden email]> 写道:

Hello Yang,

Specifying defaultFetchSize=-2147483648 in JDBC URL should enable the streaming resultset feature, with the recent versions at least.
If you still have difficulty with verifying it, please show us how you test it.

Regards,
Iwao

On Mon, Oct 26, 2020 at 8:43 PM yang yu <[hidden email]> wrote:
Set defualtFetchSize=-2147483648 in jdbcUrl, and do not configure defualtFetchSize in the mapper or mybatis configuration file. Should streaming query take effect?

At present, from the test, if you do this, the streaming query does not take effect, I want to confirm

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/e1f599ff-ed18-4a71-a438-dc9ce2df902dn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2R91g9QogSoFE1aNjNiA%3DU9Mr0R8xjiyHSrBqvyNEzFoA%40mail.gmail.com.


--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/99A49D49-199D-4A0E-B89B-F2160BCE7371%40pingcap.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2RGyKQEOowLg666pntrKr1vFZKp2X-G-ZYFEL_Z-QL-qw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/7868C6BB-22A6-4A43-A9DD-3BDC542A865C%40pingcap.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

Iwao AVE!
OK...here is how I tested.

I set a breakpoint on org.apache.ibatis.executor.statement.PreparedStatementHandler#query() and run the test case below in debug mode.

defaultFetchSize=-2147483648 is set in the JDBC URL.

And the actual fetchSize is -2147483648.

image.png

I didn't check if the result set is actually being streamed or not, but if you see fetchSize=0, it may be set by something else.

Regards,
Iwao

On Sat, Oct 31, 2020 at 11:34 AM yang yu <[hidden email]> wrote:
Because the use of streaming query still causes the client server to oom, check the hprof file and find that fetchsize=0

 

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2RytH8YYLEKSxxTsjmOA7KPmOkmUP6SmUHdcywysqCXaQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

yang yu
Thank you very much for your patience. I read your code, it is a simple demo, but I am currently using springboot + mybatis + hikari architecture, I debugged the query() method, because I used hikari, the statement is HikariProxyPrepareStatement, no I found the fetchsize property, and I think the streaming query did not take effect. I wonder if this has anything to do with the connection pool like c3p0 or hikari ?



here is I tested
thank you!

2020年10月31日 上午11:35,Iwao AVE! <[hidden email]> 写道:

OK...here is how I tested.

I set a breakpoint on org.apache.ibatis.executor.statement.PreparedStatementHandler#query() and run the test case below in debug mode.

defaultFetchSize=-2147483648 is set in the JDBC URL.

And the actual fetchSize is -2147483648.

<image.png>

I didn't check if the result set is actually being streamed or not, but if you see fetchSize=0, it may be set by something else.

Regards,
Iwao

On Sat, Oct 31, 2020 at 11:34 AM yang yu <[hidden email]> wrote:
Because the use of streaming query still causes the client server to oom, check the hprof file and find that fetchsize=0

 

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2RytH8YYLEKSxxTsjmOA7KPmOkmUP6SmUHdcywysqCXaQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/B666E850-8F9D-4DC1-B22F-E7791FAA0582%40pingcap.com.
Reply | Threaded
Open this post in threaded view
|

Re: Question of defualtFetchSize

Iwao AVE!
I assumed that you got the expected result when you set fetchSize=-2147483648 in a mapper or MyBatis configuration, but that was not the case, right?
If you want to process result objects one by one, you need to use Cursor.

There are a few tests in the following directory.

Regards,
Iwao

On Sat, Oct 31, 2020 at 2:24 PM yang yu <[hidden email]> wrote:
Thank you very much for your patience. I read your code, it is a simple demo, but I am currently using springboot + mybatis + hikari architecture, I debugged the query() method, because I used hikari, the statement is HikariProxyPrepareStatement, no I found the fetchsize property, and I think the streaming query did not take effect. I wonder if this has anything to do with the connection pool like c3p0 or hikari ?



here is I tested
thank you!

2020年10月31日 上午11:35,Iwao AVE! <[hidden email]> 写道:

OK...here is how I tested.

I set a breakpoint on org.apache.ibatis.executor.statement.PreparedStatementHandler#query() and run the test case below in debug mode.

defaultFetchSize=-2147483648 is set in the JDBC URL.

And the actual fetchSize is -2147483648.

<image.png>

I didn't check if the result set is actually being streamed or not, but if you see fetchSize=0, it may be set by something else.

Regards,
Iwao

On Sat, Oct 31, 2020 at 11:34 AM yang yu <[hidden email]> wrote:
Because the use of streaming query still causes the client server to oom, check the hprof file and find that fetchsize=0

 

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2RytH8YYLEKSxxTsjmOA7KPmOkmUP6SmUHdcywysqCXaQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/B666E850-8F9D-4DC1-B22F-E7791FAA0582%40pingcap.com.

--
You received this message because you are subscribed to the Google Groups "mybatis-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/mybatis-user/CA%2Buep2S%2BjxUcHjn%3DzBWsLtuyXT81oD0akNRGF7Ch4bHaDCSx%2Bg%40mail.gmail.com.