one project

[rachelwxm]

I have created a program that actually go through my handhistories and pick up chips count in different stages like bubble, 3handed, HU, etc and calculate the equity according to ICM and compare it with how I did to show my ROI breakdown of different stages.

Anybody interested in seeing results? /images/graemlins/smile.gif
Of course, I am still accumulating handhistories.

[lorinda]

Sounds like fun, and also useful if you can find the equity dips.

Lori

[rachelwxm]

what do u mean by "equity dips"? I do track everybody's chips count if that's what you mean.

[lorinda]

If you start from, say, five handed, you will get a chart that shows your value at each point when a player gets knocked out.

You may find that your EV is lower at some points than others, which means you are playing worse around that point.

If you have an EV of $4.55 at the start of four handed play, $3.33 at the start of three handed play and a return of $4.01 after the tourney, it might mean that your bubble play needs work.

Lori

[rachelwxm]

[ QUOTE ]

You may find that your EV is lower at some points than others, which means you are playing worse around that point.

If you have an EV of $4.55 at the start of four handed play, $3.33 at the start of three handed play and a return of $4.01 after the tourney, it might mean that your bubble play needs work.

Lori

[/ QUOTE ]

That's exactly what I aim for. Since SNGs have very different stages, early, bubble, 3handed, HU, I believe each stage require slight adjustment and different skills. So to see my equity at different stages will help me work on specific stages. /images/graemlins/cool.gif

[spentrent]

What language do you use? I can hack Perl/PHP/C/SQL -- if you have more ideas to implement I'd be more than happy to pitch in!

[skirtus]

Very Cool. Im not bright enough to write programs like this. I have trouble just getting my confuser to work properly. If you need more hand histories to test out I can send you some. Are you going to share this program once you have it working?

[rachelwxm]

[ QUOTE ]
What language do you use? I can hack Perl/PHP/C/SQL -- if you have more ideas to implement I'd be more than happy to pitch in!

[/ QUOTE ]
VBA, and C++. I have not used UNIX for a few years now.

[AleoMagus]

What seems most interesting to me about this is not the practical aspects of evaluating your play, but rather the theoretical aspect of looking at maybe thousands of hand histories (If a lot of people contributed) and comparing the eventual $ Equity associated with chip counts at certain stages with the ICM predictions.

Tallstack was discussing an idea ike this with me and I think it would be a really good idea.

In this way, we could get a much better idea of whether the ICM values make any sense at all, and could even match the resulting $EV values to some other, more appropriate model if it did not match up well.

Regards
Brad S

[silversurfer]

had i any inkling you were this intelligent when we played heads up a few weeks ago, i might have been a little more nervous /images/graemlins/wink.gif

[rachelwxm]

AleoMagus, you are always thinking one step ahead of me. /images/graemlins/cool.gif
I guess you are talking about a totally different level or problem than what I try to handle right now. You want to prove/disprove ICM using millions of emperical data. Are you aware someone try this before?

[rachelwxm]

Did I lose that one? I have to admit /images/graemlins/smile.gif one reason trigger my work is I want to prove to myself that I suck at HU play in a quantitative way.

[The Yugoslavian]

Aleo,

This is a great idea! I am willing to pitch in the SNG HH I have but I have to warn that it's not much (in the scheme of things).

I can pitch in a couple hundred 20/2 HH, over one hundred 10/1 HH and a dozen 30/3 HH. These are all from PP.

Frankly, if enough people will volunteer HH and Rachel et al are up for it, then your (and Tallstack's) idea could be used on the different limits to see if there are any weird equity shifts that take place depending on the higher buyins (perhaps due to tighter/looser play).

The ICM model developed could be the 'TallMagus' model (or the AleoStack model for added confusion).
/images/graemlins/grin.gif

[The Yugoslavian]

This would definitely be a cool program to have access to since I am not nearly sophisticated enough to program it on my own.

Hell, I'd love to have a program that will calculate the $equity of pushing/folding in any given situation without me having to first calculate the possible random hands and then crunching the different chip counts in ICM to get a $EV. I've been bugging my CS inclined roommate to work on this but he doesn't care and only plays ring games right now.

Does anyone have code that does the random hands and ICM calculations automatically for push/fold situations?

[rachelwxm]

One thing I just realized, in order to test the ICM theory, you need to get average player's HH. Unfortunately HHs of people from this board would have some systematic bias on average hopefully. /images/graemlins/smile.gif

[silversurfer]

despite your loss, your play was solid throughout the game, from what i can recall. to say you suck hu is harsh.

i think any method of analyzing one's game and results can only be a good thing, as long as you don't overdo it and continue to work on everythign as a whole. what do you hope to infer from the results? are you using it to assess your aggressiveness and how it relates to your ROI? not sure I follow the correlation.

[rachelwxm]

The goal here is (assuming ICM is correct here) break down ROI into different stage and work on leaks that way. I also like to adress questions like aggressiveness on bubble try to get lots of 1st or 4th finishes.

[silversurfer]

groovy. i'd be interested to see how it turns out.

[tallstack]

While I am replying to Aleo's post I first want to say to Rachel that I think that your program idea is fantastic. I have also been toying with a very similar idea and have not yet started on it. I do have some rough text parsing code, but I am nowhere close to ready to use it for this application.

AleoMagus (Brad) and I have talked a little about a very similar idea to evaluate the accuracy of the ICM using a large pool of HHs. If you would be interested, then I think that this could be a pretty useful study for the SNG crowd.

I think that your idea will be great for diagnosing weaknesses in one's play as describe by Lori. For that reason alone, you may not want to share the program with others who will improve their play against you. But if you are interested then that would be great.

I also think that Brad's point of view regarding the accuracy of the ICM should be attempted first regardless of whatever other uses are planned. This is for the reason that you should have some evidence the ICM is accurate enough before you interpret a result as a relfection of your play as opposed to an error in the model.

From zephyr's post, he outlined some potential flaws that he sees in the ICM in terms of linear assumptions. In particular, HU win% varying linearly with chip counts could be easily evaluated using a large HH sample. If it works well in this situation, then evaluate 3 handed play, 4 handed play, etc. I think that a varied pool of HH data would not be too hard to put together and would generate a lot of emperical data that could be measured against both the ICM and any other mathematical model. We may end up preferring a different model much more than the ICM.

Dave S

[tallstack]

I am not sure that this is a big problem or not. For one, you can use every hand history, including those where you bust out early. I know that I make it to HU play about 26% of the time, so 74% of my hand histories would involve two unknown players when HU. Also, it is now possible to get HHs from party just by observing tables. They are exacty the same as the tourney histories mailed out at the end, but if you wanted a purely random sample then you could get a large amount of data by this 'data mining' approach.

Dave

[rachelwxm]

[ QUOTE ]
From zephyr's post, he outlined some potential flaws that he sees in the ICM in terms of linear assumptions.

[/ QUOTE ]
could you give a link?
As I mentioned before, it's hard to get losing players' HH in order to testify ICM. You would need a average sample pool of -9% ROI. /images/graemlins/tongue.gif

[tallstack]

I am referring to the 'The ICM is not all its cracked up to be' post. It is still hanging around on page 3 as we speak. zephyr link (http://forumserver.twoplustwo.com/showthreaded.php?Cat=&Number=1169224&page=2&view=c ollapsed&sb=5&o=14&fpart=1)

Dave S

[rachelwxm]

[ QUOTE ]

From zephyr's post, he outlined some potential flaws that he sees in the ICM in terms of linear assumptions. In particular, HU win% varying linearly with chip counts could be easily evaluated using a large HH sample.

[/ QUOTE ]
Dave, I am not too familiar with the derivation of ICM. But there are two ways to show HU equity is linear to chips, by either random walk model (applied to low blinds case) or by random all in method(assume blinds are very large).

Random walk model says: if you are a bug at x between 0 and 1 moving randomly, what is the probability that you hit 0 before hit 1? The answer is 1-x.

Allin model, lets start with 50% 50% chips p=1/2 then work out 25% 75%, 1/3 2/3 chips and you would get the same thing.

The thing I am not sure is how to work out 3 person case.

[tallstack]

I have to admit I have never looked at any models other than the ICM so far, and I don't know the origins of ICM either. I have recently ordered TPFAP, and I am hoping that this is covered in there somewhere. I do hear about the random walk method from time to time, but I do not know how it works from an algorithm point to view. It seems like it would be a method that requires n trials of simulation, but I could be wrong.

I do believe that when tested, the HU equity will look quite linear, but who knows for sure until you try. This would be a quick one to complete before moving on to more interesting multi-player scenarios. If for some reason the results are not very linear for HU play then that would throw a big wrench in using ICM for any number of players. /images/graemlins/laugh.gif

Dave S

[rachelwxm]

[ QUOTE ]
I do believe that when tested, the HU equity will look quite linear, but who knows for sure until you try. This would be a quick one to complete before moving on to more interesting multi-player scenarios. If for some reason the results are not very linear for HU play then that would throw a big wrench in using ICM for any number of players. /images/graemlins/laugh.gif

Dave S

[/ QUOTE ]

Just to estimate the data you need, let's say HU equal stack, your equity= 40, STD=10 or 100%. In order to be accurate to 1%, you need to have 10000 HHs. And forming a linear curve would certainly need more data. And that's just for HU. /images/graemlins/ooo.gif

[tallstack]

I see what you are saying, but do we need an accuracy of +/- 1% on each data point?

Here is what I have been thinking for an initial comparison method. Calculate the equity for each player at the start of HU play assuming a linear relationship between .3 and .5 based on chip count. Put them into bins say .3 to .305, .305 to .31, etc. I am thinking of 20 bins to start. Find the average equity for a given bin and compare that to the average $equity for the end payouts from that bin.

You would get two data points (certainly not two independant data points) from each SNG. If you could get 1000 SNGs then you would have an average of 100 points in each bin (some of the end bins would likely have quite a few less). This should be enough to see a trend. I think that with 10000 SNGs you would have a very good idea of whether a linear relationship was working well or not.

There will be some uncertainty, but if you graphed the bin average vs $equity average for that bin then you should have a good visual display of what the relationship is, even if the individual data points have errors much greater than 1%.

What do you think?

Dave S

[rachelwxm]

Sounds like a fair plan. But 1000 HU HH is still like a pretty daunting number. My program is not conpatible with party poker yet it was originally for poker star.

[tallstack]

I only play on Party now. /images/graemlins/frown.gif I likely do have some $5+$.5 HHs from my PS two-table days that you are welcome to use if you would like. I don't know how many I would have, but certainly less than 100.

I only have about 500 or so HHs saved from PP, but I am thinking there would some interested people who could volunteer some of theirs. Or alternatively I could supply the code for finding the HU data and let them just send in the HU data if they were worried about giving out a huge amount of data on how they play. Even getting one full-time 4 or 8 tabler to join in would send the HH number skyrocketing /images/graemlins/smile.gif

Anyways, I am putting the cart before the horse here. I will try to devote some programming time this weekend. Maybe I can get some text searching code working pretty quickly.

Dave S

[eastbay]

[ QUOTE ]

Anybody interested in seeing results? /images/graemlins/smile.gif
Of course, I am still accumulating handhistories.

[/ QUOTE ]

I clicked on about 20 replies before giving up on finding results.

My answer: YES.

Can you post them now?

eastbay

[eastbay]

[ QUOTE ]
[ QUOTE ]

From zephyr's post, he outlined some potential flaws that he sees in the ICM in terms of linear assumptions. In particular, HU win% varying linearly with chip counts could be easily evaluated using a large HH sample.

[/ QUOTE ]
Dave, I am not too familiar with the derivation of ICM. But there are two ways to show HU equity is linear to chips, by either random walk model (applied to low blinds case) or by random all in method(assume blinds are very large).

Random walk model says: if you are a bug at x between 0 and 1 moving randomly, what is the probability that you hit 0 before hit 1? The answer is 1-x.


[/ QUOTE ]

Staying with the bug model, I think the analogous thing giving rise to the "S-curve" result depends on the idea that the bug's likelihood of choosing a step left or a step right is not independent of his current position on [0,1] (which you have to assume to get the linear result). I'm not advocating either side, just pointing out how to view the original argument in the context you are putting forth.

eastbay

[rachelwxm]

eastbay,
I am going to let you wait a while. /images/graemlins/smile.gif As I said, I am still accumulating results, I have 40 HHs ready, but I am trying to expand them to 140 by the end of the week to make results more robust. I would post results in a few days once I get most of the HHs in the database. One thing sick about ps is that you have to request one HH every time. /images/graemlins/frown.gif

[rachelwxm]

Some preliminary results:
All right, here are some preliminary results for a small sample I have (all from ps turbo 15+1). 38 HHs total.

Some simple statistics:
ROI 35.4% (I know it’s little bit shaky after a very bad run last week)
ITM 47.4%
Realized equity 14.447 (assuming starting with 10 before rake)

Finishes:
1st 5
2nd 10
3rd 3
4th 8
5th 5
6th 2
7th 2
8th 1
9th 2

I have lots of 2nds finishes mostly because I am still not good at HU and lost some important coin flips HU.

So among 38 HHs, I have survived till bubble 26 times, ITM 18 times, HU 15 times.

To illustrate some of the results, I use HU as an example; here is the list of games

Finishes My chip Opponent chip Equity
1 2210 11290 29.94667
2 6990 6510 36.32
1 8970 4530 38.96
2 9285 4215 39.38
5
4
4
5
2 1510 11990 29.01333
3 18
2 6340 7160 35.45333
5
4
2 3400 10100 31.53333
1 12450 1050 43.6
7
9
2 4830 8670 33.44
4
3 18
4
2 8360 5140 38.14667
4
8
6
2 4740 8760 33.32
1 10735 2765 41.31333
6
1 7365 6135 36.82
3 18
4
2 7490 6010 36.98667
2 9220 4280 39.29333
5
7
4
9
5

Notice for 3rd finishes, I substitute 18 since that’s the payout for 9 people SNG.
So the average ICM Pnl for HU is the average of last column or 15.72. Notice it’s a lot higher than my final realized equity 14.45.

Similarly, I run this for bubble as well as ITM case and get ICM Pnl to be
Starting 10
Bubble 14.316
ITM 14.196
HU 15.724
Final realized 14.44

I have to point out due to small sample; these number has pretty big measuring error. But that is already pretty interesting to me.
Looks my ICMPnl is flattish around bubble and increases from 3 to 2 and lost some during final HU battle. Overall I did not increase any ICMPnl since bubble. /images/graemlins/frown.gif And I also kind of surprised that it jumped from ITM to HU quite a bit sort of offsetting HU lost?

I have not yet address any statistics regarding these numbers and the possible reasons for the difference.


Hope this will generate some thoughts and comments.