Datamining with PT

Despite my system being pretty new, opening more than 30 PP tables slows PT to a crawl. I checked my windows task manager and its running at ~100% when PT is importing hands. When I have 30 tables open it takes about 5 minutes to import. Any more tables and it takes longer and longer with each import. I have a 3GHz w/ 800 FSB Pentium 4 and 160 GB harddrive.

What do I need to do in order to up the tables I can mine to 50-60 tables?

My data base has about 500K hands and it will be growing rapidly. What is the best option when dealing with a large data base?

[Nomad84]

I only got PT up and running on my new machine today. I have been mining for a week now. I've just been manually copying the .hhf files every few hours into a new directory. I am curious how long it will take to import these into PT. I will try running PT along with Party later this week, but so far, this is working fine. I can easily mine 70+ tables at once with no slowdowns. I was mining every 3/6 full game, but I've started mining all of the 1/2 6max games as well since I plan to get some 6max experience in after I read WtO. Who knows what will happen when I start using PT also, but I plan to set the import interval pretty high. I'll probably import once/hour or so since there will be no real benefit to importing more frequently.

Does PT remain at 100% CPU usage? If it never drops back down, then you may be importing too often. It is normal for PT to go to 100% CPU usage when importing. This is part of the reason I decided to go dual core with my new computer. As for large databases, I assume you are using PostGRE SQL? I have never used it before since I was running WinME on my old machine. Now that I can use it, I need to figure out how :P

[SamIAm]

[ QUOTE ]
What do I need to do in order to up the tables I can mine to 50-60 tables?

[/ QUOTE ]I bet you're importing WAY too often. Try setting it to import every 30 minutes. Of course PT will max-out your processor while importing; that's the bottleneck. The question is whether it can import 50x30 hands in 30 minutes. (I bet it can.)
-Sam

[econ_tim]

How do you open more than 10 tables for datamining? I tried looking around but couldn't find anything.

[econ_tim]

[ QUOTE ]
How do you open more than 10 tables for datamining? I tried looking around but couldn't find anything.

[/ QUOTE ]

nevermind, i looked around some more and found an answer

http://forumserver.twoplustwo.com/showfl...part=1&vc=1

I've been importing every minute, but-to use your term-it has been bottlenecking and I end up with ~6 minutes between imports when data mining 30 tables. I'll try every thirty minutes. As long as the system doesn't get overwhelmed, I'm sure it will clear in well under 30 minutes.

Thanks for the advice!

[Nomad84]

[ QUOTE ]
I've been importing every minute

[/ QUOTE ]

Basically your computer is attempting to start the second import before it finishes the first one, as I understand it. If you are just datamining, there is no need to import any more than once ever 30 minutes or even every hour. Let us know if this fixes your problem. I'm betting it will.