Pages

Wednesday, March 2, 2011

Benchmarking Java IO on Solaris.

So my boss has informed me that our application is due for a large volume increase.  To that end he is wanting some testing done to determine if the application can handle the increase.  The application performs full text searches on file ranging from 1k to 900 MB uncompressed.  With the volume increase the 900 MB may go to 4 GB.  Right now all the files are gzipped and they are searched real time when the user submits the query.   To bench mark the capabilities for uncompressing and searching a file, I need to come up with an average case and a worst case scenario.  My worst case scenario is when the file is on the file system but not in file cache, however, once I read the file then all subsequent tests are invalid because the file-cache will be delivering the file instead of the disk.

I pinged my SA about this and did some research.  I tried touching the file, and deleting the file and copying in new version, anything to get the file cache to invalidate the file.  Nothing worked.  Then in my research I found two techniques to achieve this.  The first is loading up the file cache with enough files that the only room left is smaller than the target file.   This seemed to be problematic, so I have not gone down this path.  I did find a jewel in the mount_ufs man page which leads me to believe that I can mount a dir and tell Solaris to keep these files out of file cache.  It is called forcedirectio.  I hope it works.

No comments:

Post a Comment