From:Steve Adams
Date:18-Oct-2000 22:49
Subject:   Mount options for VxFS

Oracle explicitly requests synchronous I/O whenever it opens its datafiles. Therefore, it is the convosync mount option that counts, rather than the mincache one.

The best source of information on the relative performance impact of the direct I/O versus buffered I/O decision (and the raw versus filesystem decision for that matter) is of course the Veritas web site itself. They consistently claim to get "near raw disk performance" when using direct I/O. Here is a quote from the conclusion of one of their white papers ...

"The VxFS file system with direct I/O offers I/O characteristics similar to raw I/O on raw partitions. Our test results confirmed the performance similarity between raw I/O and VxFS’s direct I/O. With proper database tuning, VxFS can deliver performance close to what raw I/O can offer ... For Oracle databases, we therefore recommend using VxFS file systems mounted with the direct I/O option (convosync=direct) ..."

For another example, have a look at the graph on page 2 of their white paper VERITAS Database Edition 2.1 for Oracle - Performance Brief – OLTP Comparison on Solaris 7. It clearly shows that buffered I/O only gives between 74% and 31% of raw performance, depending on the Oracle buffer cache size, whereas direct I/O consistently gives about 60% of raw disk performance. Or to put that in other words, if your Oracle buffer cache is big enough and if your workload is heavy enough, then direct I/O can give twice the performance of buffered I/O.

By the way, don't be overly impressed with the numbers given for Cached Quick I/O. If you read the rest of the white paper you will see that raw still wins if you allow Oracle to manage the memory itself.

I've just inherited a (Sun) system where the "physical" decisions are deemed outside of the realm of the DBA. For example, comments like "we not you will decide whether you need raw devices or not" are the norm.

So now its all based on vxfs file systems and I've noticed that all them are mounted with "mincache=closesync". Whilst I've not done any benchmarking, all the literature I've perused makes me very uneasy about this, that is, everything points toward this option haviong a significant performance impact. I would have thought "mincache=direct,convosync=direct" (ie direct io) would have been a far better option, but before I head off into battle, I'm curious to hear your thoughts.