vdbench 50407.pdf

文件系统测试工具官方文档，一般不好下载，有需要的可以收藏着Vdbench 5 04.07TABLE OF CONTENTSContentsVDBENCHUSERS GUIDEVERS|ON:5,04.07.AUTHOR: HENK VANDENBERGHVDBENCH: DISK IO WORKLOAD GENERATOR1.1 ntroduction························4·····································4············9991.2 Objective…..1.3 Terminology.oooeeocene101.4 Installing Vdbench......。101.5 How to start Vdbench:111.6 Execution parameter overview.1. 6.1 Execution parameters1.7 Parameter file(s).…,.,..,.,.,..,,.,,.,,,,141.7.1 Variable substitution147.2 Multi- host parameter replication…………………………1.7.3 include-parmfile ....157.4 General Parameters: Overview.151. 7.5 Host Definition(HD)Parameter overview171.76 Replay group(RG) Parameter overview……………,181.7.7 Storage Definition(SD) Parameter OverviewD音4音垂.191.7.8 File system Definition1.7.9 Workload definition(WD) Parameter Overview……………201.7.10 File system Workload Definition(FWD) Parameter Overview......207.11 Run Definition(RD)Parameter Overview(For raw lO testing).........2118 Execution parameter detail.,……231. 8. -fxxx: Workload Parameter File(s))41. 8.2-OXXX': Output Directory .............24Copyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04.071.8.3′v: Activate data validation..………………,…………………241. 8.4-j: Activate Data Validation and Journaling .....251.8.5"-sl; Simulate execution....………251.8.6-k. Kstat Statistics on Console251. 8.7-m nn Multi jvM Execution261. 8. -t'or-tf: Sample Vdbench execution·······.·······…261.8.9 -e nn'Override elapsed time8.10 -i nn Override report interval time.271.8.1I -wnn'Override warmup time271.9 Vdbench utility functions .1.9. 1. /bench sds: Generate Vdbench Sd parameters281.9.2. /vdbench jstack: Display java exccution stacks of active Vdbench runs.... 281.9.3. /vdbench rsh: Vdbench rsh daemon............................281.9.4 /vdbench print: Print any block on any lun or file281.9.5. /vdbench edit: Simple full screen editor, or back to the future.............. 291.9.6. /vdbench compare: Compare Vdbench test results垂垂。垂垂D非。垂。·。垂垂·。垂垂D垂。垂299.7 /vdbench parse: Parse Vdbench flatfile.....................29198/ bench csim: Compression simulator…………………291.9.9/ vdbench dsim: Dedup simulator……………………………291.9.10 /vdbench printjournal: print(subset of).jnl file......301.9.11./vdbench showlba: Visualize data access pattern…301.10 General parameter detail321.10.1includeparmfile1.10.2 'data errors=xxX Terminate After data Validation or lo errors ..........................321. 10.3 'startcmd='and 'el331. 10.4 pattern =: Data Pattern to be used1.10.5‘ corpration=n’: Set compression for data patterns.………………151.10.6port-nnn∴: Specify port number for Java sockets.……………….3510.7 'create anchors-yes': Create anchor parent directory361. 10.8 report=: Generate extra SD reports3610.9 histogram=: set bucket count and bucket size for response time histograms...361. 10.10 formatxfersize nnnn垂番垂…3710.11 monitor=. External control of vdbench termination ..............................................371.10. 11.1 Shutdown via temporary file1.10.11.2 Shutdown via monitor parameter71.10. 12 'messagescan- suppress /var/xxx/messages scan:···3810.13 timeout=(nn, script)1.11 Replay group (rg parameter detail..391.12 Host Definition parameter detail.。·。。垂401.12.1 'hd=host label?…401.12.2‘ system= system name…….….….…..….….….….….…,-401. 12.3 master-system name/IP address401.124jVms=nnn,........1.12.5bench=/vdbench/dir/nCopyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04.071. 12.6 shell-rsh ssh vdbench4112.7user-XXXX垂看看垂·垂音非··垂.看垂看垂非垂音。.垂垂·非垂,音垂·垂垂垂看垂。垂垂非·音,音垂垂音垂411.12.8‘ mount-xxx1.13 Storage Definition parameter detail。4313.1 'sd-name Storage Definition Name431.13.2 lun=lun name. lun or File Name1.13.3host=name’∴………141. 13. count=(nn, mm)1.13. 5 size=nn: Size of lun or file451.13.6' range=(rmin,max)}: Limit Seek range…………………4513.7 threads=nn: Maximum Number of Concurrent outstanding i/os461. 13. 8 hitarea=nn Storage Size for Cache Hits.....................4613.9 journal- name! Directory Name for Journal File…………461. 13. offset=: Dont start at byte zero of a LuN4713. 11align-: Determine lba boundary for random seeks471.13.12‘ openflags=’: control over open and close of luns or files……4813 13 streams-: Independent sequential streams1. 14 Workload Definition parameter detail501. 14. wd=name. Workload definition name ..................................................................511. 14.2host=host label14.3 'sd=name. SD names used in Workload…511.14.4 rapt-nnl: Read Percentage.………..………1. 14.5 rhpct-nn' and'whpct-nn: Read and Write Hit Percentage1. 14.6 xfersize= nn Data Transfer Size521. 14.7 'skew=nn Percentage skew31. 14.8 seekpct=nn: Percentage of Random Seeks5314.9 stride=(min, max ) Skip-sequential I/O41.14.10' range=nn': Limit Seek Range………….…....….…....…...14.11‘ lorate= Workload specific Io rate.……551. 14.12 priority=Workload specific IO priority1.15 Run Definition for raw lyo parameter detail1.15.1 rd=name. Run Name. ..............................................................................................57.15.2dWd-. Names oof workloads to run715.3‘sd=xxx’………………5715. 4 lorate=nn. One or more l o rates5715.5 'curve=nn Define Data points for Curve1.15.6' elapsed-ni: Elapsed Time……………………,…………….5815.7 interval=nn: Reporting Interval.·591. 15.8 warmup=nn: Warmup period ...........59.15.9 maxdata=': stop after nnn bytes.……………….…..2591. 15.10 'distribution=xxx'. vo arrival time distribution .....................601.15. 11 pause=nn: Sleep nn Seconds................605.12 Workload parameter specification in a Run Definition.……….601. 15.12. sd=xxx Specify sds to use61Copyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04071. 12.2 (for)xfersize-nn: Create for' Loop Using Different Transfer Sizes....611. 15. 12.3 for)threads-nn: Create or' Loop Using Different Thread Counts....621. 15. 12.4 (forrdpct nn Create for' Loop Using Different Read Percentages .........621. 12.5 (for)=nn: Create for' Loop Using Different Read Hit Percentages ......621. 15.12.6 ' (for)whpct=nn Create or' Loop Using Different Write Hit Percentages.6315.12. 7 (for )seekpct-nn': Create for'Loop Using Different Seek Percentages...6315.12.8 (forhitarea=nn Create for' Loop Using Different Hit Area Sizes.....631. 15.12.9 (for)=nn: Create for' Loop Using Different compression ratios. 641. 15. 12.10 Order of execution using ' forxxx' Parameters641.16 Hot banding and SD concatenation:1.17 'Hotband-(min, max): Create a skewed workload over a Limited Seek Range.oo, 67l.18 Data Deduplication:,,………,,:;;;681. 18.1 Unique blocks4中·垂垂音·辛音。··.·..·······6918.2 Duplicate blocks.....................691. 18.4 Duplicate blocks and duplicate'sets'1.18.3 Dedup flipflop…69,看垂垂。D91. 18.5 Duplicate ' hot sets'.·::·····:1. 18.6 Bench xfersize= limitations1.18.7 Use of Data Validation code7218.8 Swat/V bench Replay with dedup…721. 18.9 offset= and align= parameter and dedup721. 19 Data Validation and Journaling1.20 Report files…771. 21 Vdbench wrappers'or how to monitor Vdbench791.22 Swat Vdbench Trace Replay.....801.23 Complete Swat Vdbench replay Example.1.24 File system testing……............….831. 24.1 Directory and file names·垂·,·垂841. 24.2 File system sample parameter file841.25 File System Definition(FSD) parameter overview:.861.26 Filesystem Workload Definition(FwD) parameter overview:∷………2861. 27 Run Definition(RD) parameters for file systems, overview ............871. 28 File System Definition(FSD) parameter detail871. 28.fsd- name: Fllesystem Storage Definition name881.28.2 anchor=: Directory anchor非垂D垂1. 28.3 ' count=(nn, mm)' Replicate parameters8Copyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04.071. 28.4 shared=, FSD sharing………881.28.5 width-’: Horizontal directory count..…....….891.28.6 depth-:Ⅴ ertical directory count…….…..….………………………891.28.7‘ files=’: File count.··91.28.8‘ Sizes=1: File sizes….891.28.9^ openflags-': Optional file systen^open' parameters…….….……….901. 28.10 totalsize=: Create files up to a specific total file size901. 28 11 workingsetsize=nn'or'wss=nn901.29 File system Workload definition1.29.1‘fwd-name’: File system Workload Definition name………….….….…………….…..9129.2tsd=: which File System Definitions to use·······1. 29.3'filcio=': random or sequential I/O9129.4‘ rapt-': specify read percentage.…,…,……,…………..,911.29.5‘ stopafter-’: how much I/O?911.29.6‘ fileselect’: which files to select..····921. 29.7 xfersizes=: data transfer sizes for read and writes ................................................9329.8 operation-': which file system operation to execute…………931. 29.9'skew-: which percentage of the total workload931. 29.10 threads=: how many concurrent operations for this workload941.30 Run Definition (RD) parameters for file system testing, detail...951.30.1‘fwd=: which File system Workload Definitions to use.…...…951.30.2fwdrate=: how many operations per second1.30. format=: pre-format the directory and file structure..961.30. operations=': which file system operations to run1.30.5‘ foroperations=’: create‘for' loop using different operations∴981.30. fordepth=: create"for'loop using different directory depths1.30.7 forwidth-’: create‘for' loop using different directory widths………….,991.30.8 forfiles’: create'for' loop using different amount of files……………….991.30.9'forsizes=: create for'loop using different file of sizes......991.30. 10 'fortotal=: createfor'loop using different total file sizes .........1001.30. 11forwss=: for'loop using working set sizes……1001.31 Multi Threading and file system testing….….…,1011.32 Operations counts vs. nfsstat countsn1021.3 Report file examples…..….,.…,1031.33. 1 summary. html103133.2 totals. html: run totals1041.33.3 totals optional. html running totals..........10041.33. 4 summary. html for file system testing1041.33.5 logfile. htm1.......….……105133.6 kstat html..1051.33.7 histogram1071.33.8nfs3/4.html…1071.33 9 flatfile. html·:·+··:·+··+·+··+··108Copyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04.071.33.10skew.html….…1091.34 Sample parameter files……n1091.34.2 Example 2: Single run, two raw disk, two workloads,...........1101.34. 1 Example 1: Single run, one raw disk1101. 34.3 Example 3: Two runs, two raw disks, two workloads1101. 34.4 Example 4: Complex run, curves with different transfer sizes..........1111.34.5 Example 5: Multi-host....l111. 34.6 Example 6 Swat 1O trace replay·:.···1.34.7 Example 7: File system test1121.35 Permanently override Java socket port numbers1.36 Java Runtime Environment.mmossmmosomommsmmososss 1131.36.1 Java and garbage Collection.…,,…1131.36.2 Java and unable to create new native thread1141.37 Solaris.1.38 How to compile for other platforms·●。●···。···························a··················1141.39 Vdbench flatfile selective parsingb鲁●·。香e鲁●·。●。D●41162 VDBENCH WORKLOAD COMPARE1173 VDBENCH SD PARAMETER GENERATION TOOL…118Copyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 0407Vdbench Disk l /o Workload GeneratorGetting started with vdbenche Installation InstructionsExecutione Sample parameter fileslntroductionVdbench is a disk lo workload generator to be used for testing and benchmarking of existingand future storage productsVdbench is written in Java with the objective of supporting Oracle heterogeneous attachment. Atthis time to has been tested on Solaris Sparc and x86, All flavors of Windows, HP/UX, AlXLinux, Mac Os X, zLinux and RaspBerry PiNote: one or more of these platforms may not be available for this latest release, this due to thefact that a proper system for a Java NI C compile may not have been available at the time ofdistribution. In this case there will be readme. txt file in the Os specific subdirectory, askinfor a volunteer to do a small java nic compileSee how to compile for other platforms1.2 ObjectiveThe objective of v bench is to generate a wide variety of controlled storage yO workloadsallowing control over workload parameters such as IO rate, LUN or file sizes, transfer sizes,thread count volume count volume skew read / write ratios read and write cache hitpercentages, and random or sequential workloads. This applies to both raw disks and file systemfiles and is integrated with a detailed performance reporting mechanism eliminating the need forthe Solaris command iostat or equivalent performance reporting tools. I O performance reportsare web accessible and are linked using html. Just point your browser to the summary. html filein the Vdbench output directoryThere is no requirement for vdbench to run as root as long as the user has read/ write access forthe target disk(s) or file system(s)and for the output-reporting directoryNon-performance related functionality includes Data Validation with Vdbench keeping track ofwhat data is written where, allowing validation after either a controlled or uncontrolledshutdownCopyright C 2000, 2018, Oracle and/or its affiliates. All rights reservedVdbench 5 04071. 3 TerminologyExecution parameters control the overall execution of Vdbench and control things likeparameter file name and target output directory nameRaw lo workload parameters describe the storage configuration to be used and theworkload to be generated. The parameters include General, Host Definition(HD), ReplayGroup(ro), storage Definition(D, workload Definition(WD)and run Definition(RD)and must always be entered in the order in which they are listed here A Run is the executionof one workload requested by a run Definition. Multiple runs can be requested within oneRun definitionFile system Workload parameters describe the file system configuration to be used and theworkload to be generated. The parameters include General, Host Definition(D), fileSystem Definition(FSD), File system Workload Definition(FWD) and Run Definition( RD) and must always be entered in the order in which they are listed here. A Run is theexecution of one workload requested by a run definition. Multiple runs can be requestedwithin one run definitionReplay: This Vdbench function will replay the lO workload traced with and processed bythe Sun StorageTek Workload Analysis Tool (Swat)Master and slave: Vabench runs as two or more Java Virtual Machines (JVMs). The JVMthat you start is the master. The master takes care of the parsing of all the parameters, itdetermines which workloads should run, and then will also do all the reporting. The actualworkload is executed by one or more Slaves. A Slave can run on the host where the masterwas started, or it can run on any remote host as defined in the parameter file. See also'-mnn. Multi vm executionData validation: Though the main objective of vdbench has always been to execute storageI/O workloads, Vdbench also is very good at identifying data corruptions on your storageJournaling: A combination of Data Validation and journaling allows you to identify datacorruption issues across executions of Vdbench. See Data Validation and JournalingLBA, or lba For Vdbench this never means Logical Block Address, it is Logical ByteAddress. 1 6 years ago i decided that i did not want to have to worry about disk sector sizechanges and it is clear that this was the right decision1.4 Installing vdbenchVabench is packaged as a zip file. Unzip the file and you re ready to goThe zip file contains everything you need for both windows and Unix systemsNote: one or more of the many supported platforms may not be available for this latest release,this due to the fact that a proper system for a Java JNI C compile may not have been available atthe time of distribution. In this case there will be a readme. txt file in the os specificsubdirectory, asking for a volunteer to do a small Java JNI C compileCopyright C 2000, 2018, Oracle and/or its affiliates. All rights reserved