Differences

This shows you the differences between two versions of the page.

--- oracle_exadata_storage_config [2020/12/02 17:23] – andonovj
+++ oracle_exadata_storage_config [2020/12/04 13:02] (current) – andonovj
@@ Line 82: / Line 82: @@
 </Code>
-=====Management=====
+=====Storage Cell=====
 ====List Cell Processes====
@@ Line 346: / Line 346: @@
 CellCLI>
 </Code>
-===Flash Based===
-The flash based modules, can be examined as the hard disk based ones:
-==List the cell disks==
-<Code:bash>
---Basic
-CellCLI> list celldisk where disktype=flashdisk
-	 FD_00_qr01celadm01	 normal
-	 FD_01_qr01celadm01	 normal
-	 FD_02_qr01celadm01	 normal
-	 FD_03_qr01celadm01	 normal
---Detail
-CellCLI> list flashcache detail
-	 name:              	 qr01celadm01_FLASHCACHE
-	 cellDisk:          	 FD_02_qr01celadm01,FD_03_qr01celadm01,FD_01_qr01celadm01,FD_00_qr01celadm01
-	 creationTime:      	 2020-11-29T02:43:46+00:00
-	 degradedCelldisks:
-	 effectiveCacheSize:	 1.0625G
-	 id:                	 7fcc1eac-2214-40a1-9f27-eb988ec75340
-	 size:              	 1.0625G
-	 status:            	 normal
-CellCLI>
-</Code>
-Apart from the flashdisk, the exadata has also flashlog, to improve the redo log latency.
-==Flash log==
-<Code:bash>
---Detail
-CellCLI> list flashlog detail
-	 name:              	 qr01celadm01_FLASHLOG
-	 cellDisk:          	 FD_01_qr01celadm01,FD_03_qr01celadm01,FD_00_qr01celadm01,FD_02_qr01celadm01
-	 creationTime:      	 2020-11-29T02:43:44+00:00
-	 degradedCelldisks:
-	 effectiveSize:     	 256M
-	 efficiency:        	 100.0
-	 id:                	 1bb5db75-8136-4952-9d20-bec44a303a17
-	 size:              	 256M
-	 status:            	 normal
-CellCLI>
---Content
-CellCLI> list flashcachecontent detail
-...
-cachedKeepSize: 0
-cachedSize: 262144
-cachedWriteSize: 0
-columnarCacheSize: 0
-columnarKeepSize: 0
-dbID: 2080757153
-dbUniqueName: DBM
-hitCount: 11345
-missCount: 9
-objectNumber: 4294967294
-tableSpaceNumber: 0
-</Code>
-==Drop/Create Flash Cache==
-We can use the distributed CLI interface, provided with the exadata: "dcli" to run a command on multiple storage cells:
-<Code:bash>
---Drop Flash Cache:
-[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e drop flashcache
-qr01celadm01: Flash cache qr01celadm01_FLASHCACHE successfully dropped
-qr01celadm02: Flash cache qr01celadm02_FLASHCACHE successfully dropped
-qr01celadm03: Flash cache qr01celadm03_FLASHCACHE successfully dropped
-[celladmin@qr01celadm01 ~]$
---Create Flash Cache
-[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e create flashcache all
-qr01celadm01: Flash cache qr01celadm01_FLASHCACHE successfully created
-qr01celadm02: Flash cache qr01celadm02_FLASHCACHE successfully created
-qr01celadm03: Flash cache qr01celadm03_FLASHCACHE successfully created
-[celladmin@qr01celadm01 ~]$
-</Code>
 ====Stop / Start Cell Services====
@@ Line 508: / Line 425: @@
 <Code:bash|Drop disks in Storage Cell Server 1>
---Verify there isn't rebalancing:
+--Verify there is not rebalancing activity:
 SQL> select * from gv$asm_operation;
 no rows selected
@@ Line 868: / Line 785: @@
 RECO_QR01		36	  608	     608         572	     592        <- New one
 </Code>
+=====Flash Cache=====
+Flash cache in Exadata is a memory which stores often accessed data on storage level. With Flash cache you can achieve very good perfmance for reading.
+There are 3 types of a caches:
+===Write through===
+Using the write-through policy, data is written to the cache and the backing store location at the same time. The significance here is not the order in which it happens or whether it happens in parallel. The significance is that I/O completion is only confirmed once the data has been written to both places.
+==Advantage==
+Ensures fast retrieval while making sure the data is in the backing store and is not lost in case the cache is disrupted.
+==Disadvantage==
+Writing data will experience latency as you have to write to two places every time.
+==What is it good for?==
+The write-through policy is good for applications that write and then re-read data frequently. This will result in slightly higher write latency but low read latency. So, it’s ok to spend a bit longer writing once, but then benefit from reading frequently with low latency.
+===Write-around===
+Using the write-around policy, data is written only to the backing store without writing to the cache. So, I/O completion is confirmed as soon as the data is written to the backing store.
+==Advantage==
+Good for not flooding the cache with data that may not subsequently be re-read.
+==Disadvsntage==
+Reading recently written data will result in a cache miss (and so a higher latency) because the data can only be read from the slower backing store.
+==What is it good for?==
+The write-around policy is good for applications that don’t frequently re-read recently written data. This will result in lower write latency but higher read latency which is a acceptable trade-off for these scenarios.
+===Write-back===
+Using the write-back policy, data is written to the cache and Then I/O completion is confirmed. The data is then typically also written to the backing store in the background but the completion confirmation is not blocked on that.
+==Advantage==
+Low latency and high throughput for write-intensive applications.
+==Disadvantage==
+There is data availability risk because the cache could fail (and so suffer from data loss) before the data is persisted to the backing store. This result in the data being lost.
+==What is it good for?==
+The write-back policy is the best performer for mixed workloads as both read and write I/O have similar response time levels. In reality, you can add resiliency (e.g. by duplicating writes) to reduce the likelihood of data loss.
+====Management====
+The flash based modules, can be examined as the hard disk based ones:
+==List the cell disks==
+<Code:bash>
+--Basic
+CellCLI> list celldisk where disktype=flashdisk
+	 FD_00_qr01celadm01	 normal
+	 FD_01_qr01celadm01	 normal
+	 FD_02_qr01celadm01	 normal
+	 FD_03_qr01celadm01	 normal
+--Detail
+CellCLI> list flashcache detail
+	 name:              	 qr01celadm01_FLASHCACHE
+	 cellDisk:          	 FD_02_qr01celadm01,FD_03_qr01celadm01,FD_01_qr01celadm01,FD_00_qr01celadm01
+	 creationTime:      	 2020-11-29T02:43:46+00:00
+	 degradedCelldisks:
+	 effectiveCacheSize:	 1.0625G
+	 id:                	 7fcc1eac-2214-40a1-9f27-eb988ec75340
+	 size:              	 1.0625G
+	 status:            	 normal
+CellCLI>
+</Code>
+Apart from the flashdisk, the exadata has also flashlog, to improve the redo log latency.
+==Flash log==
+<Code:bash>
+--Detail
+CellCLI> list flashlog detail
+	 name:              	 qr01celadm01_FLASHLOG
+	 cellDisk:          	 FD_01_qr01celadm01,FD_03_qr01celadm01,FD_00_qr01celadm01,FD_02_qr01celadm01
+	 creationTime:      	 2020-11-29T02:43:44+00:00
+	 degradedCelldisks:
+	 effectiveSize:     	 256M
+	 efficiency:        	 100.0
+	 id:                	 1bb5db75-8136-4952-9d20-bec44a303a17
+	 size:              	 256M
+	 status:            	 normal
+CellCLI>
+--Content
+CellCLI> list flashcachecontent detail
+...
+cachedKeepSize: 0
+cachedSize: 262144
+cachedWriteSize: 0
+columnarCacheSize: 0
+columnarKeepSize: 0
+dbID: 2080757153
+dbUniqueName: DBM
+hitCount: 11345
+missCount: 9
+objectNumber: 4294967294
+tableSpaceNumber: 0
+</Code>
+==Drop/Create Flash Cache==
+We can use the distributed CLI interface, provided with the exadata: "dcli" to run a command on multiple storage cells:
+<Code:bash>
+--Drop Flash Cache:
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e drop flashcache
+qr01celadm01: Flash cache qr01celadm01_FLASHCACHE successfully dropped
+qr01celadm02: Flash cache qr01celadm02_FLASHCACHE successfully dropped
+qr01celadm03: Flash cache qr01celadm03_FLASHCACHE successfully dropped
+[celladmin@qr01celadm01 ~]$
+--Create Flash Cache
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e create flashcache all
+qr01celadm01: Flash cache qr01celadm01_FLASHCACHE successfully created
+qr01celadm02: Flash cache qr01celadm02_FLASHCACHE successfully created
+qr01celadm03: Flash cache qr01celadm03_FLASHCACHE successfully created
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Drop Flash Cache===
+We can drop the flash cache only after, we have to stop the cluster.
+<Code:bash|Drop Cache>
+--Stop the cluster:
+[root@qr01dbadm01 ~]# /u01/app/12.1.0.2/grid/bin/crsctl stop cluster -all
+CRS-2673: Attempting to stop 'ora.crsd' on 'qr01dbadm01'
+CRS-2673: Attempting to stop 'ora.crsd' on 'qr01dbadm02'
+CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'qr01dbadm01'
+CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'qr01dbadm01'
+CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'qr01dbadm02'
+CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'qr01dbadm01'
+CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'qr01dbadm01'
+CRS-2673: Attempting to stop 'ora.DBFS_DG.dg' on 'qr01dbadm02'
+CRS-2673: Attempting to stop 'ora.dbm.db' on 'qr01dbadm02'
+--Drop Flash cache
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e drop flashcache
+qr01celadm01: Flash cache qr01celadm01_FLASHCACHE successfully dropped
+qr01celadm02: Flash cache qr01celadm02_FLASHCACHE successfully dropped
+qr01celadm03: Flash cache qr01celadm03_FLASHCACHE successfully dropped
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Shutdown Services===
+<Code:bash|Shutdown Services>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e alter cell shutdown services cellsrv
+qr01celadm01:
+qr01celadm01: Stopping CELLSRV services...
+qr01celadm01: The SHUTDOWN of CELLSRV services was successful.
+qr01celadm02:
+qr01celadm02: Stopping CELLSRV services...
+qr01celadm02: The SHUTDOWN of CELLSRV services was successful.
+qr01celadm03:
+qr01celadm03: Stopping CELLSRV services...
+qr01celadm03: The SHUTDOWN of CELLSRV services was successful.
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Enable Flash Through Cache===
+<Code:bash|Enable Flash Through Cache>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e alter cell flashCacheMode = WriteBack
+qr01celadm01: Cell qr01celadm01 successfully altered
+qr01celadm02: Cell qr01celadm02 successfully altered
+qr01celadm03: Cell qr01celadm03 successfully altered
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Restart Flash Cache Services===
+<Code:bash|Restart Flash Cache services>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e alter cell startup services cellsrv
+qr01celadm01:
+qr01celadm01: Starting CELLSRV services...
+qr01celadm01: The STARTUP of CELLSRV services was successful.
+qr01celadm02:
+qr01celadm02: Starting CELLSRV services...
+qr01celadm02: The STARTUP of CELLSRV services was successful.
+qr01celadm03:
+qr01celadm03: Starting CELLSRV services...
+qr01celadm03: The STARTUP of CELLSRV services was successful.
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Flush Caches===
+<Code:bash|Flush Caches>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e alter flashcache all flush
+qr01celadm01: Flash cache qr01celadm01_FLASHCACHE altered successfully
+qr01celadm02: Flash cache qr01celadm02_FLASHCACHE altered successfully
+qr01celadm03: Flash cache qr01celadm03_FLASHCACHE altered successfully
+[celladmin@qr01celadm01 ~]$
+</Code>
+====Monitoring====
+We can monitor the flash cache, using several commands
+===Determine the Flash Cache Type===
+<Code:bash|Determine Type>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e list cell attributes flashCacheMode
+qr01celadm01: WriteBack
+qr01celadm02: WriteBack
+qr01celadm03: WriteBack
+[celladmin@qr01celadm01 ~]$
+</Code>
+===Determine the amount of Dirty Data===
+<Code:bash|Determine the amount of dirty Data>
+[celladmin@qr01celadm01 ~]$ dcli -c qr01celadm01,qr01celadm02,qr01celadm03 cellcli -e list metriccurrent FC_BY_DIRTY
+qr01celadm01: FC_BY_DIRTY	 FLASHCACHE	 0.000 MB
+qr01celadm02: FC_BY_DIRTY	 FLASHCACHE	 0.000 MB
+qr01celadm03: FC_BY_DIRTY	 FLASHCACHE	 0.000 MB
+[celladmin@qr01celadm01 ~]$
+</Code>