Exadata X4-2 刷机至18.1.34版本遇到的问题

Posted 2023-05-15 石云华

客户有一套Exadata X4-2，计划从目前的11.2.3.3.1版本升级至18.1.34版本，但其中的一台计算节点在升级过程中遇到很多的故障，最终决定将这个已经升级失败的计算节点刷机至18.1.34版本。 

本文主要记录将Exadata X4-2刷机至18.1.34版本时，遇到的一些异常情况。 在刷机刚开始不久， 会进入到交互模式，要求输入网卡信息，具体如下：

问题1：
Ethernet interface (eth0,1,2,3) with optional VLAN id (ethX.YYYY) [eth0]:
IP Address of this host:
Netmask of this host:
2023-05-14 06:01:26 +0000 Try it again
Ethernet interface (eth0,1,2,3) with optional VLAN id (ethX.YYYY) [eth0]:
IP Address of this host: 10.16.152.100
Netmask of this host: 255.255.255.0
Default gateway: 10.16.152.1
[INFO ] Check eth0 for 1000Mb/s or 10000Mb/s speed and Full duplex mode
[INFO ] Root of distribution is set to /mnt/iso
[INFO ] File system type of boot partition: ext4
[INFO ] File system type: ext4
[INFO ] File system type for DOMU partitions: ext3
[INFO ] Terminal is linux
[INFO ] Checking if all cores are enabled and enabling them if not : no

刷机过程刚开始，就要求输入板载网卡的IP信息，如果不输入任何信息，则该过程就重复出现。 只能按照规划好的网络配置信息，填入eth0的网络信息，则后续的刷机工作才会正常进行。

Exadata 12.2.1.1.0 Highlights

突然发现，在中国农历新年的这几年里，Exadata 12.2版本已经发布了。

本起去docs.oracle.com上看看它的新特性，结果发现文档还没有更新：

下面是找到的一些资料，让我们来一睹为快吧：

Smart Analytics：

(1). In-Memory Formats in Columnar Flash Cache

• In-Memory formats used in Smart Columnar Flash Cache
• Enables vector processing on storage server during smart scans
  • Multiple column values evaluated in single instruction
• Faster decompression speed than Hybrid Columnar Compression
• Enables dictionary lookup and avoids processing unnecessary rows
• Smart Scan results sent back to database in In-Memory Columnar format
  • Reduces Database node CPU utilization
• In-memory performance seamlessly extended from DB node DRAM memory to 10x larger capacity flash in storage
  • Even bigger differentiation against all-flash arrays and other in-memory databases
• Supports Database 12.1.0.2 and Database 12.2.0.1

(2). Storage Index Set Membership

• Storage Index help skip unnecessary IOs
  • Currently contains up to 8 columns of min/max summary
  • Created automatically and kept in memory
• What about queries with low cardinality columns?
• Traditional min/max not good enough
• Database gathers stats and find that column has less than 400 distinct values
• Database requests storage to compute bloom filter
• Storage will compute distinct values and create a bloom filter
• Smart Scans check value \'CA\' against bloom filter and saves performing I/O
• Supports Database 12.1.0.2 and Database 12.2.0.1

(3). Smart Write Bursts and Temp IO in Flash Cache

• Write throughput of four flash cards has become greater than the write throughput of 12-disks
• When database write throughput exceeds throughput of disks, Smart Flash Cache intelligently caches writes
• When queries write a lot of temp IO, Smart Flash Cache intelligently caches temp IO
  • Writes to flash for temp spill reduces elapsed time
  • Reads from flash for temp reduces elapsed time further
• Smart Flash Cache prioritizes OLTP data and does not remove hot OLTP lines from the cache
• Smart flash wear management for large writes
• Supports Database 11.2.0.4, 12.1.0.2 and 12.2.0.1

(4). More Enhancements

• Storage Index will now store information about 24 columns instead of 8
  • Metadata space shared by set membership summary and min/max summary
  • Workload determines which summary gets stored
• Columnar Flash Cache for encrypted tablespaces
  • Data stays encrypted in columnar flash cache
  • Works seamlessly with In-Memory Formats in Flash Cache
  • Benefits Exadata Cloud Service as all data is encrypted in Oracle Public Cloud
• Supports Database 12.1.0.2 and Database 12.2.0.1
• Smart Scan Offload for Compressed Index Scan
• Smart Scan enhancements for XML
• Enhancements to XMLExists, XMLCast and XMLQuery
• Smart Scan offload enhancements for LOBs
• Extended to "LENGTH, SUBSTR, INSTRM CONCAT, LPAD, RPAD, LTRIM, RTRIM, LOWER, UPPER, NLS_LOWER, NLS_UPPER, NVL, REPLACE, REGEXP_INSTR, TO_CHAR"

   

Smart Consolidation:

(1). Up To 4,000 Pluggable Databases

• Exadata offers unique end-to-end resource management and consolidation capabilities
• Multitenant Option now allows greater than 252 Pluggable Databases within a single Oracle Multitenant Container Database
  • Up to thousands
• Exclusively available on Exadata and DB PaaS

(2). Additional Ethernet Connectivity

• Additional Ethernet card offered for customers that require additional connectivity
  • Dual port 10GE copper connectivity (part number 7100488)…OR
  • Dual port 10GE optical connectivity (part number X1109A-Z)
• Installed in PCIe slot 1 on the Oracle Exadata X6-2 database server
• Exadata SW automatically recognizes the new card and configures the two ports as eth6 and eth7 interfaces on the database server
• Additional ports can be used for any purpose

 

Smart OLTP:

(1). End-to-End IO Latency Capping

• Exadata Storage Server software detects and automatically eliminates IO latency outliers on disk and flash media
• On very rare occasions network outliers can deteriorate latency between database and storage servers
• Database 12.2 automatically redirects slow read I/O operations to another Exadata storage server
• Ensures end-to-end low latency for OLTP read I/Os

(2). Extended Distance Clusters

• Data Mirroring between two nearby sites
• InfiniBand connected for high performance
  • Limited to 100m optical cables in 2016 (best for fire cells)
• Implemented using 12.2 ASM Extended Diskgroups
  • Nested failure groups
• Compute nodes at each site read data local to that site
• Data is written to all sites
• Smart Scans scan across cells on both sites increasing throughput
  • Row filtering, column projection, storage index, and flash cache provide extreme performance
• Data Guard continues to be the recommended DR solution

(3). New Redo Log Write Metrics

• Redo Log Write response times are very critical to large scale OLTP systems
• New metrics to measure
  • Overall IO latency
  • Networking and other overhead
  • IOs serviced by Flash Log
  • Overall latency per storage server
• Improves visibility of redo log write performance

 

Smart Manageability:

(1). Super Fast and Robust Software Updates

• Oracle Public Cloud is the largest Exadata deployment with hundreds of Exadata Database Machine deployed
  • Each software release goes through thousands of upgrade cycles
  • All upgrade utilities get exercised thousands of times as well
  • Contributes to the robustness of the software release and utilities
• 5x speed up in Storage Server Software Update
  • Parallel firmware upgrades across components such as hard disks, flash, ILOM/BIOS, InfiniBand card
  • Reduced reboots for Software updates, use kexec where possible
  • Manage a Cloud instead of managing a single rack
  • Use single patchmgr utility to upgrade hundreds of racks
• Enable patchmgr to run from a non-Exadata system and run as low privileged user

(2). Miscellaneous Enhancements

• Storage Server Rescue Plan
  • Create a per server repository of customizations such as IORM plans, thresholds, etc.
  • Invoke the rescue plan upon restoration
• Make DNS, NTP, and other IP address changes online
• Storage server to storage server offload support for ASM Scoped-Security
  • Enable keys, called cell keys, on storage servers to authenticate offload operation
  • Applies to ASM resync, rebalance, resilver, rebuild and database high throughput write operations
• Quarantine Manager support for high throughput write
  • Automatically detects and quarantines any anomalous operation and uses alternate IO path
• IPv6 + Virtual machine + VLAN deployments
• Get graphs from Exawatcher
• Seamless customer service with Automatic Service Requests sending diagnostic attachments
• Oracle installer support for Quorum Disks in Database 12.2.0.1
• Manage Compute nodes using a RESTful service
  • ExaCli enabled for compute nodes in addition to storage cells