Oracle Agile doesn’t follow Oracle best practices

Uncategorized, ,

So right now I’m doing an upgrade of one of our test systems from Agile Advantage SP4 on Windows to Agile 9.3.0.1 on Linux. It’s quite the involved process and I’ll post more at length about it later. However, two things I just finished fixing that saddens me:

Oracle doesn’t follow Oracle’s own best practices with the seeded Agile 9.3 database. I have no idea why they did this. It’s not because this best practice is new – it’s been the recommendation since I learned it back in 1997 with Oracle 7.3.

Agile 9.3.0.1 lays down a database with 4 redo log file groups, each with 1 200MB member. Oracle best practices with redo logs is 3 redo log file groups each with 2 members. Depending on the system I/O characteristics you may add more redo log groups and the size is highly dependent on the I/O characteristics. All Oracle databases should at least have 2 redo log groups each with 2 members. So instead of the minimum 2×2 or best practice 3×2 configuration, Oracle provides one of their products with a 4×1 configuration which is just asking for issues.

On a related note, the Oracle Agile 9.3.0.1 database also came with an UNDO tablespace that’s sized to autoextend automatically until it fills up the file system. Best practice is to NEVER set an UNDO tablespace to autoextend because a poorly written query can possibly use all the disk space on the system.

Seriously Oracle, what were you thinking?

A support story with EMC and Oracle

Uncategorized, ,

Like I’ve said before, my three main job duties involve Oracle, Linux and VMware.  This also tends to extend to the hardware this all runs on.  I sometimes need to open support tickets on these products and recently had two support experiences I wish to share.  In both cases, my initial support request didn’t get the attention it needed, but in both cases, support managers went above and beyond to give my client and I exemplary support.

In these days of off-shoring, near-shoring, and other other cost avoidance initiatives, I’m reminded that it’s the performance of support individuals who deal with end users that most greatly influence customer opinions.

The first experience involves EMC support with our SAN.  We’ve recently expanded our primary server room.  Part of that expansion involved adding additional UPS resources and an UPS bypass.  For awhile, we were running our SAN on non-redundant power.  When we provided redundant power, one of the power supplies in our SAN failed and I opened a support ticket with EMC.  Somehow the ticket didn’t get routed or assigned correctly and even after placing two additional support calls, here I was two days in and still with a failed power supply.  Not a comfortable feeling.  At that point I reached out to our sales rep (Bryan Coston of PBS Now) at the end of business day who in turn reached out to EMC’s area support representative ( Scott Marnell).  Even though I hadn’t even contacted them till close of business, I received calls and emails from them that night and had the option to get the defective part replaced that night.  It’s that sort of above the call of duty support that makes me very happy to have EMC products in my environment.  These two individuals (and perhaps others behind the scenes) in a couple of hours turned around an unpleasant support situation into a support success.  Thank you PBSNOW and EMC.

The second support experience involves Oracle Support with Oracle Agile software.  I’m currently migrating our Agile system to the latest versions in a test environment.  I’ve run into some issues I’ve been unable to resolve on my own, and opened a couple of technical support tickets with Oracle.  Both tickets ended up being assigned to same analyst.  On one ticket involving data migration errors, I received no updates for a whole week.  On the other ticket, the analyst and I got into a disagreement regarding Oracle’s support of EXPort and IMPort utilities with Oracle Database 11g.  After getting frustrated and not receiving any updates to either issue, including my written request that the requests by duty managed, I called in and requested a call from a duty manager.  Within an hour I had the duty manager on the phone.  Not only did he apologize for the delays and reassign my support tickets to a senior level support analyst, the duty manager (I’m sorry, I didn’t write down his name) also spoke with me at length regarding my experiences with the upgrade and asking specific questions on what could be done to make the upgrade documentation and experience better.  I’ll be posting my experiences and notes from the upgrade once it’s complete.  The analyst that the duty manager assigned to my tickets,  Mehdi Hamzezadeh, was fantastic.  Not only did he take ownership of my support tickets and resolved both issues in a matter of hours, it turns out he wrote many of the data cleanup scripts that Oracle Support provides to their end customers.  I’ve very grateful to the duty manager for assigning Medhi to my tickets and to Medhi for his quick and complete assistance.


	

	
	


							


	

		
				

			vExpert 2010
		

		
		

			Uncategorized, 					

	


		

		
VMware has an advocacy recognition program called VMware vExpert.  The program’s goal is to recognize individuals who significantly contribute to the community of VMware users.  Such individuals include authors, bloggers, VMUG leaders, tool builders, and  other IT  professionals.


I waited until the last minute to submit my application because I debated if I was even qualified to apply.  This blog is very new.  I’m not the knowledge expert on VMware products like many of the other vExperts.  Heck, I don’t even have VCP (VMware Certified Professional) credentials. I’m one of the IT professionals out there buying the books that many of the other vExperts publish!  The VMware vExpert judges deemed me worthy of the recognition and I’m thrilled and proud to have been awarded it on Friday.


Although I’m not the pure VMware subject matter expert many are, I have and will continue to contribute my subject matter expertise with Oracle products on Linux under VMware.  My main client made the switch to running our Tier 1 production Oracle systems under VMware about 2 years ago.  I believe it was one of the best decisions we ever made, from both a technical and financial perspective.


Since then in addition to my main duties as Sr. Oracle Applications DBA and Sr. Linux Administrator, I’ve been learning and leveraging many of the features VMware virtualization buys us.  I’ve done many reference calls both for Oracle (my main client was one of the first companies in the world to run Oracle 11g database in Production), EMC (we utilize many EMC products in our environment) and VMware and will continue to in the future

	

	
	


							


	

		
				

			Oracle licensing under VMware and how to get the best bang for your buck
		

		
		

			Uncategorized, , , 					

	


		

		
I’ve been meaning to write an article on how Oracle’s licensing works with regards to VMware and how to minimize costs, but Oracle Storage Guy (Jeff Browning) said it better in this blog post ( http://oraclestorageguy.typepad.com/oraclestorageguy/2010/05/oracle-license-costs-on-the-vmware-vsphere-platform.html ) than I could have said it.  Go read his blog post in another tab and then continue on..


Back?  Good.


So a real world example.  My main vSphere cluster is a 16 blade Dell Cluster.  11 of those blades are Dell M600 blades each with I2 Intel Xeon E5430 processors (those are pre-Nahelm for those curious) and 5 of those blades are Dell M610 blades each with 2 Intel Xeon 5550 (Nahelm).  In both cases, each blade has 2 sockets, with each core having 4 cores.  With the Xeon 5550 processors, they also have 2 logical (hyperthreading) cores per physical core, giving me a total of 16 logical processors per M610 blade.  Oracle licensing does not count logical cores, only physical ones.


Rather than paying to license 8 cores on 16 blades (which at list rates would work out to 8 * 16 * 0.5 * $47,500 ) for $3,040,000 or just over $3 MILLION DOLLARS in Oracle licensing, we have two clusters in vCenter.  One of these contains 8 M600s and 3 M610s and basically contains everything in our cluster with the exception of our Oracle products licensed by processor.  The other cluster is 2 M610 nodes and it holds all our Oracle databases and other Oracle products that are licensed by processor.  That works out to (8 * 2 * 0.5 * $47,500) $380,000 or the same performance for about 1/10 of the price.


Now, you may be asking how those two blades are handling the load – I’m writing this blog entry during the middle of our production day.  I’m running 20 Oracle database servers (one instance per VM server) combined on the two blades.


Blade #1 – 1044 MHz CPU utilized, 25897MB RAM utilized


Blade #2 – 4448 MHz CPU utilized, 27093MB RAM utilized


I didn’t make a typo.  I’m running 20 database servers essentially off of 2 CPUs worth of processing power (each Xeon 5550 is 2.67 MHz) and I’ve got 32 CPUs of processing power in the two node cluster (4 sockets, each with 4 physical cores, each physical core with 2 logical cores), so that’s not going to be a bottleneck at any time soon.


On the RAM side, each M610 has 48G of RAM and right now I’m utilizing about 48G of RAM.  Each M610 can handle up to 192G of RAM, but we went with 12 4G sticks due to that being the current cost sweet spot.  Again, not a bottleneck.


So like Jeff said, utilization on Oracle database servers is typically low.  If I wasn’t virtualized at all, I’d have to have 20 physical servers and license all those processors for Oracle Enterprise Edition.  If I allowed my Oracle VMs into my regular VMware cluster, I’d have all the advantages of VMware (HA (high availability), vMotion, Storage vMotion, etc) but I’d have to pay 10x the cost in Oracle licenses.  Instead I’ve got a separate VMware cluster only licensing what I need and getting all the advantages of VMware.


For those curious about what we’re doing with all those instances, some are development / support environments, but we’re running our production Oracle E-Business Suite, Oracle Hyperion, Oracle Agile, and various other smaller load systems on two nodes supporting hundreds of end users.


What can I say? VMware rocks.

	

	
	


							


	

		
				

			Upgradation
		

		
		

			Uncategorized, 					

	


		

		
If you’ve been an Oracle DBA for any length of time, you’ve heard the term upgradation.  But what is an upgradation?


Is it an upgrade?  Is it a migration?  What the heck’s the difference between an upgrade and a migration?


If you’re switching operating system platforms (Windows to Linux for example), you’re migrating


If you’re switching database vendors (MS SQL Server to Oracle for example), you’re migrating


If you’re moving from Oracle 9i to Oracle 10g, you’re upgrading


If you’re moving from Oracle 11gR1 to Oracle 11gR2, you’re upgrading


If you’re moving from Oracle 11gR1 on Windows to Oracle 11gR2 on Linux, you’re performing both an upgrade and a migration.


So what is an upgradation?  Well damn if that isn’t a good question, because UPGRADATION ISN’T A WORD OR A REAL TECHNICAL TERM. In my experience, it tends to be used by people very new to the Oracle eco-system and by people who don’t have English (or American) as their primary language.


I’ve been at multiple clients where if upgradation is listed on a resume, the resume is no longer considered.  Unless a prospective candidate is otherwise amazing, I’m typically the same way.  It’s the same thing as having a misspelling on your resume.


So, make sure you don’t use upgradation on a resume and learn the difference between a migration and an upgrade.

	

	
	


							


	

		
				

			Ironman 2 and Oracle
		

		
		

			Uncategorized, 					

	


		

		
So last night I got away from the keyboard and checked out Ironman 2 with a few friends, including one guy who works at Oracle.  It was a fun sequel, but there were a few Oracle related things I especially liked.


First, about 5 mins in, Larry Ellison makes a cameo trying to get a lunch meeting with Tony Stark.  Nice acting Larry!


Later on in the movie, Tony Stark is researching information and his computer system (J.A.R.V.I.S) intones something about accessing the Oracle Grid.  Then later on in the movie Ivan Vanko hacks into the Oracle Grid to gain control of the Ironman-style drones.  I would only expect the Oracle Grid to be running Oracle Unbreakable Linux, and it appeared Ivan Vanko hacked into the grid in a minute or two… hopefully I’m missing something because I’m not sure this is the message Oracle really wants to pass on to prospective customers…


Finally, at the Stark Expo, you can see one of the stylistic Oracle buildings.


There was also a smattering of various Sun and Dell hardware.


I’m sad but not surprised that Enterprise Manager 11g didn’t make it into the movie, but with the teaser about Thor’s hammer at the end of the credits, maybe Larry is saving that for Ironman 3?

	

	
	


							


	

		
				

			Oracle VirtualBox 3.2 is now out
		

		
		

			Uncategorized, 					

	


		

		
So for those who have a use for type 2 hypervisors (aka, running virtual machines under windows or linux on your desktop), Oracle has just released Oracle Virtualbox 3.2 – this is the first version with the Oracle moniker – previously it was Sun VirtualBox. You can read about all the new features at http://blogs.oracle.com/mapledesk/VirtualBox%203.2%20Community%20Bulletin.pdf


If you check out the document that it looks like Oracle is going to try leveraging some of this technology in the VDI (Virtual Desktop Interface) space.  Should be interesting.


On my todo list for this week is to build up a windows 7 32-bit and a window 7 64-bit VM for the rest of IT to start beta testing for usage for an eventual company wide deployment.  We typically use VMware Workstation / Player here but if I’ve got the time and inclination I’ll grab a copy of VirtualBox and see how the VMs respond under each hypervisor.

	

	
	


							


	

		
				

			Mysql class
		

		
		

			Uncategorized, 					

	


		

		
So although I’ve  been working as an Oracle Apps DBA for over a decade, I’m spending the next 10 Tuesday evenings going to a beginner class for mysql.  You can find the class syllabus at http://geekaustin.org/news/2010/05/15/geekaustin-mysql-certification-class-starts-tuesday-may-18th


With Oracle’s acquisition on Sun earlier this year, Oracle now has greater influence on the path of mysql (yes, yes, it’s open source and like Java it isn’t *owned* by Oracle, but let’s face it, when you are the company providing much of the developer resources, you can influence the product direction).  Although my main job responsibilities are running all of my client Oracle systems, I also am a Redhat Certified Engineer (RHCE) and provide whatever linux services are necessary.  Lots of products don’t need the robustness of products like Oracle and MySQL, being free as-in-beer, is a good choice.  So I’m off to class.


Last night was the first class.  I’d say about 50 people showed up and class covered topics like client vs. server vs. host and how to do basics like importing a mysql database, doing a mysqldump, and “root”ing your MySQL dbs if/when you forget the passwords.  There was also discussion on the various clauses of a SELECT statement.  Overall, a well taught and fun couple of hours.  I look forward to the next class.


The goal of the class is to prepare the students to take and pass the Certified MySQL Associate certification offered at http://www.mysql.com/certification/candguide.html#t21  .  I’m not sure if I’m going to go for that certification or for the more appropriate for me ( Certified MySQL 5.1 Database Administrator (CMDBA 5.1)) certification which will require quite a bit of extra studying on my own.  Have to see how my free time pans out over the next couple of months.  I do wonder if the MySQL exams will be re-written to be like the other Oracle exams now that Oracle owns Sun.


For those who want to study on their own, supposedly everything for the exams can be found in the book titled “MySQL 5.0 Certification Study Guide” .  It’s available on amazon and also online at Safari Books.  For those curious, it’s quite readable on an iphone.

	

	
	


							


	

		
				

			I press Help for help… and get a 404?
		

		
		

			Uncategorized, , 					

	


		

		
A quick heads up to other Oracle Applications DBAs out there who apply the quarterly security (CPU) patches from Oracle to their Apps 11i instances – the patches break the online help for the end users.  Luckily, Oracle has released a patch for this (yay!) but they haven’t updated the CPU documents (which have a section just for that – Section 5, Document Modification History) nor have they re-released the patches or notified people who downloaded the patches – things easy to do and things that would make customers better value Oracle Support.


Failures like this by Oracle to get their documentation in sync drive DBAs and clients crazy and makes customers question why they pay 20% a year for Oracle Support.


For those looking to fix the issue, check out My Oracle Support document ID 1080465.1  .  It’ll tell you you need patch 9506302.


This was an issue caused by the OCT 09 CPU patches and was also in the JAN 10 CPU patches.  You can check for yourself that Oracle didn’t update the documents – Note IDs are 880170.1 and 985520.1 respectively.

	

	
	


							


	

		
				

			What is cloud?
		

		
		

			Uncategorized					

	


		

		
Even in mainstream news these days you here about cloud computing and how it will change the world.  Yet really, what is cloud?


The whole topic reminds me of a discussion I led in college.  I asked the group how many people at the start of discussion how many of them were feminists.  Very very few raised their hands.  Fine.  I then asked each person to define what a feminist was.  We then read out and discussed those definitions.  Like defining cloud, they were all over the map.  We finally decided on a definition of feminist as someone who believes in equal rights for women and men.  Armed with this definition we polled ourselves again on what is a feminist.  The result?  Almost every hand was raised.


I believe it’s the same with cloud.


The fact is, cloud computing is used by everyone on the internet every day.  It is in fact the Internet itself – a group of remotely accessibly computing resources.


Glad we got that out of the way.