Category Archives: Uncategorized

On Oracles commitment to Linux

Oracle prides itself on its strong support and commitment to Linux. On a webpage at Oracle’s site entitled Oracle’s Technical Contributions to Linux”

Oracle waxes eloquently on Oracle’s support, commitment and leadership for Linux. The paragraphs describe Oracle’s “long history of strong support and commitment to Linux, as evidenced by numerous, on-going technical contributions to the Linux community.”

The page then states that “Oracle continues to strengthen its involvement in the Linux community by providing enhancements that facilitate the development and deployment of enterprise Linux solutions. By developing enhanced capabilities and contributing code, Oracle’s Linux engineering teams continue to make the Linux experience better for all.”

Finally, the page lists a variety of projects and contributions where Oracle is involved. Very impressive stuff and I applaud Oracle for their contributions.

Ksplice is an extension of the Linux kernel which allows you to apply security patches to a running kernel without having to reboot the operating system. What’s the point in having a highly redundant clustered database that never needs to go down if it’s running on an operating system with a security hole that requires a reboot to patch? I highly recommend you read the wikipedia article I linked to above to read up how Ksplice works – it’s very cool.

From its founding in 2008 thru July 20th, 2011, Ksplice, Inc. the company that developed the Ksplice technology, provided prebuilt and tested updates for RedHat, CentOS, SuSE Enterprise Linux and other linux distributions – though not Oracle Linux.

On July 21st, 2011, support for RedHat Enterprise Linux and SuSE Enterprise Linux was dropped. It was announced that going forward this would be a feature only on Oracle Linux to customers who pay for premier support and only then when running Oracle Unbreakable Enterprise Kernel (UEK).

What happened on July 21st to cause Ksplice to drop the most popular enterprise linux distributions?

On July 21st, 2011, Oracle completed its acquisition of Ksplice, Inc.

[update] I also saw this tweet by ORCL_Linux, Oracle’s official twitter account for it’s Linux group:
“#Oracle Buys Ksplice…makes Oracle #Linux the ONLY OS with zero downtime patching bit.ly/qCZTBq ”

Not all Oracle databases require a license

Recently there was some discussion on Twitter about if infrastructure databases such as RMAN and OEM databases required licenses. I always figured they had the same licensing requirements as any other Oracle database.

However, I was incorrect.

If you read Oracle® Database Licensing Information 11gR2 it has this section:

Infrastructure Repository Databases

A separate Oracle Database can be installed and used as a Recovery Manager (RMAN) repository without additional license requirements, provided that all the Oracle databases managed in this repository are correctly licensed. This repository database may also be used for the Oracle Enterprise Grid Control repository. It may not be used or deployed for other uses.

A separate Oracle Database can be installed and used as a Oracle Enterprise Manager Grid Control (OEM Grid Control) repository without additional license requirements, provided that all the targets (databases, applications, and so forth) managed in this repository are correctly licensed. This database may also be used for the RMAN repository. It may not be used or deployed for other uses.

Good to know. For my clients who use VMware clusters to limit what hosts Oracle VMs can run on for licensing purposes, you do not need to restrict your Oracle RMAN and OEM Grid Control DBs / VMs. This allows me to free up those CPU / RAM / Network resources on my Oracle-only hosts for other license-restricted databases.

Choosing the right input output scheduler for Oracle on Linux

As I’ve wrote about before, having an understanding of the stack of software and hardware you are running workloads on is critical to getting the best performance out of your environment.

In all Linuxes (SuSE, RHEL, OL) for which Oracle supports running Oracle database, the kernel is responsible for scheduling the I/O in the system. There are multiple schedulers built into the kernel to allow you to choose the best scheduler for your disk I/O profile. For example, a database server is going to have a much different disk I/O profile than a webserver.

As you can read in this 2005 article from RedHat on RHEL 4 or this last revised 2008 pdf, there are significant benefits to be gained for your OLTP database server by changing your I/O scheduler from the default cfq (Completely Fair Queuing) to “deadline”. Deadline scheduler attempts to reduce latency of any given single I/O by waiting as long as possible before writing buffers to disk.

It’s interesting to note that, according to the Oracle Linux 6 release notes, Oracle’s Unbreakable Linux (UEK) kernel uses the deadline scheduler. Compare this with RedHat’s kernel which uses cfq by default. I wonder how much of Oracle’s performance improvements supposedly from UEK are actually from using a more appropriate scheduler for typical database workloads? I also wonder if deadline would be the right choice for Exadata, which uses UEK, since it has SAN type storage built in.

So the answer here is always use deadline scheduler, right?

Not so fast. What if I’ve virtualized my database and I’m running it under VMware or on a Storage Area Network (SAN)? VMware has designed vSphere as an Operating System (OS) optimized to run other OSes. You can find a fascinating thread on quora about vSphere and how it compares to other OSes.

Is deadline still most likely the best choice?

No. Even as of May 2010 in this paper from VMware on Oracle Databases on vSphere 4, there is NO mention of what scheduler to use.

So, as always, you should be diligent and work with your system administrators to test out what works best in your environment. Having said that, based on my experience and that of others, I typically set the scheduler to noop (No Operations) on all my linux VMs, regardless of if they are running Oracle or not.

More support available when virtualizing Oracle under VMware

What if your management wants more assurances about support for Oracle under VMware?

I’ve talked with many consultants and a few companies over the last year who have been concerned about getting support for their Oracle environment once it’s virtualized under VMware. I’ve written about this multiple times (Oracle listened, customers win! RAC supported on VMware, Oracle support on VMware, and Number One question at VMware booth at Oracle Openworld)

Oracle database, including the latest version (11.2.0.2) of Real Application Clusters (RAC) IS supported under VMware. It’s not certified by Oracle, but neither is almost any other hardware not made by Oracle (i.e. Your Dell servers and Cisco switches aren’t certified by Oracle). What this means is that (according to My Oracle Support (MOS) note 249212.1 ), in the unlikely event Oracle Support determines your known problem’s solution doesn’t work when virtualized, or if the problem is determined not to be a known Oracle issue, Oracle Support may refer you to VMware Support and will continue to work the issue when the customer can demonstrate the issue occurs on native (non-virtualized) hardware.

This has caused some organizations to give pause to virtualizing their Oracle environments under VMware. No organization wants to pay thousands of dollars in support only to find it isn’t there when they need it the most. To help reduce this anxiety over virtualizing Oracle products under VMware, VMware Global Support Sevices (GSS) provides support for VMware customers running Oracle 10g or 11g on VMware vSphere. You can read more about VMware’s Oracle Support policy at on VMware’s dedicated Oracle Support page.

In the event you are running into an issue with Oracle 10g or 11g issue under VMware vSphere 4, you should not only open a ticket with Oracle Support, but also a separate ticket with VMware Global Software Support (GSS). VMware will then use their expertise and resources to troubleshoot your issue to determine if the virtualization layer is the cause of the issue. If VMware deems the issue is not related to virtualization, VMware will escalate the ticket back through TSANet to Oracle Support.

TSANet (thankfully not associated with that TSA) is a vendor-neutral infrastructure that allows members such as Oracle, RedHat, Microsoft, NetApp, EMC and VMware to collaborate behind the scenes when a possible multivendor problem exists to resolve the customer issue. Typically customers aren’t even aware TSANet is being used between the vendors for communication.

In addition to support from Oracle and VMware, your storage vendor also has expertise you can leverage when experiencing issues.

If you’re running NetApp storage, check out their best practices for Oracle on NetApp. I’ve also been in contact with numerous people at NetApp regarding support resources and every NetApp person I contacted was extremely quick and resourceful in helping me find information. In a matter of hours, I had responses from a Virtualization Solution Architect, the Director of Global Support Services and Solutions, and the Senior Vice President of Support. Wow. Anyhow, NetApp has dedicated Virtualization and Oracle teams and also has a Joint Escalation Team (JET) with Oracle, VMware, Cisco etc. Even if you’re running a NetApp v-series controller in front of an EMC array, NetApp will support you and help you out. One final note, Oracle Corporate runs their Global Single Instance (their EBS instance) on NetApp according to the last published documentation I can find.

If you’re running EMC storage, they also have a Virtual Escalation Team process for Oracle on VMware vSphere on EMC. You can read more about EMC’s support of Oracle under VMware vSphere at Chad Sakacc’s blog post on Oracle, x86, VMware and update on support.

Odds are, whatever issue you’re running into or concerned about with virtualizing Oracle has been seen by someone else at VMware and your storage provider. With all the major vendors talking to each other under TSANet, you won’t be left to fend for yourself.

Don’t be scared to run your Oracle products under VMware vSphere. It’s supported by Oracle. It’s supported by VMware. Your storage vendor probably even has a specific team dedicated to Oracle on VMware.

Which would your CEO prefer

Would your CEO choose to give up a negligible amount of system performance at peak system load in exchange for a reduction of risk during system upgrades and an alternative to hours of downtime?

I suspect the answer is yes.

What’s the cost of such a technology? Believe it or not, it’s free. It’s the snapshot functionality that’s built into VMware’s vSphere Hypervisor (based on ESXi) that you can leverage once you virtualize your Oracle database or any other x86 systems. Is the performance impact truly negligible? Don’t take it from me; you can read it in Oracle’s own words.

Recently RedHat and Oracle have come out with updates to their mainstream distributions (RedHat Enterprise Linux 5.6 and Oracle Linux 5.6) and each has come out with an entirely new version (RedHat Enterprise Linux 6 and Oracle Linux 6). System and database administrators all around the world are updating their critical systems. Applying those updates to critical systems requires testing and downtime.

The best practice for operating system upgrades is to test the upgrade on the same exact hardware, software and data as in production. But are your test and production systems identical? Same motherboard? Same processors? Same amount and brand of RAM and same exact operating system packages installed? Pretty unlikely. With virtualization, all those configurations are identical.

With VMware virtualization, I can take take a clone of the entire production server while it’s live and being used by users. Now I have a truly identical copy of production to test my upgrade. Note that to do a clone of the entire production server while it’s live requires VMware vCenter which isn’t free, but vSphere Essentials (which includes vCenter) starts at $1000 as of the time of this writing.

With snapshots (which are free and don’t require vCenter), you take a snapshot (3 mouse clicks and less than 10 seconds) of the virtual machine and then do the upgrade. If you run into issues, just revert / rollback the snapshot to the state it was in when you took the snapshot. The time to do that revert / rollback is only the time necessary to reboot your virtual machine – 5 minutes? If the upgrade and testing goes smoothly, you just merge the snapshot into the virtual machine while the system is up and available to users. Total time spent doing non-upgrade activities such as backups or restores? Essentially zero.

Compare this to how you would handle a critical system without virtualization: you’d take the system offline for the upgrade, take a full backup of the system, do the upgrade (hoping it works just like it did in the similar but most likely not identical test system), have the users test, and, if there’s an issue, possibly spend hours restoring from backup. You do trust your backups… right?

Enterprise DBAs and the companies that employ them tend to be risk adverse when it comes to losing data or experiencing downtime. Virtualizing the hardware allows you to ensure your test and development systems are the exact same systems as production, thereby reducing risk of unforeseen issues during the upgrade. Utilizing snapshots allows you to very quickly take a save point of your entire server and rollback to it very quickly in the event of unforeseen issue, almost entirely eliminating (minutes vs. hours) the downtime associated with recovering from an unforeseen issue.

Licensing Oracle on VMware vSphere

Honestly, I thought this issue was done and buried, but over the past few weeks I’ve seen this question come up multiple times, so let’s get this cleared up.

Let’s go right to the source – Oracle’s own documentation. If you read Oracle’s partitioning document you will see that this is Oracle’s stance as of January 24, 2011. In it, it discusses soft partitioning and hard partitioning. Soft partitioning is leveraging the Operating System features to limit the number of CPUs an Oracle instance (or Oracle virtual machine) can run on. Hard partitioning physically partitions a large server into smaller self contained servers. The document lists what Oracle considers valid examples of each type of partitioning. In the document, Oracle specifically defines VMware’s partitioning (and Oracle VM’s partitioning) as soft partitioning. In the document, Oracle states that soft partitioning isn’t a “valid” means of restricting the amount of software licenses and you must license all the processors on a given server. Note that later in the document Oracle states that Oracle VM CAN be used for hard partitioning if you set it up as described in this document which goes into detail on how to bind an OracleVM VM to physical processors / cores. There is no mention in the documents if binding a VMware VM to a physical processor/core would also count as hard partitioning. Oracle does state that their list of partitioning technologies isn’t comprehensive, so things are left open to interpretation.

Please note I highly recommend you go and read these documents yourself and draw your own conclusions, and of course you can and should talk with an Oracle-employed licensing expert. In these documents Oracle states I cannot reproduce the document in any manner without express written consent so I am only telling you my interpretation.

VMware has three different techniques for restricting a VM to a specific subset of processors / cores. They are VMware vCenter clusters, VMware DRS affinity rules, and vSphere CPU affinity (pinning). I advise my clients to use the VMware vCenter cluster technique, but your organization might have a different interpretation. To describe the different VMware techniques, I will use an example of a 10 host VMware vCenter datacenter, with each host having 2 physical sockets and 4 cores per socket. Therefore, this entire VMware vCenter datacenter has 80 physical x86 cores (4*2*10) of processing power.

VMware vCenter clusters are logical clusters inside of vCenter made up of one or more hosts. By assigning a VM to that cluster, you are forcing that VM to run ONLY on the host(s) that make up that cluster. For example, if you create a 2 host VMware vCenter cluster inside your 10 host VMware datacenter, your VM can run on any processors / cores inside that 2 host cluster. As a result, Oracle licensing requires you to license all 16 (4*2*2) cores in that cluster. Note that you are restricting other non-Oracle workloads from also running on these hosts, so your Oracle VMs will get the best possible performance available on those hosts, possibly at the detriment to your non-Oracle workloads running on other hosts.

In vSphere 4.1, a DRS rule called “Virtual Machines to hosts” became available. That rule allows you to limit the location of a VM to specific host(s) in the VMware vCenter cluster. For example, if you create a DRS affinity rule assigning a VM to a single host inside your VMware cluster, your VM can run on any processors / cores inside that host. As a result, Oracle licensing requires you to license all 8 (4*2*1) cores on that host. You can read more about the VM to hosts affinity rule in this post by Frank Denneman who is a co-author (along with Duncan Epping) of vSphere 4.1 HA and DRS technical deepdive. Note that you aren’t restricting other non-Oracle workloads from also running on this host and thus you could have less than optimal Oracle performance.

VMware vSphere itself allows you to pin a virtual machine to one or more physical cores on a server using vSphere’s CPU affinity settings. You can read about the details of this in the vSphere resource management guide version 4.1 starting on page 20. This is the technical equivalent of the Oracle VM technique of binding a VM to a specific subset of physical processors / cores. For example, if you pin your Oracle VM to two physical cores, your VM can only run on those two physical cores. As a result, Oracle licensing requires you to license those 2 cores. Note that you aren’t restricting other non-Oracle workloads from also running on this core and thus you could have less than optimal Oracle performance.

Does Oracle consider VMware’s CPU affinity settings an acceptable form of partitioning? What about VMware DRS VM to host affinity rules? I have seen no official documentation from Oracle either way. I advise my clients to always buy enough Oracle licenses to allow Oracle to run on at least two hosts. This allows the customer to not be concerned about Oracle’s licensing ambiguity (as you’re licensing the entire hosts Oracle can run on) and also allows the customer to get the benefits of VMware such as vMotion, HA, DRS and FT to reduce and possibly eliminate downtime or less than optimal performance for their Oracle systems. I have had a client who went from running Oracle physical (with the one physical server having 8 processors / cores) to virtual (with the physical server having 8 processors / cores) and the client wanting the benefits of vMotion, HA, DRS and FT but without having to buy Oracle licenses for an additional 8 CPU host. According to the Oracle partitioning document I referenced earlier, Oracle does allow customers to only licenses processors / cores that are turned on. For this customer I therefore recommended that they turn off half the processors / cores in each host. Please note this limited their VMs to a maximum of 4 cores each- the amount of cores available on each host.

Licensing Oracle on VMware vSphere is an area of much confusion and disagreement due to Oracle not presenting clear public guidelines on whether DRS Affinity rules or vSphere CPU affinity are valid methods of partitioning.  I hope that Oracle addresses this licensing confusion soon, but until then, separate VMware vCenter clusters are the least legally risky way to virtualize Oracle.  I would love for someone from Oracle to officially and on the record address the techniques I mentioned in this post.

Software. Hardware. Complete.

So recently I was browsing Walmart.com ‘s Electronics section and was amazed at the selection they have.

You want to buy a computer? They’ve got it.

You want an operating system for that computer? They’ve got it.

You want to buy a network switch and cables to link multiple computers together? They’ve got it.

You want to buy 4TB of NAS storage? They’ve got it.

You can get them all from one vendor. The switches say their certified with the OS. The computer says its certified with the OS. Your storage is certified with your OS.

You can even install Oracle database on the hardware and be fully supported by Oracle (thought not certified by Oracle because Oracle doesn’t certify 3rd party hardware).

Have you ever bought a wireless Microsoft keyboard and mouse that didn’t work right with your Microsoft Windows OS running on a PC with a sticker on it that said “Designed for Windows” ? It’s all from one vendor. Just one throat to choke, right?

So why isn’t most of your data center running off of what’s at Walmart?

Because those products might not be leaders in their category.

Because the technical support backing those products might be crappy.

Because the software might not be enterprise ready .

Just because you can buy everything from one company doesn’t mean you should.

Oracle VM Security: Sometimes you need hip waders

Have you ever read something and thought, “what a load of crap. I had better get my hip waders out.”

Well, as a cynical jaded DBA, I have that experience regularly.

Take this Oracle.com blog post on Oracle VM where Rene Kundersma who is a Technical Architect with Oracle explains Oracle’s reasons for NOT shipping Oracle VM with a “fancy Gnome X-Window” environment:

“Oracle has it reasons to NOT ship Oracle VM with all the bells and whistles of a fancy Gnome X-Window environment. This has to do with vulnerabilities, not tested situations of software combination’s and whatever reason that makes Oracle VM not to behave as tested and intended.”

Vulnerabilities as the reason for Oracle VM not having a “fancy X-Window environment”. Vulnerabilities… really? But isn’t Oracle VM running on a special version of Oracle Unbreakable Linux (hint: yes – they’re both based off of RedHat Enterprise Linux)?

Want to get to the console of a VM running under Oracle VM? It uses VNC. Sure, you need to know the password to connect to the VNC Desktop, but guess what, the VNC traffic isn’t encrypted. The password is sent in cleartext.

Unbreakable indeed.

I find this all the more contradictory when one of Oracle’s talking points for why to use Oracle VM is Secure Live Migration which SSL encrypts the live migration (aka vMotion) traffic. My favorite line: “No need to purchase special hardware or deploy special secure networks. “

No need to deploy special secure networks! VLANs? Who needs them? We’ve got encrypted live migration!

Oh wait, in Oracle’s own Oracle Real Application Clusters in Oracle VM Environments guide, there’s this tidbit

“While Secure Live Migration is enabled by default, it should be considered that a secure connection to the remote guest (using –ssl) adds overhead to the Live Migration operation. It is therefore recommended to avoid such secure connections, if permitted. In general, a secure connection can be avoided, if the Live Migration network is inherently secure. “

Seriously Oracle, which is it?

But let’s get back to the main point Rene was trying to get across – that Oracle VM doesn’t come with a GUI to reduce vulnerabilities. Oracle’s October 2010 CPU (Critical Patch Update) was released on October 12th, 2010 and for the current version of Oracle VM (2.2.1) it lists 4 vulnerabilities, 3 of which have a base score of 9.0 (the scale is from 0.0 to 10.0, with 10.0 representing the highest severity of vulnerability). All 3 of those 9/10 severity vulnerabilities have a low access complexity (they’re easy to do) and result in complete access.

Oracle, thank you for not including a “fancy Gnome X-Window” with Oracle VM so as to reduce vulnerabilities. Given how insecure your product appears without a GUI, I shudder to think what things would be like with a GUI.

An alternative to scrambling data: Restricting access with Virtual Private Database (VPD)

Back in June, I wrote a blog post on scrambling HR data in our EBS instances . Although effective, it was a bit of a kludge – it involved an excel spreadsheet, and giving everyone the same salary and banking info.

As we went with this solution in our development and test environments, we ran into some issues where the salary data would totally screw up the benefits data as it’s calculated as a percentage of salary. The solution was effective at keeping the data secure, but it wasn’t optimal. After some investigation, we turned to Oracle VPD – Virtual Private Database – functionality. With this we are able to restrict access to certain columns (such as salary or national identifiers) to all but necessary users. With an EBS database, where every connection is connect as APPS, this poses special considerations.

I’ll cover the technical details of implementing VPD in an EBS environment. Then I’ll talk about the changes you need to make to keep things functional for your business analysts and yet keep the data secure.

First it was necessary to create a policy function. In our case this is very generic, basically just returning the predicate.

CREATE OR REPLACE FUNCTION “APPS”.”LUM_HIDE_HR_COLS” (schema in varchar2, tab in varchar2)

return varchar2 as predicate varchar2(8) default ’1=2′;

begin

return predicate;

end;

/

Next we add a policy on the column we want to restrict access to point it at the policy function we created

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PER_ALL_PEOPLE_F’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘NATIONAL_IDENTIFIER,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;

/

In this case we’re ADDing a policy and ENABLing it, applying the LUM_HIDE_HR_COLS function on the NATIONAL_IDENTIFIER column of table PER_ALL_PEOPLE_F in the HR schema, preventing users from SELECTing data and stating that this is for ALL_ROWS.

Once we issued that, all users (besides SYS and SYSTEM) will get NULLs when they select NATIONAL_IDENTIFIER from that table. That took care of our social security number concern.

We also set up additional policies on other data:

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PER_PAY_PROPOSALS’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘PROPOSED_SALARY_N,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;

/

The above policy restricts access to PROPOSED_SALARY_N column of HR.PER_PAY_PROPOSALS. That took care of our salary data concern.

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PAY_EXTERNAL_ACCOUNTS’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘SEGMENT3,SEGMENT4,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;
/

The above policy restricts access to the SEGMENT3 and SEGMENT4 columns of HR.PAY_EXTERNAL_ACCOUNTS. That took care of our banking data concern.

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PER_ADDRESSES’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘ADDRESS_LINE1,ADDRESS_LINE2,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;
/
The above policy restricts access to ADDRESS_LINE1 and ADDRESS_LINE2 columns of HR.PER_ADDRESSES. That took care of our concern of employees looking up addresses of other employees.

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PER_ALL_ASSIGNMENTS_F’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘GRADE_ID,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;
/

The above policy restricts access to the GRADE_ID column of HR.PER_ALL_ASSIGNMENTS_F. That took care of our concern about employees looking up the pay grade of other employees.

begin dbms_rls.add_policy(object_schema => ‘HR’, object_name => ‘PAY_ELEMENT_ENTRY_VALUES_F’, policy_name => ‘LUM_HIDE_HR_COLS’, function_schema => ‘APPS’, policy_function => ‘LUM_HIDE_HR_COLS’, statement_types => ‘select,’, update_check => FALSE , enable => TRUE , static_policy => FALSE , policy_type => dbms_rls.STATIC , long_predicate => FALSE , sec_relevant_cols => ‘SCREEN_ENTRY_VALUE,’ , sec_relevant_cols_opt => DBMS_RLS.ALL_ROWS ); end;
/

The above policy restricts access to the SCREEN_ENTRY_VALUE column of HR.PAY_ELEMENT_ENTRY_VALUES_F. That took care of our figuring out salary based on insurance coverage concern.

That’s all there is to it. You can just issue the code above in an Apps 11i instance and at that point no user besides SYS and SYSTEM can see that data.
Now that you’ve handled the technical details, there’s the issue of cloning the instances and being able to test HR functionality in development environments while still restricting the data. Here’s what we do.
In PROD, in addition to the function and policies listed above, we create a read-only user calls APPSHR:

create user APPSHR identified by XXXXXXXXXXXXX;
GRANT ALTER SESSION TO “APPSHR”;
GRANT CREATE DATABASE LINK TO “APPSHR”;
GRANT CREATE PUBLIC SYNONYM TO “APPSHR”;
GRANT CREATE SESSION TO “APPSHR”;
GRANT CREATE SYNONYM TO “APPSHR”;
GRANT EXECUTE ANY PROCEDURE TO “APPSHR”;
GRANT SELECT ANY DICTIONARY TO “APPSHR”;
GRANT SELECT ANY SEQUENCE TO “APPSHR”;
GRANT SELECT ANY TABLE TO “APPSHR”;
GRANT UNLIMITED TABLESPACE TO “APPSHR”;
GRANT “RESOURCE” TO “APPSHR”;
ALTER USER “APPSHR” DEFAULT ROLE ALL;
That APPSHR user now has the ability to select any data in the system but it’s read only (no updating).
We then exempt the APPS and APPSHR user in PROD ONLY from the policies we created:

GRANT EXEMPT ACCESS POLICY to APPS;
GRANT EXEMPT ACCESS POLICY to APPSHR;

At this point only users who connect at the database level as APPS (that would be all forms based users) and APPSHR (our HR analysts) can see the restricted data. The APPS password in our PROD environment is known only to the DBAs. The APPSHR password is known only to the HR Business Analysts / HR Developers / DBAs. All other business analysts have access to another read-only account called APPSXXXX that is NOT exempt from the security policies. With that APPSXXXX account, the regular business analysts and developers can query the database directly for all but the restricted data and can access any data via the forms that their forms responsibilities allow.
When we clone an instance, we don’t have to do any scrambling. For all of our DEV, TEST and PSUP (Production Support) instances we merely have to change the APPS password to a commonly known password and issue

REVOKE EXEMPT ACCESS POLICY FROM APPS;

Now everyone can read and write data as APPS, but because APPS now is restricted by the policies we put in place, APPS can’t see the sensitive data. The only time this causes a problem is for the HR Business Analyst or HR Developers who need a non-PROD place to work issues or develop code. For them we created a special HR cloned instance with the same security setup as PROD but with the APPS password known to the HR Analysts and HR Developers.

This solution has worked out much better for us than the scrambling. Give it a shot and let me know how it goes for you in the feedback!

Oracle Advanced Compression Advisor

My main Oracle Applications Database has been growing steadily and is now around 270GB. In terms of databases this isn’t huge, but when you keep multiple development and test copies around on enterprise class storage, AND replicate it to your DR site, that can get expensive.

With Oracle 11g database, Oracle came out with two products to help manage space (and improve performance!) in your Oracle database – Oracle Advanced Compression and Oracle SecureFiles. Although both are for reducing disk usage, they are aimed at different areas of disk usage within an Oracle database.

SecureFiles is the next generation of storage for Oracle LOBs (Large OBjects). Oracle LOBs stored in the format before SecureFiles are said to be stored in BasicFiles. SecureFiles is aimed at attachments to the database – CLOBS (Character LOBs), BLOBs (Binary LOBs), and NCLOBs (multi-byte character LOBs). SecureFiles offers a number of benefits over BasicFiles. Two are relevant to reducing space usage – de-duplification and compression. SecureFiles is a free feature of Enterprise Edition and has no additional licensing costs. As a result, it’s the sort of low hanging fruit that should be of interest to any Oracle DBA out there – free improved performance and free reduced disk storage. What’s not to like? Because this feature is free, we’re actively testing with this in our environments and plan on rolling this out by end of year. I’ll post a much longer blog post with our space savings and details of converting data from BasicFiles to SecureFiles later.

Advanced Compression is aimed at table data – compressing the data stored in the tables. This not only saves space on the file system, but actually improves performance by reducing the amount of data that needs to be read from disk (reading data from disk is frequently the bottleneck with Oracle databases – which is why Oracle is so memory hungry and tries to cache much of the data it needs in the System Global Area (SGA)). Advanced Compression is a add-on feature to Enterprise Edition at a cost of $11,500 per x86 license (and remember it takes TWO x86 CORES to equal one x86 LICENSE) – and like everything Oracle, that is based on how many cores are in the box, not how many your database cpu_count is set to or VM (if you virtualize your Oracle database) utilizes.

With Oracle Enterprise Manager (OEM) 11g, one of the new features is a Compression Advisor. You can read about other reasons to upgrade to OEM 11g at this blog post on OEM 11g new features. When run against an Oracle 11gR2 database, this advisor will analyze your database objects, estimate the compression ratio you’ll achieve and even make recommendations on the best compression settings for your environment. Although my largest database is 11gR2, I have a variety of other database versions on those same physical hosts (gotta love virtualization!) that aren’t 11gR2 and hence don’t have the DBMS_COMPRESSION package.

Luckily, I stumbled across a standalone version on Oracle Technology Network. This standalone version will work with Oracle 9iR2 (9.2.0.X) through 11gR1 (11.1.0.X) and can give you the data you need to convince business areas to upgrade to 11g database.

One thing to be aware of with this script: it will create a temporary table of the compressed data so you may wish to reating a tablespace specifically for storing the temporary table and making that the default tablespace of the user executing the script. The temporary table gets dropped at the end.

Note: The example on the Oracle Technology Network link above is incorrect. It is using the DBMS_COMPRESSION package which is in 11gR2 Oracle database and NOT provided by this package. So if using an 11gR2 database, you use DBMS_COMPRESSION package, but if using a 9iR2 thru 11gR1 database, use the DBMS_COMP_ADVISOR package like in my example below

Here’s the output from running it against a 9.2.0.8 database with a table OM_DATA in a schema called OO_MAIL. The table has 4.5 million rows and is 9.5 GB in size. (The product that uses this database requires Oracle 9iR2, for those wondering)

SQL> exec DBMS_COMP_ADVISOR.getratio(‘OO_MAIL’,'OM_DATA’,'OLTP’,25);

Sampling table: OO_MAIL.OM_DATA

Sampling percentage: 25%

Compression Type: OLTP

Estimated Compression Ratio: 1.62

PL/SQL procedure successfully completed.


I also ran this against my largest table in my Oracle Applications (11gR2) instance (INV.MTL_TRANSACTION_ACCOUNTS) – a 2.5GB table with 14 million rows:


Sampling table: INV.MTL_TRANSACTION_ACCOUNTS

Sampling percentage: 25%
Compression Type: OLTP
Estimated Compression Ratio: 2.57

So that works out to 3.64GB space I would save on the 9i database and 1.57GB in my 11gR2 database. A total of about 5GB saved. Every database (and the data it contains) is different, so run the numbers against your database to decide if Advanced Compression is worth it in your environment… and check out SecureFiles. It’s free.