Toby's Tumblr

Button Hotkeys in SQL Developer on OS X

Wed, 09 Jan 2013 14:09:45 -0500

In Windows, most applications provide menu and button accelerator keys that allow you to control the application without having to use a mouse. Java Swing apps like SQL Developer allow you to do the same, ostensibly in a cross-platform compatible manner.

One place I use this frequently is the Enter Binds dialog window that pops up when running a query using bind variables.

In Java 6 on OS X, the hotkey for pressing the Help and Apply buttons was OPTION-H and OPTION-A, respectively.

I initially thought this capability had been broken in Java 7 on OS X, but I discovered today that it has been merely changed to CONTROL-OPTION-H and CONTROL-OPTION-A.

Does anyone know if this is a configurable property of the application or Java virtual machine? If possible, I’d like to set it back to the simpler 1.6 way.

Using Jaspersoft’s free iReport Designer tool and the open...

Mon, 17 Dec 2012 22:03:42 -0500

Using Jaspersoft’s free iReport Designer tool and the open source PL-JRXML2PDF project, you can generate PDF reports from Oracle Apex for free, without having to install the JasperReports and JasperReports Server components. Watch my video tutorial to find out how.

Fall is almost over, but it hardly feels like winter here in...

Wed, 05 Dec 2012 15:53:01 -0500

Fall is almost over, but it hardly feels like winter here in Dallas.

Query an XML File Like An External Table

Tue, 13 Nov 2012 19:53:00 -0500

Oracle’s external table feature allows you to treat structured files on the database server as tables for query and DML purposes. To use an example from the linked documentation, a tab-delimited file like this:

56november, 15, 1980  baker             mary       alice     09/01/2004
87december, 20, 1970  roper             lisa       marie     01/01/1999

…could be queried like this:

SELECT employee_number,
       employee_first_name,
       substr(employee_middle_name, 1, 1),
       employee_last_name,
       employee_hire_date,
       to_date(employee_dob,'month, dd, yyyy')
  FROM employees;

After defining the external table like this:

CREATE TABLE employees
       (employee_number      CHAR(5),
        employee_dob         CHAR(20),
        employee_last_name   CHAR(20),
        employee_first_name  CHAR(15),
        employee_middle_name CHAR(15),
        employee_hire_date   DATE)
     ORGANIZATION EXTERNAL
       (TYPE ORACLE_LOADER
        DEFAULT DIRECTORY def_dir1
        ACCESS PARAMETERS
          (RECORDS DELIMITED BY NEWLINE
           FIELDS (employee_number      CHAR(2),
                   employee_dob         CHAR(20),
                   employee_last_name   CHAR(18),
                   employee_first_name  CHAR(11),
                   employee_middle_name CHAR(11),
                   employee_hire_date   CHAR(10) date_format DATE mask "mm/dd/yyyy"
                  )
          )
        LOCATION ('info.dat')
       );

Assuming of course that the file’s name is info.dat, and its path on the filesystem has already been created as a directory object DEF_DIR1 in the database.

A similar technique exists for querying simple XML files, taking advantage of Oracle’s XML DB features. HT to odie_63 on the XML DB forums for introducing me to this method.

Suppose you have a simple XML file (elements, but no attributes; no complex column types - it could work for the preceding, but I’m not sure) like so.

<Employees>
    <Employee>
        <id>1234</id>
        <lastname>Jetson</lastname>
        <firstname>George</firstname>
        <department>10</department>
        <salary>50000.00</salary>
    </Employee>
    <Employee>
        <id>1235</id>
        <lastname>Crackorn</lastname>
        <firstname>James</firstname>
        <department>10</department>
        <salary>40000.00</salary>
    </Employee>
    <Employee>
        <id>1236</id>
        <lastname>Hoffman</lastname>
        <firstname>Burl</firstname>
        <department>20</department>
        <salary>75000.00</salary>
    </Employee>
</Employees>

Place it on the database server in a readable directory, e.g. /home/oracle. Name it employees.xml.

Now execute the following in SQL*Plus:

SQL> CREATE OR REPLACE DIRECTORY XML_DIR AS '/home/oracle';

SQL> CREATE OR REPLACE VIEW employees_v AS
SELECT *
FROM XMLTable('/Employees/Employee'
        passing xmltype(
                 bfilename('XML_DIR','employees.xml')
                , nls_charset_id('AL32UTF8')
                )
        columns id           number path 'id'
              , lastname     varchar2(30) path 'lastname'
              , firstname    varchar2(30) path 'firstname'
              , department   number path 'department'
              , salary       number path 'salary'
       );

SQL> SELECT * FROM employees_v;
ID   LASTNAME FIRSTNAME DEPARTMENT SALARY
---- -------- --------- ---------- ------
1234 Jetson   George    10         50000    
1235 Crackorn James     10         40000
1236 Hoffman  Burl      20         75000

You can modify construction of the view to read from multiple files by using a simple UNION operator, or you can make the source filename dynamic so you can switch datasets at runtime. Here’s how you might do something like that:

SQL> CREATE OR REPLACE VIEW employees_v AS
SELECT *
FROM XMLTable('/Employees/Employee'
        passing xmltype(
                 bfilename('XML_DIR', userenv('CLIENT_INFO'))
                , nls_charset_id('AL32UTF8')
                )
        columns id           number path 'id'
              , lastname     varchar2(30) path 'lastname'
              , firstname    varchar2(30) path 'firstname'
              , department   number path 'department'
              , salary       number path 'salary'
       );

Then at runtime, specify the file you want to use like this:

SQL> exec dbms_application_info.set_client_info('employees_01.xml');
SQL> SELECT * FROM employees_v;
ID   LASTNAME FIRSTNAME DEPARTMENT SALARY
---- -------- --------- ---------- ------
1234 Jetson   George    10         50000    
1235 Crackorn James     10         40000
1236 Hoffman  Burl      20         75000

SQL> exec dbms_application_info.set_client_info('employees_02.xml');
SQL> SELECT * FROM employees_v;
ID   LASTNAME FIRSTNAME DEPARTMENT SALARY
---- -------- --------- ---------- ------
1234 Ringwald Malory    30         63000    
1235 Dulles   Bob       10         100000
1236 Stein    Frank     50         55000

Why might this be useful? Suppose you have an arbitrary number of identically structured XML files in a directory that you want to import into a relational database table. Using a shell script you can, for each file in the directory,

log in to SQL*Plus
set the CLIENT_INFO variable to the name of the current file
perform a simple insert into your table reading all records from the view

You might even create a cron job to monitor a particular directory for new XML files, processing them into tables and then deleting or archiving the files when done.

Restricting Access to BI Publisher Reports

Mon, 12 Nov 2012 13:12:00 -0500

In my last post I described how to configure BI Publisher to authenticate against an LDAP server, typically a corporate Active Directory server. Aside from authentication, you can also use LDAP group membership to determine who has access to a given report. Here’s how you do it.

Create a group in your LDAP/AD directory and assign your target users to that group.
Revisit your LDAP authentication settings. These are located on the Admin Tab, on the Security Configuration page. In particular, look at the value for Group Search Filter.

e.g.
(&(objectClass=group)(|(CN=XMLP*)(CN=DEPT_GROUP_*)))

In case you don’t recognize it, this is an LDAP query string. This query string determines which groups BI Publisher “sees”, or recognizes, when setting up report access. The groups returned by this query will determine the set from which you can choose when configuring which groups see which reports.

Here is a deconstruction of the query string:

(& Logical AND over the following conditions

(objectClass=group) Return LDAP groups, as opposed to users or computers, etc.

(| Logical OR over the following conditions

(CN=XMLP*) Return groups whose name starts with “XMLP”

(CN=DEPT_GROUP_*) Return groups whose name starts with “DEPT_GROUP_”

A quick primer on LDAP query syntax can be found here.

Modify this query string to include the group you just created.
Bounce BI Publisher.
Log in to BI Publisher as an administrator. Go to the Admin tab, and then to the Roles and Permissions page.
Click on the role/group you just created, and using the shuttle control on the next page, select which reports your new group should have access to. If necessary, you may need to revoke permissions on your report from other groups, or move your report to the top level of the report folder hierarchy in order to segregate access appropriately.

Configure Oracle BI Publisher to Authenticate With Active Directory

Mon, 12 Nov 2012 12:32:00 -0500

Log in to BI Publisher as an administrator.
Click on the Admin tab.
Click on Security Configuration.
For Security Model, choose LDAP and enter the following parameters:

Setting	Example Value
URL	ldap://ad.mycompany.com:389 Your company’s LDAP server and port.
Administrator Username	CN=ADMIN,OU=Users,OU=Shared Services,DC=mycompany,DC=com The fully qualified DN of the account you use to sign in to LDAP.
Administrator Password	myxlplyx The password for the above account.
Distinguished Name for Users	DC=mycompany,DC=com The “root” level at which to search for users.
Distinguished Name for Groups	OU=Global Groups,DC=mycompany,DC=com The “root” level at which to search from groups.
Group Search Filter	(&(objectClass=group)(\|(CN=XMLP)(CN=DEPT_GROUP_))) LDAP query string defines which groups are relevant to BI Publisher.
Group Attribute Name	cn
Group Member Attribute Name	member
Member Of Group Attribute Name
Group Description Attribute Name	description
JNDI Context Factory Class	com.sun.jndi.ldap.LdapCtxFactory
Group Retrieval Page Size
attribute used for RDN	sAMAccountName Allows users to log in to BI Publisher using their corporate AD account name.
Automatically clear LDAP cache	Checked
Ldap Cache Interval	1
Ldap Cache Interval Unit	Hour
Default User Group Name
Attribute Names for Data Query Bind Variables

Bounce BI Publisher.
Try logging in with your corporate AD credentials. Remember, you need to be assigned to one of the following special groups in Active Directory if you want developer or admin privileges in BI Publisher:

XMLP_ADMIN
XMLP_DEVELOPER
XMLP_SCHEDULER
XMLP_ANALYZER_EXCEL
XMLP_ANALYZER_ONLINE
XMLP_TEMPLATE_DESIGNER

You can read more about those roles and what they do in the Oracle Documentation.

Use Firebug's console.debug Command to Analyze Javascript, DOM Objects

Fri, 26 Oct 2012 17:09:00 -0400

Occasionally when debugging my Javascript code I need to be able to analyze the properties of an unknown object; for instance, an unexpected DOM object passed to an event handler. How do you analyze the properties of an object when you don’t know what the object is? I was looking for a way to list all the properties of a generic Javascript or DOM object when I stumbled across this blog post by Andrew Peters. In it, he very clearly outlines two ways of doing this, but the one I found most useful was Firebug’s console.debug command. You’ll need to use Firefox with the Firebug extension for this, but the results are impressive.

When you want to debug an object, simply pass it as a parameter to console.debug. Here is an example where I wanted to find out about the event trigger object passed to my event handler:

After triggering the event, I opened the Firebug window and navigated to the console. From there I could drill down on the displayed results to get information about my unknown DOM object.

Turns out in this case that the event handler was attached to the wrong object, due to the “id” value being placed one level above in the DOM hierarchy than I was expecting. Tricky, but I was able to spot the problem immediately using this method.

BI Publisher: "Job scheduling failed because the user has no permission to access this report"

Mon, 27 Aug 2012 16:48:08 -0400

You can get this error if you are trying to schedule a report and your BI Publisher installation uses LDAP authentication.

To avoid it, make sure that the “Administrator” account you use to connect to your LDAP server is in the XMLP_ADMIN group. The LDAP settings are located on the Admin tab, under Security Settings / Security Configuration.

Fix for Page Load Issues in Safari 5.1.7

Wed, 25 Jul 2012 10:29:37 -0400

Fix for Page Load Issues in Safari 5.1.7:

Lately I had been experiencing strange problems when clicking on links in Safari 5.1.7. Usually the progress indicator would appear for only a second with no obvious results, or else never appear at all. Pages would sometimes load, but after an unusual delay and with no indication that the browser was actually doing anything. I switched to Chrome as my default browser for a few days, but that was about all I could stand. A little research on the forums showed this fix, which appears to have worked for me.

Easy Fix for High CPU Usage in SQL Developer on OS X

Fri, 20 Jul 2012 12:32:00 -0400

I’ve been fighting for a couple of years now with an annoying performance bug in SQL Developer for OS X. After using the application for a while CPU usage would spike to around 100%, even when idle. This is a known issue associated with the “Look and Feel” preference.

Basically, don’t use the “Mac OS X” setting. Instead, use the “Oracle” setting. The theme doesn’t matter.

That fixes the CPU issue, but somewhat annoyingly moves the menu out of the menu bar at the top of the screen and into the application window, a la Windows, making the interface inconsistent with pretty much every other app I use on my Mac. Until recently, that is. The big news is that as of SQL Developer 3.1, the menu is restored to the top of the screen when using the Oracle Look and Feel. So for those who had been holding out (like myself) with the more native LaF, you can make the switch now and avoid the most annoying part of the performance/usability compromise.

Run Oracle Forms 11g Applications on OS X Lion

Thu, 12 Jul 2012 18:28:24 -0400

Need to run your company’s Oracle Forms application from your Mac, and can’t? Annoyed with having to crank up a virtual machine for just that one task? Even with the latest published Java version for OS X (1.6.0_33, as of today), you may find that some of the newer Forms applications just won’t run. Well, fret no longer. A solution may already be here, at least for Safari and Firefox.

Although it’s not a strictly certified configuration, you may have better luck by running the JRE 1.7 Java plug-in, which was recently made available as a pre-release download. This works for OS X Lion 10.7.3 and above. Download the plug-in from here and follow the simple instructions on the page to install it. Now give your application another try, and hopefully you’ll get the results I did!

Use a Perl CGI Script to Show Real Web Server Host Name

Mon, 02 Jul 2012 18:40:00 -0400

Here is a short and simple Perl CGI script that you can use to expose the hostname of your web server, as opposed to just the server/domain portion of the URL string:

#!/usr/bin/perl

print "Content-type: text/html\n\n";
print `hostname`;

1;

Name the file something like hostname.cgi and place it in your web server’s /cgi-bin/ directory, or if your web server is configured appropriately, just name it hostname.pl and drop it wherever you like.

The following script can be used to dump all the CGI environment variables.

#!/usr/bin/perl

print "Content-type: text/html\n\n";
print "<font size=+1>Environment</font>\n";
foreach (sort keys %ENV)
{
  print "<b>$_</b>: $ENV{$_}<br>\n";
}

1;

Note that this includes SERVER_NAME, but what that really gives you is the “server” portion of the URL string. In other words, if your URL is http://www.google.com/, then the SERVER_NAME CGI environment variable would be “www.google.com”. Fine, but the first script can be used to identify the actual server being called, for example, when your page is hosted on multiple servers load-balanced under a virtual domain name.

Of course, you could combine the two scripts to give you everything:

#!/usr/bin/perl

print "Content-type: text/html\n\n";
print "<font size=+1>Environment</font>\n";
foreach (sort keys %ENV)
{
  print "<b>$_</b>: $ENV{$_}<br>\n";
}

print "<b>REAL SERVER HOSTNAME:</b> ",`hostname`;

1;

OS X Software for Oracle Developers

Wed, 27 Jun 2012 17:39:00 -0400

One of the great things about working for Enkitec is that it’s a very Mac-friendly shop. When I arrived there in 2007 there were a handful of us running Macs and the attitude at the top seemed to be respectful, but skeptical. They allowed me to use my own laptop, which at the time was a 15-inch Intel Core2 Duo Macbook Pro, with the understanding that I could request a standard-issue Dell laptop at any time should the need arise. It never did. Since that time a large number of my coworkers have made the switch, abandoning their bulky, slow Dell Latitude laptops for 13-inch Macbook Pros, as have both owners of the company.

I’ve been a Mac user since 1998, since the advent of the original bondi blue iMac, a machine that I greatly loved, as far as you can love a machine (and no farther!). I’ve used a Mac at work since about the year 2000, a good while before OS X really hit the mainstream. Back in those days about the only thing I could hope to accomplish on it at the office was email, iTunes, looking up something on the internet, and running an SSH session or two. Still, I faithfully carried my little white plastic iBook G3 to work every day just because I enjoyed using the damn thing so much. I looked for any opportunity at all to integrate it into whatever I was doing at work. Over the years that came to include more and more stuff. I still remember how excited I was the day Oracle Applications earned Gold Certification on OS 9! Woohoo! I desperately needed a life.

The advent and acceptance of OS X changed everything. The Unix underpinnings gave legitimacy to the Mac as a serious “office” platform. I went from using my laptop as a toy to using it as a companion system, then a few years later as my primary system, then finally as my only system. Likewise, the number of Windows applications I needed to run on a daily basis, even in VMWare, was reduced from several to none. Hallelujah. In the same timeframe I also went from being the “office mac weirdo” to one of several “elite mac-using jackasses”, to just another one of the guys. Or so they tell me to my face.

One of the questions I’m asked most often by coworkers switching to the Mac is what software they should download first. This is my list.

[Note: I provide links to the developer’s sites, but many of these apps are available in the App Store.]

Crucial Stuff

These are the tools any Mac user, but particularly those working in IT, should have in their arsenal.

Butler

The one and only truly critical app in this list, Butler is an application launcher, search tool, universal bookmark manager, iTunes remote controller, Applescript launcher, keyboard macro tool, pasteboard manager and more. It puts the contents of your Mac at your fingertips and I couldn’t imagine using a computer without it. Indispensable.
1Password

Universal password manager, syncs with iPad and iPhone version. Use it to store the hundreds of passwords you accumulate both personally and on the job. A plugin for Safari, Chrome, and Firefox handles automatic entry of password credentials on websites you visit.
MS Office

If you’re in an office environment, chances are you’ll need it at some point when working with more complex Word documents or spreadsheets. Never use Powerpoint. See below.
iWork

Yes, I recommend both office suites, as Pages is vastly superior to Word as a composition tool and Keynote simply demolishes Powerpoint for presentations. Faster, lighter, easier to use — the whole iWork suite is worth it just for Keynote. Companion apps for iPad and iPhone are cheap and automatically sync your documents.
Growl

Perhaps soon to be obsoleted by the new Notification Center feature of OS X Mountain Lion, Growl provides a heads-up display for various things happening on the Mac. I use it mostly for email notifications, but the feedback it provides when attaching an external device or connecting to a network is often helpful.
BBEdit/TextWrangler

A good text editor goes a very long way, as any coder can tell you. Some people organize their whole lives with text files, though. Emacs is also available, but it doesn’t have the Mac sensibilities of this storied old tool. If you’d rather not spend the money, TextWrangler is the free, stripped-down counterpart.
Evernote

Solves the problem of how to catalog, archive, and make available the vast library of information you accumulate over the years. Companion apps for iPad and iPhone put all your information at your fingertips. Can also be used as a collaborative tool for teams.

Software for the Oracle Developer


SQL Developer	Oracle’s free tool for SQL and PL/SQL development.
NetBeans and/or JDeveloper	For J2EE development. Netbeans is the better tool, IMO, unless you need to work with Oracle ADF.
VirtualBox	I’ve used VMWare Fusion for years and recently tried VirtualBox to take advantage of Oracle’s developer VMs, and have been impressed with the performance and configurability. Beat’s VMWare’s price, as well.
Oracle 10g Instant Client	Can’t beat the convenience of being able to connect to a database without having to boot a VM.
CoRD	For remote desktop (RDP) connections. Traditionally, it’s been a better, more reliable option than Microsoft’s Remote Desktop app
Citrix Receiver	Needed If your company publishes corporate apps with Citrix Webapps, and many do.
VisualVM	Performance analysis/debugging tool for Java apps.
Apache Directory Studio	A useful and fully-featured LDAP tool.
Wireshark	Packet sniffer.
SmartCVS/SmartSVN	My tools of choice for easy source code manaagement.
MacPorts	Download and compile Unix software.
Kindle /Send to Kindle	Take advantage of Kindle’s massive collection of technical ebooks. The ‘Send to Kindle’ tool allows you to upload your own PDFs for reading in the Kindle app.
Bwana	Read `man` pages in a browser window.

Productivity Apps


OmniFocus	GTD task/life management tool. Companion apps for iPhone, iPad keep everything in sync and support location-based notes.
Marked	Markdown preview tool, used for blogging and documentation.
Acorn	Also might have chosen Pixelmator, but this is the one I own. Almost everybody could use a good image editor at some time or other. Pixelmator is supposed to be closer to Photoshop.
Skitch	Screenshot markup tool. Integrates with Evernote.

For Sharing and Collaboration


Dropbox	Disk space in the cloud with sharing and privacy features. Integrates with many applications on Mac, iPhone, iPad.
SpiderOak	More secure alternative to Dropbox for sensitive files.
Wuala	Another secure cloud-based storage option.
DropKey	A brain dead easy way to send secure, encrypted email attachments!
Skype	Video phone for talking to your PC-using buddies. For your Mac brothers you should use Facetime.
TeamViewer	Free screen sharing software, if you don’t have access to GoToMeeting.

System Tools


MenuMeters	Lightweight system monitor that sits in the menu bar. Measures CPU, memory, network, and disk usage.
SuperDuper!	Creates bootable, up-to-date copies of your entire hard drive! Great, possibly indispensable, for recovering from a hard disk failure. Can also be used just to sync disk contents. Only copies changed data during file sync. Very efficient.
iDefrag	Defrag utility. You shouldn’t need to do it ever in OS X, but I’m hoping it prolongs the life of my hard drive. Anybody got any real statistics on that?
iPartition	Disk Utility does pretty much everything this app does, except for allowing you to partition a disk without data loss. Unfortunately, this will not work if the disk is too full or badly fragmented. That’s when iDefrag really comes in handy.
CLIX	GUI tool for executing Unix commands. Comes with a large library of useful commands, including many Mac-specific ones. Can be used as a learning tool and you can add your own scripts.
OmniDiskSweeper and/or GrandPerspective	Analyze the contents and sizes of the stuff on your hard drive. GrandPerspective provides a better visualization, but you don’t always need the “fluff”.

Internet Tools


Interarchy	FTP/SFTP tool with numerous useful features. Download an entire website or keep remote sites in sync with local folders. Transmit is also popular.
Chrome	The web browser. (I use Safari.)
Firefox	The web browser. (See above.)
Webkit	See what’s coming down the pipe in Safari.
Transmission	BitTorrent client.
VLC	Open source media player. Plays a wide variety of formats.

If You Have The Money…


PDFPenPro	Create, edit, sign, mark up, footnote, redact, and otherwise manipulate PDFs. Also does OCR and tables of contents. Very nice tool. Companion app for iPad.
OmniGraffle	Like Visio, only a thousand times better. The Omni Group’s flagship product. Wish very much I owned this one.
OmniPlan	Does to MS Project what Keynote does to Powerpoint. Bought my copy a few versions back when it was far less expensive, and it still has all the project tracking features I need.

Obscure Use Cases


MailServe for Lion	Automates configuration of OS X’s built in Postfix mail server. I’ve had to use this in the past when I was unable to connect to the corporate email server and didn’t want to send files through Gmail. (Did I mention I despise most web mail clients?)
TNEF’s Enough	Used it once to extract the Word document embedded in a `winmail.dat` attachment from MS Outlook.
PST Bridge	Used convert or read PST files (basically mail archives) generated by MS Outlook.

Just for Fun


Nocturne	Invert your screen colors to reduce strain on the eyes at night, or to conserve battery power. Neat effect.
Audio Overload	Pass the time by listening to music from the console video games of long ago. Video game music files found separately, somewhere out there on the internet.

That’s it for now. Hope you find something useful.

DML Error Logging

Thu, 07 Jun 2012 13:04:00 -0400

Introduced with Oracle 10gR2, DML Error Logging allows you to make standard SQL INSERT, UPDATE, and MERGE statements behave more like the SQL*Loader utility, logging errors to a table rather than failing and rolling back the first time an error condition is encountered.

The syntax is the same for all DML statements. Just add the following clause to the end of your DML:

LOG ERRORS

The database will automatically create an error log table for you. Per the documentation (11g, 11gR2):

… the database assigns the default name generated by the DBMS_ERRLOG package. The default error log table name is ERR$_ followed by the first 25 characters of the name of the table upon which the DML operation is being executed.

In most cases you’ll probably want to specify your own log table, like this:

LOG ERRORS INTO error_table

If you do, you’ll need to create the table beforehand with DBMS_ERRLOG.

EXEC DBMS_ERRLOG.CREATE_ERROR_LOG('table', 'error_table');

Like SQL*Loader, you can specify a maximum number of errors to tolerate before failing and rolling back the transaction. You can also just say UNLIMITED if you don’t ever want to fail the transaction.

LOG ERRORS INTO error_table REJECT LIMIT 99
LOG ERRORS INTO error_table REJECT LIMIT UNLIMITED

Since the default limit is zero, it makes sense to always include this clause.

The last option is to specify a string that identifies that particular DML operation, so that you can group sets of errors in the log table. Here is an example that incorporates SYSDATE to tag all errors belonging to this DML operation.

LOG ERRORS INTO error_table ('Import from external datasource '||TO_CHAR(SYSDATE)) REJECT LIMIT UNLIMITED

Putting it all together, here is a short example demonstrating the use of DML Error Logging for merging data between two similar, but non-identically structured tables. An alternate strategy might have involved creating a custom interface table with a block of PL/SQL to perform inserts and mark the failed rows. Here, we take advantage of the native capabilities of the database to simplify and accelerate the process.

create table users_loc (id number primary key, name varchar2(75));

create table users_ext (uuid varchar2(10), fname varchar2(15), lname varchar2(20));
insert into users_ext values('1000', 'Scotty',  'P');
insert into users_ext values('2000', 'Melinda', 'Manning');
insert into users_ext values('200J', 'Jamie',   'Brooks');
insert into users_ext values('1000', 'Jason',   'Jennings');
insert into users_ext values('4000', 'Marcus',  'Nevada');
insert into users_ext values('AAAA', 'Alfred',  'Butler');
insert into users_ext values('ABAB', 'Simon',   'Jones');
insert into users_ext values('6000', 'Joon',    'Patel');
insert into users_ext values('7000', 'Wanda',   'Lutz');
insert into users_ext values('8000', 'Joseph',  'McBride');

exec dbms_errlog.create_error_log('users_loc','users_errs');

insert into users_loc (id, name) 
select uuid, 
       fname||' '||lname 
  from users_ext 
log errors into users_errs ('import from users_ext: '||to_char(sysdate)) 
reject limit unlimited;

col ora_err_mesg$ format a40
col ora_err_rowid$ format a15
col ora_err_tag$ format a50
col id format a10
col name format a30
select * from users_errs;

ORA_ERR_NUMBER$ ORA_ERR_MESG$                            ORA_ERR_ROWID$  ORA_ERR_OPTYP$ ORA_ERR_TAG$                                       ID         NAME                         
--------------- ---------------------------------------- --------------- -------------- -------------------------------------------------- ---------- ------------------------------
           1722 ORA-01722: invalid number                                I              import from users_ext: 07-JUN-2012 12:00:09        200J       Jamie Brooks                   


              1 ORA-00001: unique constraint (NUCDBA.SYS                 I              import from users_ext: 07-JUN-2012 12:00:09        1000       Jason Jennings                 
                _C00128181) violated                                                                                                                                                 


           1722 ORA-01722: invalid number                                I              import from users_ext: 07-JUN-2012 12:00:09        AAAA       Alfred Butler                  


           1722 ORA-01722: invalid number                                I              import from users_ext: 07-JUN-2012 12:00:09        ABAB       Simon Jones

The Database Is More Than Just Storage

Fri, 01 Jun 2012 16:00:00 -0400

The Database Is More Than Just Storage:

To folks writing database-driven reports for the web: Craig Shallahamer shows why you should let the (Oracle) database handle as much of your selection and transformation logic as possible.

SQLFiddle.com: An Online Multi-Vendor SQL Playground

Fri, 25 May 2012 15:04:00 -0400

SQLFiddle.com: An Online Multi-Vendor SQL Playground:

Build your own mini-schema and run queries against it. Publish your experiments for your friends to tweak. Or try out the examples from your favorite SQL book without having to build a database first. Works for Oracle, PostgreSQL, MySQL, and SQL Server flavors. Pretty cool!

Oracle's App Server Offerings: A High-Level Comparison

Fri, 18 May 2012 18:04:00 -0400

Ran across this post from the Dutch software and services company Yenlo that compares and contrasts Oracle’s major application server offerings. It would serve as a good starting point for deciding whether you truly need to pay the enormous licensing costs associated with a professional-grade application server platform, or if you were just wondering how to reconcile the continued existence of Oracle Application Server with WebLogic.

ODBC Connection Fails for MS Access App on Citrix

Fri, 11 May 2012 16:54:00 -0400

I ran into an interesting problem recently when attempting to publish an MS Access application with Citrix XenApp. The application pulled data from an Oracle database, so an ODBC connection was created on the Citrix server. What we found was that when a non-privileged user tried to run the application, the connection failed. When the Citrix server admin ran it, however, it worked. Furthermore, as long as the admin was logged in, anybody else could run the application successfully; when he logged out, the users’ connections broke again.

Since Access is not my area of expertise, we finally contacted Microsoft tech support after trying a few suggestions we dug up in our own research. The support analyst clued us in to the following solution, which I pulled from an Oracle Primavera setup document.

From what I’ve seen on different support forums, this seems to be a pretty standard setup task when running Oracle on Windows over Citrix.

If using an Oracle database platform, confirm [that] the Create Global Objects Windows Security Policy is set up on the Terminal Servers and add the domain Users or Groups of the end-users who will be accessing the published application. If the local security policy is not setup, the following steps can be used to add the policy:

Click Start, Programs, Administrative Tools, and then click Local Security Policy.

Expand Local Policies, and then click User Rights Assignments.

In the right pane, double-click Create Global Objects.

In the Local Security Policy Setting dialogbox, click Add.

In the Select Users or Group dialog box, select the user group that these end users are a part of, click Add, and then click OK.

Click OK.

If using an Oracle database platform, the Oracle client software also requires that you give the Authenticated User Group, Read and Execute privilege to the Oracle Home Directory and ALL its sub directories and files on the Terminal Servers.

Discovering Command-line Java Profiling Tools

Wed, 02 May 2012 17:41:00 -0400

At my current client I am responsible for administering a third-party Java appplication that unfortunately generates a lot of performance complaints. It’s a multi-tiered system with the middle-tier running on Oracle Appication Server 10.1.3. I am already familiar with GUI profiling tools like jconsole and jvisualvm, both of which are included with Oracle’s JDK, and I have played around with the profiling capabilities in NetBeans. All of these profiling tools are fairly similar to one another in look and functionality.

Using any one of these tools, I can view the size and utilization of the heap, see the status of running threads, monitor CPU loads, pull garbage collection statistics, and generate a memory dump. All of these are adequate for monitoring application performance in real-time, but while working with the application vendor to diagnose problems I found it necessary to be able to report on application performance over longer spans of time, say days or even weeks. We were trying to show patterns of eroding performance, or possibly spot instances of very slow memory leaks in parts of the application that got used less frequently. In any event, it required a way to monitor all of these details without having to have my eyes on the screen the whole time.

My goal was to get all that data exposed by the aforementioned GUI tools into a database that I could report off of. A little research on the internet (HT to Ted Neward) and some digging around in man pages provided the solution.

Recent versions of the JDK come packaged with some nifty command line tools to assist with profiling your Java applications. They can all be found in the /bin directory under the JDK base directory.

jps

The jps command is a lot like the ps command in Unix/Linux, in that it lists running processes. Jps, however, will only list discoverable Java processes. If a running Java process is not included in the list returned by jps, you may still be able to profile it; it just doesn’t advertise itself as attachable.

Each Java process is identified by the “VMID” returned by jps, usually corresponding to the process ID on the host system. It is this VMID which is required as an argument to all of the other tools we will discuss, so be sure to always use jps to correctly identify your processes to the other tools.

jstack

Use jstack to generate a thread dump for a running Java process. It gives you a nice snapshot of thread activity comparable to what you might get in one of the GUI tools mentioned above. You get the name of the thread, the current state, a stack trace, and with the -l option you get additional information about locking conditions. If an actual deadlock is detected you get a detailed breakdown of which threads are holding the locks! For comprehending the overall state of threads in an application it can actually prove superior to some of its GUI counterparts, as it’s faster to scan the results; i.e. no need to click from thread to thread or drill down to view the details.

Jstack can be used in conjunction with jmap and jhat to get a more comprehensive picture of application state at a critical point in time.

Note: If you have a 64-bit JDK, you’ll want to use the -J-d64 flag when running the command.

jmap & jhat

These two tools work together to help you analyze the contents of the heap. Use jmap to generate a heap dump, and jhat to help analyze the dump file.

Jhat works by spawning an HTTP server that presents the contents of the heap as a web page, allowing you to drill down on a specific object to view its members and references. In order to get the most out of jhat, however, it would be worth investigating OQL, the Object Query Language, as the web page generated by jhat allows you to run OQL queries against the heap. If you know a little about your application class structure, this makes your heap analysis far more efficient. Thankfully, the jhat web page provides you with a short primer.

Jmap is good for more than just generating heap dumps, however. When used with the -heap option it gives you a nicely formatted heap usage summary.

Now I was getting closer to my goal. In fact, this was originally the output I parsed during my first attempt at statistics gathering. Unfortunately, what I found while running on an RHEL 4.7 box with JDK 1.6.0_30 was that jmap would from time to time inexplicably refuse to attach to my Java process, leaving gaping holes in my report. I noticed similar complaints online, so I figured there may be problems with it.

# jmap -J-d64 -heap 11028
Attaching to process ID 11028, please wait...
Error attaching to process: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the process

Note: Once again, be sure to use the -J-d64 flag with jmap if you’re running a 64-bit JVM.

jstat

Jstat was what saved the day for me. Jstat displays detailed heap usage, class loader, and garbage collection statistics for the JVM; basically everything I wanted to report on. Jstat has numerous options that allow you to focus in on usage statistics for different parts of the JVM.

-class
Statistics on the behavior of the class loader.
-compiler
Statistics of the behavior of the HotSpot Just-in-Time compiler.
-gc
Statistics of the behavior of the garbage collected heap.
-gccapacity
Statistics of the capacities of the generations and their corresponding spaces.
-gccause
Summary of garbage collection statistics (same as -gcutil), with the cause of the last and current (if applicable) garbage collection events.
-gcnew
Statistics of the behavior of the new generation.
-gcnewcapacity
Statistics of the sizes of the new generations and its corresponding spaces.
-gcold
Statistics of the behavior of the old and permanent generations.
-gcoldcapacity
Statistics of the sizes of the old generation.
-gcpermcapacity
Statistics of the sizes of the permanent generation.
-gcutil
Summary of garbage collection statistics.
-printcompilation
HotSpot compilation method statistics.

Like the heap profiling features of the GUI tools mentioned earlier, the results of jstat can be used to check for memory leaks or to better tune your JVM startup parameters. Like the vmstat command, jstat can be set to repeat at a specified interval, for an infinite or specific number of iterations. Since jstat has so many options and output formats, I will examine it in more detail in a later blog post when I discuss how I used it to populate my application instrumentation table.

One thing that should be noted is that all of these tools are documented as being “unsupported” and experimental. Oracle make no guarantees that they will be included in future versions of the JDK or that their output or arguments will not change. Still, though, they’ve been around for a while and since they’re there, why not make use of them? Just don’t build any critical production processes off of them if you want total flexibility in taking future JDK upgrades.

In my next post I’ll examine how I used these tools to automate the process of instrumenting my third-party Java app. I’ll also provide you with a script I wrote that formats the output of jstat to look something like the -heap option of jmap, providing an equivalent, but more reliable, solution.

Table Functions: 9i Feature Worth Revisiting (Part 2)

Fri, 27 Apr 2012 18:35:00 -0400

In part I of this series I showed a simple example of how table functions can be used to treat PL/SQL collection types as tables in a SQL query. We used a nested table collection type to which we manually added a few entries. In this example, I intend to show a more practical use of table functions using all three PL/SQL collection types: associative arrays, nested tables, and VARRAYs.

Let’s revisit a use case I alluded to in my earlier blog entry: incorporating LDAP data into a query. Suppose your task is to code a custom authorization scheme for an enterprise application reporting system. You want row access to be determined by a user’s group memberships in the company’s Active Directory server. If a user is assigned to “the Company A group” in Active Directory, he gets to see Company A’s data. You can configure that kind of restriction at a database level using Oracle’s Virtual Private Database feature. Configuring VPD is beyond the scope of this blog entry, but ultimately you will be associating a table to a portion of a SQL WHERE clause that gets appended to every query written against that table. You can find more documentation on VPD on the OTN site. In this case, let’s assume you know that another team has written a PL/SQL database package to query data from Active Directory, and you want to make use of that code. The only problem is that all the functions in that package return collections of various types, and you need to be able to access that information from the WHERE clause you are building with VPD. This is where table functions come in handy.

Let’s begin by setting up our example schema. We’ll be looking at client sales data. First, let’s set up our clients.

CREATE TABLE clients (
  client_id NUMBER,
  client_name VARCHAR2(50));

INSERT INTO clients VALUES (100, 'Timco Industries');
INSERT INTO clients VALUES (200, 'Moneybros');
INSERT INTO clients VALUES (300, 'The Monacle Corporation');
INSERT INTO clients VALUES (400, 'Schmapple');

We also need application users who will be logging in to view the report data. We’ll assume each application user is uniquely linked to a user in Active Directory via the external_id column.

CREATE TABLE users (
  sequence_number NUMBER,
  user_name VARCHAR2(20),
  external_id VARCHAR2(20),
  first_name VARCHAR2(30),
  last_name VARCHAR2(30));

INSERT INTO users VALUES (1, 'JACKA', 'jaugusti', 'Jack', 'Augustine');
INSERT INTO users VALUES (2, 'KENKI', 'kkidd', 'Ken', 'Kidd');
INSERT INTO users VALUES (3, 'BRIAH', 'bhill', 'Brian', 'Hill');
INSERT INTO users VALUES (4, 'KEOVO', 'kvongkas', 'Keo', 'Vongkaseum');

Finally, we need a data table on which to report.

CREATE TABLE quarterly_sales_data (
  sequence_number NUMBER,
  client_id NUMBER,
  sales_year NUMBER,
  sales_quarter NUMBER,
  sales_amount NUMBER);

INSERT INTO quarterly_sales_data VALUES (1, 100, 2011, 1, 20000);
INSERT INTO quarterly_sales_data VALUES (2, 100, 2011, 2, 17500);
INSERT INTO quarterly_sales_data VALUES (3, 100, 2011, 3, 33000);
INSERT INTO quarterly_sales_data VALUES (4, 200, 2011, 1, 4500);
INSERT INTO quarterly_sales_data VALUES (5, 200, 2011, 2, 7000);
INSERT INTO quarterly_sales_data VALUES (6, 200, 2011, 3, 6300);
INSERT INTO quarterly_sales_data VALUES (7, 300, 2011, 1, 11100);
INSERT INTO quarterly_sales_data VALUES (8, 300, 2011, 2, 12800);
INSERT INTO quarterly_sales_data VALUES (9, 300, 2011, 3, 14000);
INSERT INTO quarterly_sales_data VALUES (10, 400, 2011, 1, 55600);
INSERT INTO quarterly_sales_data VALUES (11, 400, 2011, 2, 80000);
INSERT INTO quarterly_sales_data VALUES (12, 400, 2011, 3, 78500);

Our basic report query will look something like this:

SELECT client_name, sales_year, sales_quarter, sales_amount
  FROM quarterly_sales_data NATURAL JOIN clients;

Our Active Directory groups will be TIMCO_GRP, MONEYBROS_GRP, MONACLE_GRP, and SCHMAPPLE_GRP. If a user belongs to one or more of these groups, he has permission to view the respective client’s data.

Now we need a function that returns data from Active Directory. Just to keep it simple we will create a package that hardcodes the group memberships and returns them as collections of various types. We will then look at how table functions can be used to transform each one of those collections into an aggregate result set. (NOTE: querying LDAP directories from PL/SQL is not too difficult, and might serve as the basis for a future blog entry.)

Here’s our function. We’ve placed it in a package called LDAP_UTIL.

CREATE OR REPLACE PACKAGE ldap_util AS

  TYPE ldap_entry_typ IS RECORD (
    cn VARCHAR2(30),
    description VARCHAR2(1000),
    distinguished_name VARCHAR2(200),
    name VARCHAR2(30),
    samaccountname VARCHAR2(30));

  TYPE ldap_groups_nt IS TABLE OF ldap_entry_typ;
  TYPE ldap_groups_aa IS TABLE OF ldap_entry_typ INDEX BY PLS_INTEGER;
  TYPE ldap_groups_va IS VARRAY(100) of ldap_entry_typ;

  FUNCTION get_user_groups_nt (p_user VARCHAR2) RETURN ldap_groups_nt;
  FUNCTION get_user_groups_aa (p_user VARCHAR2) RETURN ldap_groups_aa;
  FUNCTION get_user_groups_va (p_user VARCHAR2) RETURN ldap_groups_va;

END ldap_util;
/
SHOW ERRORS

CREATE OR REPLACE PACKAGE BODY ldap_util AS

  FUNCTION get_user_groups_nt (p_user VARCHAR2) RETURN ldap_groups_nt IS
    l_groups ldap_groups_nt := ldap_groups_nt();
  BEGIN
    CASE p_user
      WHEN 'jaugusti' THEN

        l_groups.extend;
        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

      WHEN 'kkidd' THEN

        l_groups.extend;
        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

      WHEN 'bhill' THEN

        l_groups.extend;
        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

        l_groups.extend;
        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

      WHEN 'kvongkas' THEN

        l_groups.extend;
        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

        l_groups.extend;
        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

        l_groups.extend;
        l_groups(3).cn := 'SCHMAPPLE_GRP';
        l_groups(3).description := 'Employees working on the Schmapple account.';
        l_groups(3).distinguished_name := 'CN=SCHMAPPLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(3).name := 'SCHMAPPLE_GRP';
        l_groups(3).samaccountname := 'SCHMAPPLE_GRP';

    END CASE;
    RETURN l_groups;
  END get_user_groups_nt;

  FUNCTION get_user_groups_aa (p_user VARCHAR2) RETURN ldap_groups_aa IS
    l_groups ldap_groups_aa;
  BEGIN
    CASE p_user
      WHEN 'jaugusti' THEN

        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

      WHEN 'kkidd' THEN

        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

      WHEN 'bhill' THEN

        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

      WHEN 'kvongkas' THEN

        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

        l_groups(3).cn := 'SCHMAPPLE_GRP';
        l_groups(3).description := 'Employees working on the Schmapple account.';
        l_groups(3).distinguished_name := 'CN=SCHMAPPLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(3).name := 'SCHMAPPLE_GRP';
        l_groups(3).samaccountname := 'SCHMAPPLE_GRP';

    END CASE;
    RETURN l_groups;
  END get_user_groups_aa;

  FUNCTION get_user_groups_va (p_user VARCHAR2) RETURN ldap_groups_va IS
    l_groups ldap_groups_va := ldap_groups_va();
  BEGIN
    CASE p_user
      WHEN 'jaugusti' THEN

        l_groups.extend;
        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

      WHEN 'kkidd' THEN

        l_groups.extend;
        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

      WHEN 'bhill' THEN

        l_groups.extend;
        l_groups(1).cn := 'TIMCO_GRP';
        l_groups(1).description := 'Employees working on the Timco Industries account.';
        l_groups(1).distinguished_name := 'CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'TIMCO_GRP';
        l_groups(1).samaccountname := 'TIMCO_GRP';

        l_groups.extend;
        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

      WHEN 'kvongkas' THEN

        l_groups.extend;
        l_groups(1).cn := 'MONEYBROS_GRP';
        l_groups(1).description := 'Employees working on the Moneybros account.';
        l_groups(1).distinguished_name := 'CN=MONEYBROS_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(1).name := 'MONEYBROS_GRP';
        l_groups(1).samaccountname := 'MONEYBROS_GRP';

        l_groups.extend;
        l_groups(2).cn := 'MONACLE_GRP';
        l_groups(2).description := 'Employees working on the Monacle Corporation account.';
        l_groups(2).distinguished_name := 'CN=MONACLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(2).name := 'MONACLE_GRP';
        l_groups(2).samaccountname := 'MONACLE_GRP';

        l_groups.extend;
        l_groups(3).cn := 'SCHMAPPLE_GRP';
        l_groups(3).description := 'Employees working on the Schmapple account.';
        l_groups(3).distinguished_name := 'CN=SCHMAPPLE_GRP,OU=Global Groups,DC=acme,DC=com';
        l_groups(3).name := 'SCHMAPPLE_GRP';
        l_groups(3).samaccountname := 'SCHMAPPLE_GRP';

    END CASE;
    RETURN l_groups;
  END get_user_groups_va;

END ldap_util;
/
SHOW ERRORS

As you can see, LDAP_UTIL has three functions that return the three kinds of collections in PL/SQL: nested tables, associative arrays, and varrays.

FUNCTION get_user_groups_nt (p_user VARCHAR2) RETURN ldap_groups_nt;
FUNCTION get_user_groups_aa (p_user VARCHAR2) RETURN ldap_groups_aa;
FUNCTION get_user_groups_va (p_user VARCHAR2) RETURN ldap_groups_va;

The collections consist of a group of records representing “group” data from Active Directory.

TYPE ldap_entry_typ IS RECORD (
    cn VARCHAR2(30),
    description VARCHAR2(1000),
    distinguished_name VARCHAR2(200),
    name VARCHAR2(30),
    samaccountname VARCHAR2(30));

The p_user parameter would be the user’s network id, corresponding to the sAMAccountName attribute in Active Directory; since we’re hardcoding group data this is not important to us at the moment. The point is you pass in a username and you get a collection of groups.

Using the same technique we demonstrated in part one, we now create a wrapper package of pipelined functions to transform the collection data into something we can query - an aggregate result set.

CREATE OR REPLACE PACKAGE ldap_util_pipelined AS

  FUNCTION get_user_groups_nt (p_user VARCHAR2) RETURN ldap_util.ldap_groups_nt PIPELINED;
  FUNCTION get_user_groups_aa (p_user VARCHAR2) RETURN ldap_util.ldap_groups_nt PIPELINED;
  FUNCTION get_user_groups_va (p_user VARCHAR2) RETURN ldap_util.ldap_groups_va PIPELINED;

END ldap_util_pipelined;
/
SHOW ERRORS

CREATE OR REPLACE PACKAGE BODY ldap_util_pipelined AS

  FUNCTION get_user_groups_nt (p_user VARCHAR2) 
  RETURN ldap_util.ldap_groups_nt PIPELINED IS
    l_groups ldap_util.ldap_groups_nt;
  BEGIN
    l_groups := ldap_util.get_user_groups_nt(p_user);
    FOR i IN 1 .. l_groups.count LOOP
      PIPE ROW (l_groups(i));
    END LOOP;
    RETURN;
  END get_user_groups_nt;

  FUNCTION get_user_groups_aa (p_user VARCHAR2) 
  RETURN ldap_util.ldap_groups_nt PIPELINED IS
    l_groups ldap_util.ldap_groups_aa;
  BEGIN
    l_groups := ldap_util.get_user_groups_aa(p_user);
    FOR i IN 1 .. l_groups.count LOOP
      PIPE ROW (l_groups(i));
    END LOOP;
    RETURN;
  END get_user_groups_aa;

  FUNCTION get_user_groups_va (p_user VARCHAR2) 
  RETURN ldap_util.ldap_groups_va PIPELINED IS
    l_groups ldap_util.ldap_groups_va;
  BEGIN
    l_groups := ldap_util.get_user_groups_va(p_user);
    FOR i IN 1 .. l_groups.count LOOP
      PIPE ROW (l_groups(i));
    END LOOP;
    RETURN;
  END get_user_groups_va;

END ldap_util_pipelined;
/
SHOW ERRORS

Note the return types of the three functions in ldap_util_pipelined.

FUNCTION get_user_groups_nt (p_user VARCHAR2) RETURN ldap_util.ldap_groups_nt PIPELINED;
FUNCTION get_user_groups_aa (p_user VARCHAR2) RETURN ldap_util.ldap_groups_nt PIPELINED;
FUNCTION get_user_groups_va (p_user VARCHAR2) RETURN ldap_util.ldap_groups_va PIPELINED;

The first two return a nested table collection, while the third returns a VARRAY. These two collections can be pipelined because they correspond to native SQL datatypes in Oracle; not so for associative arrays. In order to pipeline the associative array returned from ldap_util.get_user_groups_aa, we need to map that collection to one of the other types. Here, we map it to a nested table. The iterative logic is the same, though, as the other two functions.

Now we can verify that our approach worked by issuing some simple queries, supported by the SQL “TABLE” construct.

SQL> select * from table(ldap_util_pipelined.get_user_groups_nt('jaugusti'));

CN        DESCRIPTION                                        DISTINGUISHED_NAME                            NAME      SAMACCOUN
--------- -------------------------------------------------- --------------------------------------------- --------- ---------
TIMCO_GRP Employees working on the Timco Industries account. CN=TIMCO_GRP,OU=Global Groups,DC=acme,DC=com  TIMCO_GRP TIMCO_GRP

As you can see, the individual attributes of the record structure form the columns of our new table, and we can reference those columns directly, if we wish.

SQL> select cn from table(ldap_util_pipelined.get_user_groups_nt('kvongkas'));

CN
---------------
MONEYBROS_GRP
MONACLE_GRP
SCHMAPPLE_GRP

All that’s left for us at this point is to construct a WHERE clause condition to use with Oracle’s Virtual Private Database feature to limit data returned from queries against the table QUARTERLY_SALES_DATA. That condition could look something like this, assuming that our application connects to the database as the logged in user.

client_id IN (SELECT CASE cn
                       WHEN 'TIMCO_GRP' THEN 100
                       WHEN 'MONEYBROS_GRP' THEN 200
                       WHEN 'MONACLE_GRP' THEN 300
                       WHEN 'SCHMAPPLE_GRP' THEN 400
                     END
                FROM TABLE(ldap_util_pipelined.get_user_groups_nt(USER)))

Here the mapping between AD group name and client ID is hardcoded in the CASE expression. A more flexible way would be to create a mapping table. Also, this is probably not a case study in writing a well performing VPD clause, but the real point here is that using SQL table functions and PL/SQL pipelined functions, we can translate data retrieved from an API, LDAP directory, or web service into table-like structures that we can query, join, and otherwise manipulate just like real tables.