Making the Most of the Best of Oracle PL/SQL - Toad World

Oracle PL/SQL Programming

Oracle PL/SQL Programming Things to change: • Overloading: for use in SQL, int vs bool • dbms_trace in tracing • API is a contract/backward compatibility

Making the Most of the Best of Oracle PL/SQL

Steven Feuerstein [email protected]


How to benefit most from this training • Watch, listen, ask questions, focus on concepts and principles. • Download and use any of my training materials:

PL/SQL Obsession

http://www.ToadWorld.com/SF

 Download and use any of my scripts (examples, performance scripts, reusable code) from the same location: the demo.zip file. filename_from_demo_zip.sql

 You have my permission to use all these materials to do internal trainings and build your own applications. – But remember: they are not production ready. – You must test them and modify them to fit your needs. Copyright 2013 Feuerstein and Associates

Page 2


Websites for PL/SQL Developers www.plsqlchallenge.com Daily PL/SQL quiz with weekly and monthly prizes

www.plsqlchannel.com 27+ hours of detailed video training on Oracle PL/SQL

www.stevenfeuerstein.com Monthly PL/SQL newsletter

www.toadworld.com/SF Quest Software-sponsored portal for PL/SQL developers

Copyright 2013 Feuerstein and Associates

Page 3


Making the Most of the Best of Oracle PL/SQL - What's the Best? • • • • • • • • • • •

Packages - the fundamental building block for PL/SQL apps Compiler optimization CASE statement and expression Autonomous Transactions Collections and set operations BULK COLLECT and FORALL Function Result Cache Table Functions NOCOPY hint Subtypes Nested subprograms


Page 4


How do we make THE MOST of all that? • Declare First, Program Second – Lift heavy only when necessary

• Craft Excellent APIs – Best form of communication

• Never Repeat Anything – For low maintenance code

• Program Socially – Never code alone.


Note: some of the "best" items will be explored in the "most" section. Page 5


Here's the Plan • First, cover many of the "best" features, so that everyone knows what they are and what they do for you. – More an overview than in-depth training

• Next, tackle each "most", bringing in specific PL/SQL features as is relevant. • Finally, a quiz! And prizes! • And now… on to our "best" features.


Page 6


Compiler Optimization • As of Oracle Database 10g, the compiler will automatically optimize your code. – Default setting of 2 is best, but 11g also offers a new level 3 "inlining" optimization.

• The optimizer takes advantage of "freedoms" to re-order the execution of statements. – In essence, changing the route that the runtime engine takes to get from point A to point B in your code.


Page 7


Some Optimizer Examples T := A + B; ... T ... ... ... T ...

... A + B ... ... ... A + B ...

for i in 1 .. 10 loop A := B + C; ... end loop; A := B + C; for i in 1 .. 10 loop ... end loop; FOR rec in (SELECT ...) LOOP ... do stuff END LOOP;


T is a generated variable. We never see it. And one operation is saved.

Automatic relocation of a loop invariant. Avoid repetitive computations.

SELECT ... BULK COLLECT INTO ... FROM ...

Execute cursor FOR loop at BULK COLLECT levels of performance.

10g_optimize_cfl.sql

Page 8


Optimizer Bottom Line • Use the default setting. – Check ALL_PLSQL_OBJECT_SETTINGS for violations.

• Apply the inlining pragma selectively if needed. • Forget all about it and enjoy the benefits! • OTN offers several whitepapers on the optimizer for those who want more. all_plsql_object_settings.sql


Page 9


CASE • Added to PL/SQL in 9i (earlier in SQL), you can use CASE statements and expressions to replace IF statements. • CASE expressions are especially good at replacing multiple IF statements with a single expression. – When building a complex string for example.

• Most lengthy ELSIFs should be replaced with a CASE statement. case*.sql


Page 10


Autonomous Transactions • Default transaction behavior in PL/SQL is at the session level – A commit saves all outstanding changes in your session. – A rollback erases all outstanding changes in your session.

• Define a PL/SQL block as an "autonomous transaction" to control the scope of commit/rollback. – Any changes made within that block will be saved or reversed without affecting the outer or main transaction.

• Two rules for autonomous transactions: – Must include the autonomous transaction pragma. – Must commit or rollback before exiting the block if any DML statements were executed.

• Most common application: error logging Copyright 2013 Feuerstein and Associates

Page 11


Logging with Autonomous Transactions CREATE OR REPLACE PACKAGE BODY log IS PROCEDURE putline ( code_in IN INTEGER, text_in IN VARCHAR2 ) IS PRAGMA AUTONOMOUS_TRANSACTION; BEGIN INSERT INTO logtab VALUES (code_in, text_in, SYSDATE, USER, SYSDATE, USER, rec.machine, rec.program ); COMMIT; EXCEPTION WHEN OTHERS THEN ROLLBACK; END; END; Copyright 2013 Feuerstein and Associates

Avoid interdependencies with the main transaction.

Save on successful exit Don't forget to rollback on error! logger.sp log81.pkg log81*.tst

retry.pkg retry.tst Page 12


Collections • Collections are PL/SQL's implementation of arrays. • All collections in PL/SQL are single dimensional lists or sets. • They provide the foundation for many performance optimization features. – Bulk processing, table functions and more.

• They can consume lots of PGA memory.


Page 13


Memory Management and Collections • Memory for collections (and almost all PL/SQL data structures) is allocated from the PGA (Process Global Area). • Accessing PGA memory is quicker than accessing SGA memory. – Sometimes much, much faster.

• Collections represent a very clear tradeoff: use more memory (per session) to improve performance. – But you definitely need to keep an eye on your PGA memory consumption. plsql_memory*.* Copyright 2013 Feuerstein and Associates

Page 14


Different Types of Collections • Three types of collections, with different characteristics and use cases. • Associative array – Use in PL/SQL only. Most flexible in that scope.

• Nested table – Use in PL/SQL and SQL. Lots of set-related operations.

• Varray (varying arrays) – Use in PL/SQL and SQL, but unlikely to ever use them.

• Collections come "with" methods. Copyright 2013 Feuerstein and Associates

Page 15


Collection Methods • The term method is used to describe procedures and functions that defined in a class or object type. – You invoke a method by attaching it, using dot notation, to the name of the type/class or to an instance of the class.

• Collection methods are procedures and functions that are attached to a collection variable. – First introduction of object-oriented syntax in PL/SQL – way back in Oracle 7.3.4! Copyright 2013 Feuerstein and Associates

method_vs_proc.sql

Page 16


Collections Methods • COUNT

– number of elements currently defined in collection.

• EXISTS

– TRUE if the specified index values is defined.

• FIRST/LAST

– lowest/highest index values of defined rows.

• NEXT/PRIOR

– defined index value after/before the specified index value .

• LIMIT

– max. number of elements allowed in a VARRAY.

• DELETE

– remove elements from a collection

• EXTEND

– add elements to a nested table or varray


Page 17


About Associative Arrays DECLARE TYPE list_of_names_t IS TABLE OF employees.last_name%TYPE INDEX BY PLS_INTEGER;

• TABLE OF datatypes can be almost any valid PL/SQL type (details to follow). • INDEX BY type can be integer or string. – This means you can essentially index by anything! – But index values can never be NULL.

• Associative arrays can be sparse. – Can populate elements in non-consecutive index values. – Easily used to emulate primary keys and unique indexes.


assoc_array_example.sql emplu.pkg, emplu.tst string_tracker*.*

Page 18


About Nested Tables CREATE OR REPLACE TYPE list_of_names_t IS TABLE OF NUMBER;

• A nested table is a type of collection, which, according to Oracle documentation, "models an unordered set of elements." – It is a "multiset": like a relational table, there is no inherent order to its elements, and duplicates are allowed/significant.

• From a practical standpoint, you can access nested table elements through an integer index. • MULTISET operators allow set-level operations on nested tables. Copyright 2013 Feuerstein and Associates

nested_table_example.sql

Unordered set of elements

1

Apple

2

Pear

3

Orange

4

Apricot

5

Pear

Integer index also available Page 19


Manipulating Nested Tables as Multisets • Nested tables are, from a theoretical standpoint, "multisets." – There is no inherent order to the elements. – Duplicates are allowed and are significant. – Relational tables are multisets as well.

• If a set has no order, then it has no index, so it must be manipulated as a set. • In Oracle Database 10g, Oracle added MULTISET set operators to manipulate the contents of nested tables (only). – Use in both PL/SQL blocks and SQL statements. Copyright 2013 Feuerstein and Associates

Page 20


Set-Oriented Features for Nested Tables • Determine if... – – – –

two nested tables are equal/unequal a nested table has duplicates, and remove duplicates one nested table contains another an element is a member of a nested table

• Perform set operations. – Join contents of two nested tables: MULTISET UNION. – Return common elements of two nested tables with MULTISET INTERSECT. – Take away the elements of one nested table from another with MULTISET EXCEPT (oddly, not MINUS). Copyright 2013 Feuerstein and Associates

10g_compare*.sql 10g_set.sql 10g_member_of.sql

10g_union.sql 10g_intersect.sql 10g_minus.sql

Page 21


Choosing the best type of collection • Use associative arrays when you need to... – Work within PL/SQL code only – Sparsely fill and manipulate the collection – Take advantage of negative index values or string indexing

• Use nested tables when you need to... – Access the collection inside SQL (table functions, columns in tables, or utilize SQL operations) – Want or need to perform high level set operations (MULTISET)

• Use varrays when you need to... – If you need to specify a maximum size to your collection – Optimize performance of storing collection as column Copyright 2013 Feuerstein and Associates

Page 22


Bulk Processing of SQL in PL/SQL (BULK COLLECT and FORALL) • The central purpose of PL/SQL is to provide a portable, fast, easy way to write and execute SQL against an Oracle database. • Unfortunately, this means that most developers take SQL for granted when writing SQL...and just assume Oracle has fully (automagically) optimized how SQL will run from within PL/SQL. Copyright 2013 Feuerstein and Associates

Page 23


The Problem with SQL in PL/SQL • Many PL/SQL blocks execute the same SQL statement repeatedly with different bind values. – Retrieve data one row at a time. – Performs same DML operation for each row retrieved. CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employee.department_id%TYPE ,newsal_in IN employee.salary%TYPE) IS CURSOR emp_cur IS SELECT employee_id,salary,hire_date FROM employee WHERE department_id = dept_in; BEGIN FOR rec IN emp_cur LOOP adjust_compensation (rec, newsal_in); UPDATE employee SET salary = rec.salary WHERE employee_id = rec.employee_id; END LOOP; END upd_for_dept; Copyright 2013 Feuerstein and Associates

The result? Simple and elegant but inefficient... Why is this?

Page 24


Repetitive statement processing from PL/SQL Oracle server PL/SQL Runtime Engine PL/SQL block FOR rec IN emp_cur LOOP UPDATE employee SET salary = ... WHERE employee_id = rec.employee_id; END LOOP;

Procedural statement executor

SQL Engine

SQL statement executor

Performance penalty for many “context switches” Copyright 2013 Feuerstein and Associates

Page 25


Bulk Processing in PL/SQL • The goal is straightforward: reduce the number of context switches and you improver performance. • To do this, Oracle "bundles up" the requests for data (or to change data) and then passes them with a single context switch. • FORALL speeds up DML. – Use with inserts, updates, deletes and merges. – Move data from collections to tables.

• BULK COLLECT speeds up queries. – Can be used with all kinds of queries: implicit, explicit, static and dynamic. – Move data from tables into collections.


Page 26


Bulk processing with FORALL Oracle server PL/SQL Runtime Engine PL/SQL block FORALL indx IN list_of_emps.FIRST.. list_of_emps.LAST UPDATE employee SET salary = ... WHERE employee_id = list_of_emps(indx); Update... Update... Update... Update... Update... Update...


Procedural statement executor

Fewer context switches, same SQL behavior

SQL Engine

SQL statement executor Update... Update... Update... Update... Update... Update...

Page 27


Impact of Bulk Processing in SQL layer • The bulk processing features of PL/SQL change the way the PL/SQL engine communicates with the SQL layer. • For both FORALL and BULK COLLECT, the processing in the SQL engine is almost completely unchanged. – Same transaction and rollback segment management – Same number of individual SQL statements will be executed.

• Only one difference: BEFORE and AFTER statementlevel triggers only fire once per FORALL INSERT statements. – Not for each INSERT statement passed to the SQL engine from the FORALL statement. Copyright 2013 Feuerstein and Associates

statement_trigger_and_forall.sql

Page 28


BULK COLLECT for multi-row querying SELECT * BULK COLLECT INTO collection(s) FROM table; FETCH cur BULK COLLECT INTO collection(s); EXECUTE IMMEDIATE query BULK COLLECT INTO collection(s);

• Retrieve multiple rows into collections with a single context switch to the SQL engine. – Collection filled sequentially from 1.

• Use the LIMIT clause to constrain PGA memory consumption. – 100 is good default, but experiment with higher values.

• When populating nested tables and varrays, Oracle automatically initializes and extends. Copyright 2013 Feuerstein and Associates

bulkcoll.sql bulkcollect.tst bulklimit.sql, cfl_to_bulk0.sql

Page 29


Use FORALL for repeated DML operations PROCEDURE upd_for_dept (...) IS BEGIN FORALL indx IN low_value .. high_value UPDATE employee SET salary = newsal_in WHERE employee_id = list_of_emps (indx); END; Bind array

• Convert loops that contain inserts, updates, deletes or merges to FORALL statements. • Header looks identical to a numeric FOR loop. – Implicitly declared integer iterator – At least one "bind array" that uses this iterator as its index value. – You can also use a different header "style" with INDICES OF and VALUES OF (covered later) Copyright 2013 Feuerstein and Associates

forall_timing.sql forall_examples.sql

Page 30


More on FORALL • Use the SQL%BULK_ROWCOUNT cursor attribute to determine how many rows are modified by each statement. – SQL%ROWCOUNT returns total for FORALL.

• Use SAVE EXCEPTIONS and SQL%BULK_EXCEPTIONS to execute all statements, saving errors for later. – You will need to handle ORA-24381.

• When collections may be sparse, use INDICES OF or VALUES OF. Copyright 2013 Feuerstein and Associates

bulk_rowcount.sql bulkexc.sql 10g_indices_of*.sql 10g_values_of.sql

Page 31


Converting to Bulk Processing • Let's take a look at the process by which you go from "old-fashioned" code to a bulk processing-based solution. • From integrated row-by-row to phased processing • With multiple DML statements in loop, how do you "communicate" from one to the other?


Page 32


The "Old Fashioned" Approach • Cursor FOR loop with two DML statements, trap exception, and keep on going. CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employees.department_id%TYPE , newsal_in IN employees.salary%TYPE) IS CURSOR emp_cur ...; BEGIN FOR rec IN emp_cur LOOP BEGIN INSERT INTO employee_history ... adjust_compensation (rec.employee_id, rec.salary); UPDATE employees SET salary = rec.salary ... EXCEPTION WHEN OTHERS THEN log_error; END; END LOOP; END upd_for_dept; cfl_to_bulk_0.sql Copyright 2013 Feuerstein and Associates

Page 33


A phased approach with bulk processing • Change from integrated, row-by-row approach to a phased approach. Relational Table

Phase 1: Bulk collect from table(s) to collection

Phase 2: Modify contents of collection according to requirements

Relational Table Phase 3: FORALL from collection to table Copyright 2013 Feuerstein and Associates

Page 34


Translating phases into code • The cfl_to_bulk_5.sql file contains the converted program, following the phased approach. Phase 1: Get Data Phase 3: Push Data Phase 2: Massage Data

Phase 3: Push Data

BEGIN OPEN employees_cur; LOOP fetch_next_set_of_rows ( bulk_limit_in, employee_ids, salaries, hire_dates); EXIT WHEN employee_ids.COUNT = 0; insert_history; adj_comp_for_arrays (employee_ids, salaries); update_employee; END LOOP; END upd_for_dept;


cfl_to_bulk_0.sql cfl_to_bulk_5.sql

Page 35


Conclusions – Bulk Processing • FORALL is the most important performance tuning feature in PL/SQL. – Almost always the fastest way to execute repeated SQL operations in PL/SQL.

• You trade off increased complexity of code for dramatically faster execution. – But remember that Oracle will automatically optimize cursor FOR loops to BULK COLLECT efficiency. – No need to convert unless the loop contains DML or you want to maximally optimize your code.

• Watch out for the impact on PGA memory!


Page 36

11g


The Oracle 11g Function Result Cache • Many of our queries fetch the same data many times, even if it hasn't changed. – Paying the price of executing SQL, no matter how optimized.

• Use the function result cache to tell Oracle to return unchanged data, without hitting the SQL layer. • This cache is... – stored in the SGA – shared across sessions – purged of dirty data automatically

• You can use and should use it to retrieve data from any table that is queried more frequently than updated. Copyright 2013 Feuerstein and Associates

11g_frc_demo.sql

Page 37

11g


Performance Impact of Result Cache • The result cache is stored in the SGA. • So we should expect it be slower than a PGAbased cache. • But accessing result cache data does not require going through the SQL engine. • So it should be much faster than executing a query. – Even if the statement is parsed and the data blocks are already in the SGA.

• Let's find out! Copyright 2013 Feuerstein and Associates

11g_emplu*.*

Page 38

11g


Result Cache – Things to Keep in Mind • If you have uncommitted changes in your session, dependent caches are ignored. – The cache will not override your own changed data.

• You can't cache everything. – Oracle has to be able to do an "=" comparison.

• Functions with session-specific dependencies must be "result-cached" with great care. – Virtual private database configurations, references to SYSDATE, reliance on NLS_DATE_FORMAT, time zone changes, Application contexts (calls to SYS_CONTEXT) – Solution: move all dependencies into parameter list.

• Work with your DBA to identify the "sweet spots" for applying this feature. Copyright 2013 Feuerstein and Associates

11g_frc_demo.sql 11g_frc_vpd.sql 11g_frc_vpd2.sql

Page 39


Table Functions

SELECT COLUMN_VALUE FROM TABLE (my_function (. . .))

• A table function is a function that you can call in the FROM clause of a query, and have it be treated as if it were a relational table. • Perform arbitrarily complex transformations of data and then make that data available through a query: "just" rows and columns! – After all, not everything can be done in SQL.

• Improve performance for…. – Parallel queries – User perceptions of elapsed time


Page 40


Building a table function • A table function must return a nested table or varray based on a schema-defined type. – Types defined in a PL/SQL package can only be used with pipelined table functions.

• The function header and the way it is called must be SQL-compatible: all parameters use SQL types; no named notation allowed until 11g. – In some cases (streaming and pipelined functions), the IN parameter must be a cursor variable -- a query result set.


tabfunc_scalar.sql tabfunc_streaming.sql

Page 41


Pipelined functions enhance performance. CREATE FUNCTION StockPivot (p refcur_pkg.refcur_t) RETURN TickerTypeSet PIPELINED

• Pipelined functions allow you to return data iteratively, asynchronous to termination of the function. – As data is produced within the function, it is passed back to the calling process/query.

• Pipelined functions can only be called within a SQL statement. – They make no sense within non-multi-threaded PL/SQL blocks. Copyright 2013 Feuerstein and Associates

Page 42


Applications for pipelined functions • Execution functions in parallel. – Use the PARALLEL_ENABLE clause to allow your pipelined function to participate fully in a parallelized query. – Critical in data warehouse applications.

• Improve speed of delivery of data to web pages. – Use a pipelined function to "serve up" data to the webpage and allow users to begin viewing and browsing, even before the function has finished retrieving all of the data.

• And pipelined functions use less PGA memory than non-pipelined functions! tabfunc_pipelined.sql


Page 43


Optimizing Table Functions in SQL • A function called in SQL is a "black box" to the optimizer - unless you provide statistics on the costs and selectivities of that function. – Use ASSOCIATE STATISTICS or hints or both. SQL> ASSOCIATE STATISTICS WITH FUNCTIONS high_cpu_io DEFAULT COST (6747773, 21, 0);

SQL> SELECT /*+ OPT_ESTIMATE(table, e, scale_rows=2.62) */ 2 * 3 FROM departments d 4 , TABLE(employees_piped) e 5 WHERE d.department_id = e.department_id; SQL> SELECT /*+ DYNAMIC_SAMPLING(e, 2) */ * 2 2 FROM TABLE(employees_piped) e;

Resources

http://www.oracle-developer.net/display.php?id=426 http://www.oracle-developer.net/display.php?id=427 http://www.dbprof.com/index.php?option=com_jdownloads&Itemid=57&view=viewcategory&catid=3 Copyright 2013 Feuerstein and Associates

Page 44


Table functions - Summary • Table functions offer significant new flexibility for PL/SQL developers. • Consider using them when you... – Need to pass back complex result sets of data through the SQL layer (a query); – Want to call a user defined function inside a query and execute it as part of a parallel query.

• Use pipelined table functions for performance improvement and reduced PGA consumption. Copyright 2013 Feuerstein and Associates

Page 45


The NOCOPY hint • By default, Oracle passes all IN OUT and OUT arguments by value, not reference. – This means that OUT and IN OUT arguments always involve some copying of data.

• With NOCOPY, you turn off the copy process. – But it comes with a risk: Oracle will not automatically "rollback" or reverse changes made to your variables if the NOCOPY-ed program raises an exception. nocopy*.* string_nocopy.* Copyright 2013 Feuerstein and Associates

Page 46


That's a LOT of BEST… Now it's Time for the MOST • Whew. • That's a lot of functionality. • And even if you know all about all of it, you still have to figure out how to use it so that your application is easy to: – Understand – Maintain (fix and enhance)


Page 47


The Big Four • Declare First, Program Second – Lift heavy (write code) only when necessary

• Craft Excellent APIs – It's the best form of communication

• Never Repeat Anything – For low maintenance code

• Program Socially – Never code alone.


Page 48


Declare First, Program Second DDL DML

• Maximize the declarative aspects of your technology before you start writing algorithms. • Get the data model right. – Use constraints and triggers

• Then maximize the SQL language and all its latest features. • Only then should you start writing Java/.Net programs/algorithms. PL/SQL


Page 49


Maximize the SQL Language • Courtesy of Lucas Jellema of AMIS Consulting, Netherlands • Analytical Functions – Especially LAG and LEAD; these allow you to look to previous and following rows to calculate differences. But also RANK, PIVOT, etc.

• WITH clause (subquery factoring) – Allows the definition of 'views' inside a query that can be used and reused; they allow procedural top-down logic inside a query

• Flashback query – No more need for journal tables, history tables, etc.

• ANSI JOIN syntax – Replaces the (+) operator and introduces FULL OUTER JOIN

• SYS_CONNECT_BY_PATH and CONNECT_BY_ROOT for hierarchical queries select d.deptno • Scalar subquery , (select count(*) – Adds a subquery to a query like a function call.

• And soon…Oracle12c new SQL features!

from emp e where e.deptno = d.deptno) number_staff from dept

Check out the SQL quizzes at the PL/SQL Challenge! Copyright 2013 Feuerstein and Associates

Page 50


Leverage Declarative Statements in PL/SQL • SQL, first and foremost - inside PL/SQL • Cursor FOR loop – Automatic optimization demonstrates the benefit

• FORALL statement • More generally, don't reinvent the wheel when built-in functions can do the heavy lifting. – Regular expressions – All nuances of string functions – TRUNC and TO_CHAR


Page 51


Craft Excellent APIs • An API is an application program interface. – A set of procedures and functions (and more) that can be used as "building blocks" for application construction.

• Clean, well-designed APIs hide details, reduce overall code volume, improve productivity, and the lower the frequency and severity of bugs. • In PL/SQL, packages are the best way to build APIs, though you can also do so in object types. Copyright 2013 Feuerstein and Associates

do_X do_Y return_Z

Page 52


Building with Packages • Employ object-oriented design principles – Build at higher levels of abstraction – Enforce information hiding - control what people see and do – Call packaged code from object types and triggers

• Encourages top-down design and bottom-up construction – TD: Design the interfaces required by the different components of your application without addressing implementation details – BU: packages contain building blocks for new code

• Organize your stored code more effectively • Implements session-persistent data Copyright 2013 Feuerstein and Associates

sf_timer.* plsql_memory.* assert.pkg

dbms_errlog_helper.sql errpkg.pkg loop_killer.pkg

Page 53


Package Data: Useful and (but?) Sticky • The scope of a package is your session, and any data defined at the "package level" also has session scope. – If defined in the package specification, any program can directly read/write the data. – Ideal for program-specific caching.

• Attention must be paid: – Package cursors must be explicitly closed. – Collection contents must be explicitly deleted. – Clean-up will not occur automatically on close of block.

• Note that with connection pools and stateless applications, you should not rely on package state between server calls.


thisuser.* emplu.pkg emplu.tst

Page 54


More on Package-level Data • Hide your package data in the body so that you can control access to it. – Only constants and "scratch" variables should be placed in the specification. – Build "get and set" subprograms

• Use the SERIALLY_REUSABLE pragma to move data to SGA and have memory released after each usage. – This also means that the package is re-initialized with each server call.

• A package with at least one variable has state, and that can greatly complicate recompilation. – Watch out for the ORA-04068 errors! – If you have these, check out Edition-Based Redefinition. Copyright 2013 Feuerstein and Associates

serial.sql valerr.pkg

Page 55


Package Initialization • The initialization section: – Is defined after and outside of any programs in the package. – Is not required. In fact, most packages you build won't have one. – Can have its own exception handling section.

• Useful for: – Performing complex setting of default or initial values. – Setting up package data which does not change for the duration of a session. – Confirming that package is properly instantiated.


init.pkg init.tst datemgr.pkg

PACKAGE BODY pkg IS PROCEDURE proc IS BEGIN END; FUNCTION func RETURN BEGIN END; BEGIN ...initialize... END pkg;

BEGIN after/outside of any program defined in the pkg.

Page 56


Overloading in Packages: key API/usability technique

myproc

• Overloading (static polymorphism): two or more programs with the same name, but different signature. – You can overload in the declaration section of any PL/SQL block, including the package body (most common).

myfunc myproc

• Overloading is a critical feature when building comprehensive programmatic interfaces (APIs) or components using packages. – If you want others to use your code, you need to make that code as smart and as easy to use as possible. – Overloading transfers the "need to know" from the user to the overloaded program. Copyright 2013 Feuerstein and Associates

Compare: DBMS_OUTPUT and p packages

dynamic_polymorphism.sql

Page 57


How Overloading Works • For two or more modules to be overloaded, the compiler must be able to distinguish between the two calls at compile-time. – Another name for overloading is "static polymorphism."

• There are two different "compile times": – 1. When you compile the package or block containing the overloaded code. – 2. When you compile programs that use the overloaded code.


Page 58


How Overloading Works, continued • Distinguishing characteristics: – The formal parameters of overloaded modules must differ in number, order or datatype family (CHAR vs. VARCHAR2 is not different enough). – The programs are of different types: procedure and function.

• Undistinguishing characteristics: – Functions differ only in their RETURN datatype. – Arguments differ only in their mode (IN, OUT, IN OUT). – Their formal parameters differ only in datatype and the datatypes are in the same family. Copyright 2013 Feuerstein and Associates

Page 59


Quiz! Nuances of Overloading • Will these specifications compile? If so, can I call the subprograms? CREATE OR REPLACE PACKAGE sales IS PROCEDURE calc_total (zone_in IN VARCHAR2); PROCEDURE calc_total (reg_in IN VARCHAR2); END sales;

CREATE OR REPLACE PACKAGE sales IS PROCEDURE calc_total (zone_in IN CHAR); PROCEDURE calc_total (zone_in IN VARCHAR2); END sales;

ambig_overloading.sql Copyright 2013 Feuerstein and Associates

Page 60


Tips for Optimal Overloading • Don't repeat code across implementations. – Single point of definition!

• Most overloadings involve doing mostly the "same thing", but with different combinations of data. • So make sure that in the package body, all overloadings are based on the same "core." • Don't overload "just in case". – Avoid hypothetically useful code. Copyright 2013 Feuerstein and Associates

lazy_overloading.sql dynamic_polymorphism.sql

Page 61


Tips for Writing Package Bodies • A well-constructed API (interface) is determined by the package specification. • But the way we implement that API in the package body has enormous ramifications on maintainability. • You must make very careful decisions about how to modularize your code. – Avoid spaghetti code – Expose only the functionality that is needed "out there" Copyright 2013 Feuerstein and Associates

Page 62


Modularization Choices • You can choose from: – – – –

Schema-level procedure or function Public packaged subprogram Private packaged subprogram Nested subprogram

• Avoid schema-level programs; put all your code in packages. – Entire package is loaded into memory – Each package provides a "namespace" in which to organize related code.

• Use nested subprograms to improve readability and maintainability of your bodies. Copyright 2013 Feuerstein and Associates

Page 63


Extreme Modularization (Write tiny chunks of code) • Spaghetti code is the bane of a programmer's existence. • It is impossible to understand and therefore debug or maintain code that has long, twisted executable sections. • Fortunately, it is really easy to make spaghetti code a thing of the past. Copyright 2013 Feuerstein and Associates

Organize your code so that the executable section has no more than fifty lines of code. Page 64


Fifty lines of code? That's ridiculous! • Of course you write lots more than 50 lines of code in your applications. • The question is: how will you organize all that code? • Turns out, it is actually quite straightforward to organize your code so that it is transparent in meaning, with a minimal need for comments. • Key technique: local or nested subprograms. Copyright 2013 Feuerstein and Associates

Page 65


Let’s write some code! • My team is building a support application. Customers call with problems, and we put their call in a queue if it cannot be handled immediately. – I must now write a program that distributes unhandled calls out to members of the support team.

• Fifty pages of doc, complicated program! But there is an "executive summary" Copyright 2013 Feuerstein and Associates

While there are still unhandled calls in the queue, assign them to employees who are under-utilized (have fewer calls assigned to them then the average for their department). Page 66


First: Translate the summary into code. PROCEDURE distribute_calls ( department_id_in IN departments.department_id%TYPE) IS BEGIN WHILE ( calls_are_unhandled ( ) ) LOOP FOR emp_rec IN emps_in_dept_cur (department_id_in) LOOP IF current_caseload (emp_rec.employee_id) < avg_caseload_for_dept (department_id_in) THEN assign_next_open_call (emp_rec.employee_id); END IF; END LOOP; END LOOP; END distribute_calls;

• A more or less direct translation. No need for comments, the subprogram names "tell the story" – but those subprograms don't yet exist! Copyright 2013 Feuerstein and Associates

Page 67


Explanation of Subprograms • Function calls_are_unhandled: takes no arguments, returns TRUE if there is still at least one unhandled call, FALSE otherwise. • Function current_caseload: returns the number of calls (case load) assigned to that employee. • Function avg_caseload_for_dept: returns the average number of calls assigned to employees in that department. • Procedure assign_next_open_call: assigns the employee to the call, making it handled, as opposed to unhandled. Copyright 2013 Feuerstein and Associates

Page 68


Next: Implement stubs for subprograms PROCEDURE call_manager.distribute_calls ( department_id_in IN departments.department_id%TYPE) IS FUNCTION calls_are_handled RETURN BOOLEAN IS BEGIN ... END calls_are_handled; FUNCTION current_caseload ( employee_id_in IN employees.employee_id%TYPE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; FUNCTION avg_caseload_for_dept ( employee_id_in IN employees.employee_id%TYPE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; PROCEDURE assign_next_open_call ( employee_id_in IN employees.employee_id%TYPE) IS BEGIN ... END assign_next_open_call; BEGIN

• These are all defined locally in the procedure. Copyright 2013 Feuerstein and Associates

locmod_step_by_step.sql

Page 69


About Nested Subprograms • They can be called only from within the block in which they are defined. – They can reference any variables defined in the parent block. – Watch out for "global" references.

• Only procedures and functions can be nested. – No packages within packages – No object types – No triggers

• Use these instead of nested blocks. – You replace code with a name – tell the story! Copyright 2013 Feuerstein and Associates

Page 70


Next: Think about implementation of just this level.

• Think about what the programs need to do. • Think about if you or someone has already done it. Don’t reinvent the wheel! Hey! Just last week I wrote another function that is very similar to current_caseload. It is now "buried" inside a procedure named show_caseload. I can’t call it from distribute_calls, though. It is local, private, hidden. Should I copy and paste? No! I should extract the program and expand its scope.


Page 71


Next: Isolate and refactor common code. CREATE OR REPLACE PACKAGE BODY call_manager IS FUNCTION current_caseload ( employee_id_in IN employees.employee_id%TYPE , use_in_show_in IN BOOLEAN DEFAULT TRUE) RETURN PLS_INTEGER IS BEGIN ... END current_caseload; PROCEDURE show_caseload ( department_id_in IN departments.department_id%TYPE) IS BEGIN ... END show_caseload;

Note the increased complexity, needed to ensure backward compatibility.

distribute _calls

show_ caseload

PROCEDURE distribute_calls ( department_id_in IN departments.department_id%TYPE ) IS BEGIN ... END distribute_calls; END;

• Now current_caseload is at the package level and can be called by any program in the package. Copyright 2013 Feuerstein and Associates

locmod_step_by_step.sql

current_ caseload

Page 72


Next: Reuse existing code whenever possible. • Just last week, Sally emailed all of us with news of her call_util package. – Returns average workload of employee and much more. – Just what I need! Don’t have to build it myself, just call it. BEGIN WHILE ( calls_are_unhandled ( ) ) LOOP FOR emp_rec IN emps_in_dept_cur (department_id_in) LOOP IF current_caseload (emp_rec. employee_id) < call_util.dept_avg_caseload (department_id_in) THEN assign_next_open_call (emp_rec.employee_id); END IF; END LOOP; END LOOP; This program has the widest scope possible: it can be END distribute_calls; executed by any schema with execute authority on the

call_util package, and by any program within the owning schema. Copyright 2013 Feuerstein and Associates

Page 73


Next: Implement what’s left. • Now I am left only with program-specific, nested subprograms. • So I move down to the next level of detail and apply the same process. – Write the “executive summary” first. – Keep the executable section small. – Use local modules to hide the details.

• Eventually, you get down to the “real code” and can deal with the actual data structures and algorithms without being overwhelmed. Copyright 2013 Feuerstein and Associates

locmod_step_by_step.sql topdown*.*

Page 74


Challenges of Nested Subprograms • Requires discipline: always be on the lookout for opportunities to refactor. • Need to read from the bottom, up. – Takes some getting used to.

• Sometimes can feel like a "wild goose chase". – Where is the darned thing actually implemented? – Your IDE should help you understand the internal structure of the program.

• You cannot directly test nested subprogams. • But how do you decide when a module should be local or defined at a “higher” level? Copyright 2013 Feuerstein and Associates

Page 75


Rule: Define subprograms close to usage. • When should the program be nested? Private to the package? Publicly accessible? • The best rule to follow is: Define your subprograms as close as possible to their usage(s). • The shorter the distance from usage to definition, the easier it is to find, understand and maintain that code.


Page 76


Craft Excellent APIs - Conclusion • The more clearly you define and control access to underlying functionality, the easier it will be to use and maintain your code. • Ideally, a developer never needs to look at the package body to use that code. • It's a real joy to be able to construct new, complex programs by pulling out pre-tested units from your "set of blocks." Copyright 2013 Feuerstein and Associates

Page 77


Never Repeat Anything • The single most important guideline for writing high quality code. • Repetition => hard coding => exposed implementation => maintenance nightmare. • Instead, aim for a Single Point of Definition (SPOD) for every aspect of your application. • The hardest part of doing this can be recognizing the different ways that hard-coding can creep into your code. hardcoding2.sql


Page 78


Never Repeat Anything - Specifically…. • • • • •

Hide magic values Hides constrained declarations Use a shared error logging utility Use a shared execution tracer Build a data encapsulation layer for SQL


Page 79


Hide Magical Values (Literals)

• The most commonly recognized form of hardcoding. • The only place a literal should appear in your code is in its SPOD. • Hide literals behind constants or functions. • Consider soft coding values in tables. Copyright 2013 Feuerstein and Associates

Page 80


Hide Behind Constant • Instead of exposing the literal value, and referencing it in multiple places, declare a constant and reference that name. • Best to put such constants in a package specification. – Can share across entire code base.

• Constants are simple and quick, but they expose the value in the package specification. – If the value needs to change, all programs that depend on that package must be recompiled. Copyright 2013 Feuerstein and Associates

constant_vs_function.sql

Page 81


Hide Behind Function • You can also define a function whose body returns the value. – Best done in a package

• Advantages over constants include – When the value changes, only the package body must be recompiled. – Developers cannot "lazily" see/use value. – You can call the function in an SQL statement

• But this is less efficient than a constant. constant_vs_function.sql Copyright 2013 Feuerstein and Associates

Page 82


Soft-Code Values in Table • You can make things really flexible by putting all literals in a table, associating them with a name, and retrieving them as needed from the table. • Downsides are: – More complex code – More overhead, but caching can avoid this problem. soft_code_literals.sql Copyright 2013 Feuerstein and Associates

Page 83


Hide error codes with EXCEPTION_INIT • Oracle doesn't provide a name for every error code, but you can do this. • Best place to put exception declarations is a package, so they can be shared across the application. Copyright 2013 Feuerstein and Associates

WHEN OTHERS THEN IF SQLCODE = -24381 THEN ... ELSIF SQLCODE = -1855 THEN ... ELSE RAISE; END;

e_forall_failure EXCEPTION; PRAGMA EXCEPTION_INIT ( e_forall_failure, -24381); BEGIN .... EXCEPTION WHEN e_forall_failure THEN ... END;

errpkg.pkg

Page 84


Hide Constrained Declarations • Every declaration requires a datatype. • If you are not careful, the way you specify that datatype could be a hard-coding. – Generally, any declaration that relies on a constrained datatype is a hard-coding. – VARCHAR2(n) is constrained; BOOLEAN and DATE are unconstrained.

• Two problems with hard-coding the datatype: – Constraints can lead to errors in future. – The datatype does not explain the application significance of the element declared. Copyright 2013 Feuerstein and Associates

Page 85


"SPODification" for Datatypes Consider every VARCHAR2(N) declaration to be a bug – unless it's a SPOD.

• Whenever possible, anchor the datatype of your declaration to an already-existing type. – That way, if the existing type or SPOD ever changes, then your code will be marked INVALID and automatically recompiled to pick up the latest version of the anchoring type.

• Use %TYPE and %ROWTYPE whenever possible – Fetch into record, never list of variables

• Use SUBTYPEs when anchoring is not possible. Copyright 2013 Feuerstein and Associates

Page 86


%TYPE and %ROWTYPE • Use %TYPE for declarations based on columns in tables. • Use %ROWTYPE for records based on tables, views or cursors. • The lookup of the datatype from these attributes occurs at compile-time. – There is no run-time overhead.


no_more_hardcoding.sql

Page 87


Fetch into record, not list of variables • If your FETCH statement contains a list of individual variables, you are hard-coding the number of elements in the SELECT list. – When the cursor changes, you must change the FETCH as well.

• Solution: always fetch into a record, defined with %ROWTYPE against the cursor.

fetch_into_record.sql Copyright 2013 Feuerstein and Associates

Page 88


SUBTYPEs • You can't always use %TYPE or %ROWTYPE in your declaration. • You can, however, always define a "subtype" or subset of an existing type with the SUBTYPE statement. SUBTYPE benefits: – Avoid exposing and repeating constraints. – Give application-specific names to types. Critical when working with complex structures like collections of records, and nested collections. – Apply constraints, such as numeric ranges, to the variable declared with the subtype. Copyright 2013 Feuerstein and Associates

Page 89


SUBTYPE Details and Examples SUBTYPE type_name IS data_type [ constraint ] [ NOT NULL ]

• Define a subtype based on any pre-defined type or other, already-defined subtype. • If the base type can be constrained, then you can constrain the subtype. – (precision,scale) or RANGE

• You can also, always specify NOT NULL. – Even if the base type could be NULL. Copyright 2013 Feuerstein and Associates

subtype_examples.sql

Page 90


Applying SUBTYPEs • Two key scenarios: – Whenever you are about to write a VARCHAR2(N) or other constrained declaration, define a subtype instead, preferably in a package specification. – Instead of writing a comment explaining a declaration, put the explanation into a subtype. Instead of this: Write this:

DECLARE l_full_name VARCHAR2(100); l_big_string VARCHAR2(32767);

DECLARE l_full_name employees_rp.full_name_t; l_big_string plsql_limits.maxvarchar2;


fullname.pks plsql_limits.pks string_tracker3.*

Page 91


Conclusions • Declarations offer a danger of hard-coding of both datatype and constraint on that type. • Assume that over time everything will change. • Apply the same "single point of definition" principle to your declarations. – Use %TYPE and %ROWTYPE whenever possible. – Fall back on subtypes to define application specific types and PL/SQL limitations.


Page 92


Error Management and Logging • Handling problems gracefully and effectively is critical. • You need to know what Oracle offers to help you diagnose issues. • And you need to make sure error handling and logging is not a mess of hard-codings.


Page 93


Oracle Built-ins For Handling Exceptions • In addition to the application-specific information you may want to log, Oracle builtins provide you with answers to the following questions: – How did I get here? – What is the error code? – What is the error message and/or stack? – On what line was the error raised?


Page 94


SQLCODE and SQLERRM • SQLCODE returns the error code of the most recently-raised exception in your session. • SQLERRM returns the error message associated with SQLCODE – but it also a generic error message lookup function. • Neither SQLCODE nor SQLERRM can be called from within a SQL statement. – You must assign them to local variables to use their values in SQL statements (like writing to an error log). sqlcode.sql sqlcode_test.sql Copyright 2013 Feuerstein and Associates

Page 95


SQLERRM Details • If you don't pass an argument to SQLERRM, it returns the error message for the SQLCODE value. – When called outside of an exception handler, always returns "success" message – no error.

• You can also pass an error code to SQLERRM and it will return the generic error message. • The maximum size of a string returned by SQLERRM is 512 bytes. – When there is a stack of errors, Oracle may truncate the string returned by SQLERRM. – Oracle recommends you use DBMS_UTILITY.FORMAT_ERROR_STACK instead. sqlerrm.sql Copyright 2013 Feuerstein and Associates

Page 96


DBMS_UTILITY error functions • Answer the question "How did I get here?" with DBMS_UTILITY.FORMAT_CALL_STACK. • Get a more complete error message with DBMS_UTILITY.FORMAT_ERROR_STACK. • Find line number on which error was raised with DBMS_UTILITY.FORMAT_ERROR_BACKTRACE.


Page 97


DBMS_UTILITY.FORMAT_CALL_STACK • The "call stack" reveals the path taken through your application code to get to that point. • Very useful whenever tracing or logging errors. • The string is formatted to show line number and program unit name. – But it does not reveal the names of subprograms in packages. callstack.sql callstack.pkg Copyright 2013 Feuerstein and Associates

Page 98


DBMS_UTILITY.FORMAT_ERROR_STACK • This built-in returns the error stack in the current session. – Possibly more than one error in stack.

• Returns NULL when there is no error. • Returns a string of maximum size 2000 bytes (according to the documentation). • Oracle recommends you use this instead of SQLERRM, to reduce the chance of truncation. errorstack.sql big_error_stack.sql Copyright 2013 Feuerstein and Associates

Page 99


DBMS_UTILITY.FORMAT_ERROR_BACKTRACE • The backtrace function (new to 10.2) answers the question: "Where was my error raised? – Prior to 10.2, you could not get this information from within PL/SQL.

• Call it whenever you are logging an error. • When you re-raise your exception (RAISE;) or raise a different exception, subsequent BACKTRACE calls will point to that line. – So before a re-raise, call BACKTRACE and store that information to avoid losing the original line number.


backtrace.sql bt.pkg

Page 100


Logging Errors • We usually, but not always, want to write error information out to a log table. How's this? WHEN NO_DATA_FOUND THEN l_code := SQLCODE; INSERT INTO errlog VALUES ( l_code , 'No company for id ' || TO_CHAR ( v_id ) , 'fixdebt', SYSDATE, USER ); WHEN OTHERS THEN l_code := SQLCODE; l_errm := SQLERRM; INSERT INTO errlog VALUES (l_code, l_errm, 'fixdebt', SYSDATE, USER ); RAISE; END;

 It's easy to "read" but only because it exposes the logging mechanism. Copyright 2013 Feuerstein and Associates

Page 101


Hide how and what you log • Don't call RAISE_APPLICATION_ERROR. • Don't explicitly insert into log table or write to file. • Don't call all those useful built-in functions in each handler. • Do use a generic and shared error management utility. – Check out Quest Error Manager at PL/SQL Obsession for an example. WHEN NO_DATA_FOUND THEN q$error_manager.register_error ( text_in => 'No company for id ' || TO_CHAR ( v_id )); WHEN OTHERS THEN q$error_manager.raise_unanticipated ( name1_in => 'COMPANY_ID', value1_in => v_id); END;


qem_demo.sql

Page 102


Execution Tracing (Instrumentation) • Tracing, also known as instrumentation, is an important technique for diagnosing production issues and quickly resolving them. – Also helpful as you build your code.

• Sadly, too many developers do not sufficiently instrument their code and too many use the wrong instrument. – You know what I'm talking about: DBMS_OUTPUT.PUT_LINE! Copyright 2013 Feuerstein and Associates

Page 103


Doesn't Oracle do tracing for us? • Sure, but we often need to retrieve additional, application-specific information from our code while running. – Especially in production.

• DBMS_OUTPUT.PUT_LINE is the "default" tracing mechanism – and should never appear in your application code. – You are exposing the trace/display mechanism (hard-coding). – Too many drawbacks, too little flexibility. Copyright 2013 Feuerstein and Associates

Page 104


Tips on Instrumenting Code • Don't remove trace calls when you are "done" writing your program. – You will need it for production problem diagnoses.

• Minimize overhead when tracing is disabled. – Put calls to the trace program inside a faster Boolean check.

• Make it possible for users to enable tracing to gather production data.


Page 105


Some Tracing Options • Trace packages in demo.zip include: – sf_trace: my latest, simple and sufficient tracer – watch.pkg: "watch" the action, ability to direct trace information to a variety of targets.

• logger utility by Tyler Muth – Very popular with APEX developers; check http://goo.gl/5jrkT for update on project.

• DBMS_APPLICATION_INFO – Writes to V$ views, good for long-running operations sf_trace*.* watch.pkg dbms_application_info_demo.sql Copyright 2013 Feuerstein and Associates

Page 106


Writing SQL in PL/SQL • The most critical aspect of our programs. • SQL statements directly reflect our business models. – And those models are always changing.

• SQL statements cause most of the performance problems in our applications. – Tuning SQL and the way that SQL is called in PL/SQL overwhelms all other considerations.

• Many runtime errors in applications result from integrity and check constraints on tables. Copyright 2013 Feuerstein and Associates

Page 107


The fundamental problem with SQL in PL/SQL • We take it entirely for granted. – Why not? It's so easy to write SQL in PL/SQL!

Order Entry Application

The Backend Order Table

Item Table

Customer • We don't set rules on Table how, when and where SQL should be written in The result? Slow, buggy PL/SQL. code that is difficult to

optimize and maintain. Copyright 2013 Feuerstein and Associates

Page 108


Set Standards for Writing SQL • Check Bryn Llewellyn's "Doing SQL in PL/SQL" whitepaper on OTN for many ideas. • Use a data encapsulation layer. – Hide SQL statements behind an interface.

• Hide all tables in schemas users cannot access. • Qualify every identifier in SQL statements. • Dot-qualify references to Oracle-supplied objects. • And some best practices for dynamic SQL Copyright 2013 Feuerstein and Associates

Page 109


Data Encapsulation: hide all tables in schemas users cannot access. • A fundamental issue of control and security. • Do not allow users to connect to any schema that contains tables. – Simply too risky.

• Define tables in other schemas. • Grant access via privileges, mostly via EXECUTE on packages that maintain the tables (data encapsulation/API). Copyright 2013 Feuerstein and Associates

Page 110


Architecture with "inaccessible schema" • The OE Data schemas own all tables. • The OE Code schema owns the code and has directly granted privileges on the tables. • User schemas have execute authority granted on the code. OE Code

Sam_Sales

Order_Mgt Place

Close Old Orders

Cancel

OE Data

Orders


X

But we also need to extend these controls to developers. Because every SQL statement you write is a hard-coding!

Cannot access table directly.

Page 111


SQL statements as hard-codings • I suggest that every SQL statement you will ever write is a hard-coding. Consider.... • I need to write a complex query to return HR data for a report. SELECT . . . FROM employees, departments, locations WHERE . . . (a page full of complex conditions)

• And Joe needs to use that same query in his business rule procedure. And so on... • And then the three way join turns into a four way join – and we have to find all occurrences of this query. A very tough thing to do! Copyright 2013 Feuerstein and Associates

PagePage 112112


What to do about SQL hard coding • Of course, you have to (and should) write SQL statements in your PL/SQL code. – PL/SQL is, in fact, the best place for SQL.

• But we should be very careful about where, when and how we write these statements. – Follow the principles; they are your guide. – Don't repeat anything!

• The best approach: hide SQL statements inside a data access layer. Copyright 2013 Feuerstein and Associates

Page 113


SQL as a Service • Think of SQL as a service that is provided to you, not something you write. – Or if you write it, you put it somewhere so that it can be easily found, reused, and maintained.

 This service consists of views and programs defined in the data access layer. – Views hide complex query construction – Packaged APIs – for tables, transactions and business entities


Application Code

Intermediate Layer

Order Table

Item Table

Page 114


With a data access layer, I can... • Change/improve my implementation with minimal impact on my application code. – The underlying data model is constantly changing. – We can depend on Oracle to add new features. – We learn new ways to take advantage of PL/SQL.

• Vastly improve my SQL-related error handling. – Do you handle dup_val_on_index for INSERTs, too_many_rows for SELECT INTOs, etc?

• Greatly increase my productivity – I want to spend as much time as possible implementing business requirements. Copyright 2013 Feuerstein and Associates

11g_frc_demo.sql 11g_emplu.*

Page 115


How to implement data encapsulation • It must be very consistent, well-designed and efficient - or it will not be used. • Best solution: generate as much of the code as possible. – This includes products like APEX and Hibernate that generate lots of their own SQL for you.

• Any custom SQL statements should be written once and placed in a shareable container (usually a package, but also views). Copyright 2013 Feuerstein and Associates

Page 116


Qualify every column and identifier in the SQL statement. • Improves readability. • Avoids potential bugs when variable names match column names. • Minimizes invalidation of dependent program units in Oracle11g. 11g_fgd*.sql

Instead of this.... PROCEDURE IS BEGIN SELECT INTO FROM WHERE

abc (...)

last_name l_name employees employee_id = employee_id_in;


Write this.... PROCEDURE abc (...) IS BEGIN SELECT e.last_name INTO l_name FROM employees e WHERE e.employee_id = abc.employee_id_in;

Page 117


Dot-qualify all references to Oraclesupplied objects with "SYS." • Another annoying, but incontestable recommendation. • If you don't prefix calls to all supplied packages with "SYS.", you are more vulnerable to injection. – More details available in "Best Practices for Dynamic SQL" in the "Dynamic SQL in PL/SQL" series. BEGIN run_dynamic_plsql_block (append_this_in => 'employee_id=101; EXECUTE IMMEDIATE ''CREATE OR REPLACE PACKAGE DBMS_OUTPUT ... ''' ); END;


code_injection.sql

Page 118


Best Practices for Dynamic SQL • Stored programs with dynamic SQL should be defined as AUTHID CURRENT_USER. • Remember that dynamic DDL causes an implicit commit. – Consider making all DDL programs autonomous transactions.

• Always EXECUTE IMMEDIATE a variable, so that you can then display/log/view that variable's value in case of an error. • Avoid concatenation; bind whenever possible. dropwhatever.sp usebinding.sp toomuchbinding.sp useconcat*.* ultrabind.* Copyright 2013 Feuerstein and Associates

Page 119


Conclusions • SQL statements are among the most critical parts of your application. – They change frequently, they consume lots of resources, result in many errors.

• You should have a clearly defined set of guidelines about when, where and how to write SQL. • Most important: Don't repeat SQL statements. – Most good practices will follow more easily if you manage to avoid SQL hard-coding. Copyright 2013 Feuerstein and Associates

Page 120


Never Repeat Anything - Conclusions • Repeat after me: Everything is going to change. • When you hide the mechanics, how you get things done, behind a procedure or function, you are "liberated." – Change the implementation, and you don't need to change all the places in which it is used.

• Back to that same principle: Never Repeat Anything. Aim for Single Point of Definition. Copyright 2013 Feuerstein and Associates

Page 121


Program Socially • Never code alone. – Writing code by yourself results in much buggier code than if you had someone with whom to consult; to ask questions; to help, in turn. – It's OK to ask for help. – Follow the 30 minute rule.

• Automated code review in PL/SQL – Compile-time warnings – Leverage the data dictionary views – PL/Scope Copyright 2013 Feuerstein and Associates

Page 122


Don't be afraid to ask for help. "Predators look for signs of illness or weakness when choosing their prey, so a prey animal needs to appear healthy, or it will be a sure target. By the time they are showing signs of disease, in many instances, the birds have become too weak to be able to disguise it." - From peteducation.com

• Our evolved survival instinct urges us to hide weakness. • On top of that, we software developers are supposed to be really smart. – We are the wizards of modern society.

• Unfortunately, ignorance leads directly to bugs and sub-optimal code. Copyright 2013 Feuerstein and Associates

Page 123


It's OK to say "I don't know. Help me!" • Just thinking about asking for help will often do the trick. • Most people like to be asked to help. – It makes them feel valued. – It strengthens the team as a whole.

• It may not really matter who you ask for help. – If there are no programmers handy, ask your spouse or parent or child to be a sounding board. – Or write an email. By the time you finish writing it, you will likely have found the answer to your problem. – The important thing is to get the issue out of your head. Copyright 2013 Feuerstein and Associates

Page 124


Code Review - Automated and Otherwise • Even if you write your code mostly by yourself, it's extremely important for someone or something to check that code. – Does it make sense? Does it follow standards? Is there a better way to do it?

• Peer code review - in a group or one-on-one is very helpful, but often intimidating. • Automated code review is less personal, might catch many issues the "eyeball" might miss. – Compile time warnings, DD views, PL/Scope Copyright 2013 Feuerstein and Associates

Page 125


Follow the Thirty Minute Rule • We are usually too deeply inside (and part of) the problem to step back and take a fresh look. • If you can't fix a bug in 30 minutes, ask for help. – For "trivial" bugs, "give up" after just a few minutes!

• Senior developers and managers must take the lead. – Ask more junior members for help. Show that you are fallible, that you can learn from anyone and everyone. Copyright 2013 Feuerstein and Associates

Page 126


Warnings help you build better code • Your code compiles without errors. Great, you can run that program! • But does it use the PL/SQL language optimally? • In Oracle 10g, Oracle added a compile-time warnings framework. – Automatically informs you of ways to improve the quality or performance of your code.

• All warnings shown in Error Messages manual, with the PLW prefix. http://docs.oracle.com


Page 127


Enable and Disable Warnings • To use compiler warnings, you must turn them on for session or for a particular program unit. – By default, warnings are disabled.

• Can specify individual warnings or categories. ALTER SESSION [ENABLE | DISABLE |ERROR]: [ALL|SEVERE|INFORMATIONAL|PERFORMANCE|warning_number] REM To enable all warnings in your session: ALTER SESSION SET plsql_warnings = 'enable:all‘; REM If you want to enable warning message number 06002 and all warnings in REM the performance category, and treat 5005 as a "hard" compile error: ALTER PROCEDURE my_procedure SET plsql_warnings = 'enable:06002', 'enable:performance', 'ERROR:05005';


Page 128


Checking for Warnings • The USER_ERRORS data dictionary view shows both "hard" errors and compilation warnings. • Use the SHOW ERRORS command in SQL*Plus. • IDEs will usually display warnings within the edit window. • Or run your own query against USER_ERRORS.


Page 129


Example: check for unreachable code • There may be lines of code that could never, ever execute. SQL> CREATE OR REPLACE PROCEDURE unreachable_code IS 2 x NUMBER := 10; 3 BEGIN 4 IF x = 10 THEN 5 x := 20; 6 ELSE 7 x := 100; -- unreachable code 8 END IF; 9 END unreachable_code; 10 / SP2-0804: Procedure created with compilation warnings

SQL> show err Errors for PROCEDURE UNREACHABLE_CODE: LINE/COL ERROR -------- ------------------------------------7/7 PLW-06002: Unreachable code


plw6002.sql

Page 130


Finally, Oracle warns me of too-large value CREATE OR REPLACE PROCEDURE plw6017 IS c VARCHAR2 (1) := 'abc'; BEGIN

• One big frustration I have had with compiletime warnings is that it did not flag code like you see above. What could be more basic? • This is finally addressed – sort of – in Oracle11g with the PLW-06017 warning. PLW-06017: an operation will raise an exception

plw6017.sql Copyright 2013 Feuerstein and Associates

Page 131


New compile-time warnings in Oracle11g

• PLW-6009: Exception handler does not re-raise an exception. – Doesn't recognize when a subprogram does the raise.

• PLW-7205: warning on mixed use of integer types – Namely, SIMPLE_INTEGER mixed with PLS_INTEGER and BINARY_INTEGER

• PLW-7206: unnecessary assignments – My own warning: I can't get this warning to "fire"!

• Lots of PRAGMA INLINE-related warnings • More feedback on impact of optimization – PLW-6007: Notification that entire subprograms were removed Copyright 2013 Feuerstein and Associates

plw*.sql files

Page 132


Treating a warning as "hard" compile error • You might identify a warning that reflects such bad coding practices, that you want to ensure it never makes its way into production code. – Just set the warning as an error and stop the use of that program "in its tracks."

• "Function does not return value" is a prime example. ALTER SESSION SET PLSQL_WARNINGS='ERROR:5005' /

plw5005.sql Copyright 2013 Feuerstein and Associates

Page 133


Watch out for "false negatives" and "nuisances" warnings • The check for unreachable code is not very useful prior to Oracle11g. – Shows as unreachable code that is removed by the optimizer.

• You might be overwhelmed by warnings about which you don't really care. – Show us "missing AUTHID clause".


Page 134


Code analysis with data dictionary views • Oracle's data dictionary provides access to many views containing information about our stored program units. With them we can... – Analyze objects defined in the database – Analyze source code for contents and patterns – Analyze program unit structure and header – Check compile-time settings of program units

• With a good set of scripts you can easily and productively analyze your code. Copyright 2013 Feuerstein and Associates

Page 135


Analyzing source code • ALL_SOURCE – Write queries against source code to identify violations of coding standards. – Which programs contain/exclude particular strings?

• Use with other data dictionary views and utilities that reference source code. – DBMS_UTILITY.FORMAT_CALL_STACK – Profiler data

• ALL_IDENTIFIERS (Oracle11g) –PL/Scope – Analyze all references to identifiers (named elements) – covered in separate lesson. Copyright 2013 Feuerstein and Associates

all_source.sql valstds.pks/pkb package_analyzer.pks/pkb

Page 136


Analyzing program unit structure/header • Source code is handy, but also "freeform" text. – The more structured the data, the better.

• ALL_PROCEDURES – Information about every subprogram you can execute – Missing some information (the type of subprogram)

• ALL_ARGUMENTS – Information about every argument of every subprogram you can execute – Rich resource of information, not that well designed. – Can use it to figure out type of subprogram – DBMS_DESCRIBE offers another access path to more or less the same data show_authid.sql show_deterministic.sql Copyright 2013 Feuerstein and Associates

all_arguments.sql show_all_arguments*.* show_procs_with_parm_types.sql is_function.sf Page 137


Compile time settings for program units • ALL_PLSQL_OBJECT_SETTINGS • Stores information about compile-time characteristics of program units. – Optimization level – Code type: NATIVE or INTERPRETED – Debug settings – Compile-time warnings – Conditional compilation flags – PL/Scope settings Copyright 2013 Feuerstein and Associates

whats_not_optimal.sql show_non_default_object_settings.sql

Page 138


PL/Scope: powerful code analysis tool • A compiler-driven tool that collects information about identifiers and stores it in data dictionary views. – Introduced in Oracle Database 11g

• Use PL/Scope to answer questions like: – Where is a variable assigned a value in a program? – What variables are declared inside a given program? – Which programs call another program (that is, you can get down to a subprogram in a package)? – Find the type of a variable from its declaration. Copyright 2013 Feuerstein and Associates

Page 139


Getting Started with PL/Scope ALTER SESSION SET plscope_settings='IDENTIFIERS:ALL'

• PL/Scope must be enabled; it is off by default. • When your program is compiled, information about all identifiers are written to the ALL_IDENTIFIERS view. • You then query the contents of the view to get information about your code. • Check the ALL_PLSQL_OBJECT_SETTINGS view for the PL/Scope setting of a particular program unit. Copyright 2013 Feuerstein and Associates

Page 140


Key Columns in ALL_IDENTIFIERS • TYPE – The type of identifier (VARIABLE, CONSTANT, etc.)

• USAGE – The way the identifier is used (DECLARATION, ASSIGNMENT, etc.)

• LINE and COL – Line and column within line in which the identifier is found

• SIGNATURE – Unique value for an identifier. Especially helpful when distinguishing between overloadings of a subprogram or "connecting" subprogram declarations in package with definition in package body.

• USAGE_ID and USAGE_CONTEXT_ID – Reveal hierarchy of identifiers in a program unit Copyright 2013 Feuerstein and Associates

Page 141


Start with some simple examples • Show all the identifiers in a program unit • Show all variables declared in a subprogram (not at package level) • Show all variables declared in the package specifications • Show the locations where a variable could be modified


plscope_demo_setup.sql plscope_all_idents.sql plscope_var_declares.sql plscope_gvar_declares.sql plscope_var_changes.sql

Page 142


More advanced examples • Find exceptions that are defined but never raised • Show the hierarchy of identifiers in a program unit • Validate naming conventions with PL/Scope

plscope_unused_exceptions.sql plscope_hierarchy.sql plscope_naming_conventions.sql


Page 143


PL/Scope Helper Utilities • Clearly, "data mining" in ALL_IDENTIFIERS can get very complicated. • Suggestions for putting PL/Scope to use: – Build views to hide some of the complexity. – Build packages to provide high-level subprograms to perform specific actions.

plscope_helper_setup.sql plscope_helper.pkg


Page 144


Break Down Coding Isolation • Ask for help, and offer help to others. • Participate willingly in code review. – Your code and your skills will both benefit.

• Automate code evaluation as much as possible. – It's always been possible with the DD views, but now with compile-time warnings and PL/Scope, the quality of feedback is much higher.


Page 145


Make the Most of the Best of Oracle PL/SQL! • This language is not evolving very rapidly these days (less change than in SQL). • Make sure that you are aware of key new (and existing) features, and put them to use. • Always prioritize the maintainability of your code. – It's going to be around for YEARS to come!


Page 146


Websites for PL/SQL Developers www.plsqlchallenge.com Daily PL/SQL quiz with weekly and monthly prizes

www.plsqlchannel.com 27+ hours of detailed video training on Oracle PL/SQL

www.stevenfeuerstein.com Monthly PL/SQL newsletter

www.toadworld.com/SF Quest Software-sponsored portal for PL/SQL developers


Page 147

Making the Most of the Best of Oracle PL/SQL - Toad World

Making the Most of the Best of Oracle PL/SQL - Toad World

Suggest Documents

The Best of Oracle PL/SQL - Toad World

Oracle Database 12c New Features.pdf - Toad World

Toad® for Oracle Tips and Tricks - Toad World

Making the Most of FeedBlitz

Making the Most of FeedBlitz

Making the Most of FeedBlitz

Making the Most of World Natural Heritage ... - Semantic Scholar

Toad Getting Started Guide.pdf - Toad World

Making the most of conference attendance

Making the Most of Partnerships - Healio

Making the most of Brexit - Demos

Making the most of flexible learning spaces

Making the most of every drop - NFU

Making the most of Brexit - Demos

making the most of every bite - HSE

Making the most of 'Tomorrow' - Transition Network

Making the most of 'Tomorrow' - Transition Network

Making the most of your conference poster

Making the Most of Your Textbook--Powerpoint

Making the Most of MOSIS - FreeLists

Making the Most of Your Special Event

Making the Most of Foreign Factories.

Making the Most of On-Line Recruiting

Making the most of study abroad: