Sep 18, 2012 ... throw it into a db. □ csvlog makes this easy. □ automated reporting. □ roll your
own. □ generic: splunk, logstash, etc. □ pg-specific: pgfouine ...
Sep 16, 2011 - Tap Revenge 2. / âCan we do this in 3 days?â âYes. ... subscriptions, and we expanded our push service to Android and. BlackBerry. 5 billion ... doesn't require much in the way of data storage -- what matters is the number of ...
Managing rights in PostgreSQL ..... In this talk : • How rights works in PostgreSQL
from connection to SQL statement execution. • How to manage roles and rights.
PostgreSQL strengths for data warehousing. ⢠Data loading on PostgreSQL. ⢠Analysis and reporting of a PostgreSQL DW
Request ticket. 2. Return ticket. 3. Present ticket. Page 18. PAM. â Provided by OS. â Can do password, LDAP, etc. â
Sep 16, 2011 - Tap Revenge 2. / âCan we do this in 3 days?â âYes. ... subscriptions, and we expanded our push service to Android and. BlackBerry. 5 billion ... doesn't require much in the way of data storage -- what matters is the number of ...
Case of one PostgreSQL node data warehouse. – This talk does not directly
address multi-node distribution of data. • Limitations on disk usage and
concurrent ...
Feb 11, 2017 - version, there was a devoted support from my family. I would like ... 1990-2000 Digital Equipment Corporation Japan / Compaq Computer Japan ...... write(5, "\0\0\0\0\210\253h\262\0\0\0\0\0\0\0\0\0\200\365?\0"..., 8192) = 8192.
Remote Authentication Dial In User. Service. â Simple single-packet UDP service. â Original use-case: ISP dialup. â
by The PostgreSQL Global Development Group. Copyright ... This document
contains assorted information that can be of use to PostgreSQL developers.
PostgreSQL 8.2.22 Documentation. The PostgreSQL Global Development ...
PostgreSQL 8.2.22 Documentation by The PostgreSQL Global Development
Group.
Setting up the database. â¢createdb ... Custom Types dict. Tuesday, March 13 ... Change/Add featu
... be seen as an application development framework with the PostgreSQL Server ... In this age of modern era the use of
Dedicated Instances. ⢠Dedicates hardware ... patterns. ⢠Doesn't change the tenancy of the servers at all. ... You
Aug 4, 2013 ... PostgreSQL runs on all major operating systems, including Linux, UNIX ... This
tutorial will give you quick start with PostgreSQL and make you ...
PostgreSQL 8.3.23 Documentation. The PostgreSQL Global Development ...
PostgreSQL 8.3.23 Documentation by The PostgreSQL Global Development
Group.
Transaction Processing in PostgreSQL. Outline. Introduction. • What is a
transaction? User's view. • Multi-version concurrency control. Implementation.
How PostgreSQL handles concurrent operations. – Snapshots. – Conflict
resolution rules. How to write application queries to get the right results without.
Apr 11, 2006 - The python API described for PostgreSQL will also work for any database which has a. DB-API compliant ada
How to write application queries to get the right results without sacrificing performance ... SELECT SUM(balance) FROM a
Feb 17, 2016 - Oracle database logo should be there... but as of this page it can't be: ...Remember that you are general
... will show you that PostgreSQL is so much more than a database server In fact it could even be seen as an application
13.3. Explicit Locking ................................................................................................... 362 13.3.1. Table-level Locks ..................................................................................... 362 13.3.2. Row-level Locks ...................................................................................... 364 13.3.3. Deadlocks................................................................................................. 365 13.3.4. Advisory Locks........................................................................................ 366 13.4. Data Consistency Checks at the Application Level.............................................. 367 13.4.1. Enforcing Consistency With Serializable Transactions ........................... 367 13.4.2. Enforcing Consistency With Explicit Blocking Locks ............................ 367 13.5. Caveats.................................................................................................................. 368 13.6. Locking and Indexes............................................................................................. 368 14. Performance Tips .............................................................................................................. 370 14.1. Using EXPLAIN .................................................................................................... 370 14.1.1. EXPLAIN Basics ....................................................................................... 370 14.1.2. EXPLAIN ANALYZE ................................................................................. 376 14.1.3. Caveats ..................................................................................................... 379 14.2. Statistics Used by the Planner .............................................................................. 380 14.3. Controlling the Planner with Explicit JOIN Clauses............................................ 381 14.4. Populating a Database .......................................................................................... 383 14.4.1. Disable Autocommit ................................................................................ 383 14.4.2. Use COPY .................................................................................................. 383 14.4.3. Remove Indexes ....................................................................................... 384 14.4.4. Remove Foreign Key Constraints ............................................................ 384 14.4.5. Increase maintenance_work_mem ........................................................ 384 14.4.6. Increase checkpoint_segments .......................................................... 384 14.4.7. Disable WAL Archival and Streaming Replication ................................. 385 14.4.8. Run ANALYZE Afterwards........................................................................ 385 14.4.9. Some Notes About pg_dump ................................................................... 385 14.5. Non-Durable Settings ........................................................................................... 386 III. Server Administration ............................................................................................................. 388 15. Installation from Source Code .......................................................................................... 390 15.1. Short Version ........................................................................................................ 390 15.2. Requirements........................................................................................................ 390 15.3. Getting The Source............................................................................................... 392 15.4. Installation Procedure........................................................................................... 392 15.5. Post-Installation Setup.......................................................................................... 402 15.5.1. Shared Libraries ....................................................................................... 402 15.5.2. Environment Variables............................................................................. 403 15.6. Supported Platforms ............................................................................................. 403 15.7. Platform-specific Notes ........................................................................................ 404 15.7.1. AIX .......................................................................................................... 404 15.7.1.1. GCC Issues .................................................................................. 405 15.7.1.2. Unix-Domain Sockets Broken..................................................... 405 15.7.1.3. Internet Address Issues................................................................ 405 15.7.1.4. Memory Management ................................................................. 406 References and Resources ................................................................ 407 15.7.2. Cygwin..................................................................................................... 407 15.7.3. HP-UX ..................................................................................................... 408 15.7.4. IRIX ......................................................................................................... 409 15.7.5. MinGW/Native Windows ........................................................................ 409 15.7.5.1. Collecting Crash Dumps on Windows ........................................ 410 15.7.6. SCO OpenServer and SCO UnixWare..................................................... 410
x
15.7.6.1. Skunkware ................................................................................... 410 15.7.6.2. GNU Make .................................................................................. 410 15.7.6.3. Readline....................................................................................... 411 15.7.6.4. Using the UDK on OpenServer................................................... 411 15.7.6.5. Reading the PostgreSQL Man Pages........................................... 411 15.7.6.6. C99 Issues with the 7.1.1b Feature Supplement ......................... 411 15.7.6.7. Threading on UnixWare .............................................................. 412 15.7.7. Solaris ...................................................................................................... 412 15.7.7.1. Required Tools ............................................................................ 412 15.7.7.2. Problems with OpenSSL ............................................................. 412 15.7.7.3. configure Complains About a Failed Test Program .................... 412 15.7.7.4. 64-bit Build Sometimes Crashes ................................................. 413 15.7.7.5. Compiling for Optimal Performance........................................... 413 15.7.7.6. Using DTrace for Tracing PostgreSQL ....................................... 413 16. Installation from Source Code on Windows ..................................................................... 414 16.1. Building with Visual C++ or the Microsoft Windows SDK................................. 414 16.1.1. Requirements ........................................................................................... 415 16.1.2. Special Considerations for 64-bit Windows ............................................ 416 16.1.3. Building ................................................................................................... 417 16.1.4. Cleaning and Installing ............................................................................ 417 16.1.5. Running the Regression Tests .................................................................. 418 16.1.6. Building the Documentation .................................................................... 418 16.2. Building libpq with Visual C++ or Borland C++ ................................................. 418 16.2.1. Generated Files ........................................................................................ 419 17. Server Setup and Operation .............................................................................................. 420 17.1. The PostgreSQL User Account ............................................................................ 420 17.2. Creating a Database Cluster ................................................................................. 420 17.2.1. Use of Secondary File Systems................................................................ 421 17.2.2. Use of Network File Systems .................................................................. 421 17.3. Starting the Database Server................................................................................. 422 17.3.1. Server Start-up Failures ........................................................................... 423 17.3.2. Client Connection Problems .................................................................... 424 17.4. Managing Kernel Resources................................................................................. 425 17.4.1. Shared Memory and Semaphores ............................................................ 425 17.4.2. Resource Limits ....................................................................................... 430 17.4.3. Linux Memory Overcommit .................................................................... 431 17.5. Shutting Down the Server..................................................................................... 432 17.6. Upgrading a PostgreSQL Cluster ......................................................................... 433 17.6.1. Upgrading Data via pg_dump.................................................................. 434 17.6.2. Non-Dump Upgrade Methods.................................................................. 435 17.7. Preventing Server Spoofing .................................................................................. 435 17.8. Encryption Options............................................................................................... 436 17.9. Secure TCP/IP Connections with SSL ................................................................. 437 17.9.1. Using Client Certificates .......................................................................... 438 17.9.2. SSL Server File Usage ............................................................................. 438 17.9.3. Creating a Self-signed Certificate ............................................................ 439 17.10. Secure TCP/IP Connections with SSH Tunnels ................................................. 439 17.11. Registering Event Log on Windows ................................................................... 440 18. Server Configuration ......................................................................................................... 442 18.1. Setting Parameters ................................................................................................ 442 18.1.1. Parameter Names and Values................................................................... 442 18.1.2. Setting Parameters via the Configuration File ......................................... 442
xi
18.1.3. Other Ways to Set Parameters.................................................................. 442 18.1.4. Examining Parameter Settings................................................................. 443 18.1.5. Configuration File Includes...................................................................... 443 18.2. File Locations ....................................................................................................... 445 18.3. Connections and Authentication........................................................................... 446 18.3.1. Connection Settings ................................................................................. 446 18.3.2. Security and Authentication..................................................................... 448 18.4. Resource Consumption......................................................................................... 450 18.4.1. Memory.................................................................................................... 450 18.4.2. Disk .......................................................................................................... 451 18.4.3. Kernel Resource Usage............................................................................ 452 18.4.4. Cost-based Vacuum Delay ....................................................................... 452 18.4.5. Background Writer................................................................................... 453 18.4.6. Asynchronous Behavior........................................................................... 454 18.5. Write Ahead Log .................................................................................................. 455 18.5.1. Settings..................................................................................................... 455 18.5.2. Checkpoints.............................................................................................. 458 18.5.3. Archiving ................................................................................................. 459 18.6. Replication............................................................................................................ 459 18.6.1. Sending Server(s)..................................................................................... 460 18.6.2. Master Server ........................................................................................... 460 18.6.3. Standby Servers ....................................................................................... 461 18.7. Query Planning..................................................................................................... 463 18.7.1. Planner Method Configuration................................................................. 463 18.7.2. Planner Cost Constants ............................................................................ 464 18.7.3. Genetic Query Optimizer......................................................................... 465 18.7.4. Other Planner Options.............................................................................. 466 18.8. Error Reporting and Logging ............................................................................... 467 18.8.1. Where To Log .......................................................................................... 467 18.8.2. When To Log ........................................................................................... 470 18.8.3. What To Log ............................................................................................ 472 18.8.4. Using CSV-Format Log Output ............................................................... 475 18.9. Run-time Statistics................................................................................................ 476 18.9.1. Query and Index Statistics Collector ....................................................... 476 18.9.2. Statistics Monitoring................................................................................ 477 18.10. Automatic Vacuuming ........................................................................................ 478 18.11. Client Connection Defaults ................................................................................ 479 18.11.1. Statement Behavior ................................................................................ 479 18.11.2. Locale and Formatting ........................................................................... 483 18.11.3. Other Defaults........................................................................................ 484 18.12. Lock Management .............................................................................................. 485 18.13. Version and Platform Compatibility................................................................... 486 18.13.1. Previous PostgreSQL Versions .............................................................. 486 18.13.2. Platform and Client Compatibility......................................................... 488 18.14. Error Handling.................................................................................................... 488 18.15. Preset Options..................................................................................................... 489 18.16. Customized Options ........................................................................................... 490 18.17. Developer Options .............................................................................................. 490 18.18. Short Options...................................................................................................... 493 19. Client Authentication ........................................................................................................ 495 19.1. The pg_hba.conf File........................................................................................ 495 19.2. User Name Maps .................................................................................................. 501
35.4.1. Arguments for SQL Functions................................................................. 886 35.4.2. SQL Functions on Base Types ................................................................. 887 35.4.3. SQL Functions on Composite Types ....................................................... 888 35.4.4. SQL Functions with Output Parameters .................................................. 891 35.4.5. SQL Functions with Variable Numbers of Arguments ............................ 892 35.4.6. SQL Functions with Default Values for Arguments ................................ 893 35.4.7. SQL Functions as Table Sources ............................................................. 894 35.4.8. SQL Functions Returning Sets ................................................................ 894 35.4.9. SQL Functions Returning TABLE ............................................................ 897 35.4.10. Polymorphic SQL Functions ................................................................. 897 35.4.11. SQL Functions with Collations.............................................................. 899 35.5. Function Overloading ........................................................................................... 899 35.6. Function Volatility Categories .............................................................................. 900 35.7. Procedural Language Functions ........................................................................... 902 35.8. Internal Functions................................................................................................. 902 35.9. C-Language Functions.......................................................................................... 902 35.9.1. Dynamic Loading..................................................................................... 903 35.9.2. Base Types in C-Language Functions...................................................... 904 35.9.3. Version 0 Calling Conventions ................................................................ 906 35.9.4. Version 1 Calling Conventions ................................................................ 909 35.9.5. Writing Code............................................................................................ 911 35.9.6. Compiling and Linking Dynamically-loaded Functions.......................... 912 35.9.7. Composite-type Arguments ..................................................................... 914 35.9.8. Returning Rows (Composite Types) ........................................................ 916 35.9.9. Returning Sets.......................................................................................... 917 35.9.10. Polymorphic Arguments and Return Types ........................................... 922 35.9.11. Transform Functions .............................................................................. 924 35.9.12. Shared Memory and LWLocks .............................................................. 924 35.9.13. Using C++ for Extensibility................................................................... 925 35.10. User-defined Aggregates .................................................................................... 925 35.11. User-defined Types ............................................................................................. 927 35.12. User-defined Operators....................................................................................... 931 35.13. Operator Optimization Information.................................................................... 932 35.13.1. COMMUTATOR .......................................................................................... 932 35.13.2. NEGATOR ................................................................................................ 933 35.13.3. RESTRICT .............................................................................................. 933 35.13.4. JOIN ....................................................................................................... 934 35.13.5. HASHES................................................................................................... 934 35.13.6. MERGES................................................................................................... 935 35.14. Interfacing Extensions To Indexes...................................................................... 936 35.14.1. Index Methods and Operator Classes .................................................... 936 35.14.2. Index Method Strategies ........................................................................ 937 35.14.3. Index Method Support Routines ............................................................ 938 35.14.4. An Example ........................................................................................... 941 35.14.5. Operator Classes and Operator Families................................................ 943 35.14.6. System Dependencies on Operator Classes ........................................... 946 35.14.7. Ordering Operators ................................................................................ 946 35.14.8. Special Features of Operator Classes..................................................... 947 35.15. Packaging Related Objects into an Extension .................................................... 948 35.15.1. Extension Files....................................................................................... 949 35.15.2. Extension Relocatability ........................................................................ 950 35.15.3. Extension Configuration Tables ............................................................. 951
xx
35.15.4. Extension Updates ................................................................................. 952 35.15.5. Extension Example ................................................................................ 953 35.16. Extension Building Infrastructure ...................................................................... 954 36. Triggers ............................................................................................................................. 957 36.1. Overview of Trigger Behavior.............................................................................. 957 36.2. Visibility of Data Changes.................................................................................... 959 36.3. Writing Trigger Functions in C ............................................................................ 960 36.4. A Complete Trigger Example............................................................................... 962 37. Event Triggers ................................................................................................................... 966 37.1. Overview of Event Trigger Behavior ................................................................... 966 37.2. Event Trigger Firing Matrix ................................................................................. 966 37.3. Writing Event Trigger Functions in C .................................................................. 969 37.4. A Complete Event Trigger Example .................................................................... 970 38. The Rule System ............................................................................................................... 972 38.1. The Query Tree..................................................................................................... 972 38.2. Views and the Rule System .................................................................................. 974 38.2.1. How SELECT Rules Work ........................................................................ 974 38.2.2. View Rules in Non-SELECT Statements .................................................. 979 38.2.3. The Power of Views in PostgreSQL ........................................................ 980 38.2.4. Updating a View....................................................................................... 980 38.3. Materialized Views ............................................................................................... 981 38.4. Rules on INSERT, UPDATE, and DELETE ............................................................. 983 38.4.1. How Update Rules Work ......................................................................... 984 38.4.1.1. A First Rule Step by Step............................................................ 985 38.4.2. Cooperation with Views........................................................................... 988 38.5. Rules and Privileges ............................................................................................. 993 38.6. Rules and Command Status.................................................................................. 995 38.7. Rules Versus Triggers ........................................................................................... 996 39. Procedural Languages ....................................................................................................... 999 39.1. Installing Procedural Languages .......................................................................... 999 40. PL/pgSQL - SQL Procedural Language ......................................................................... 1002 40.1. Overview ............................................................................................................ 1002 40.1.1. Advantages of Using PL/pgSQL ........................................................... 1002 40.1.2. Supported Argument and Result Data Types ......................................... 1002 40.2. Structure of PL/pgSQL....................................................................................... 1003 40.3. Declarations........................................................................................................ 1005 40.3.1. Declaring Function Parameters.............................................................. 1005 40.3.2. ALIAS ..................................................................................................... 1008 40.3.3. Copying Types ....................................................................................... 1008 40.3.4. Row Types.............................................................................................. 1008 40.3.5. Record Types ......................................................................................... 1009 40.3.6. Collation of PL/pgSQL Variables .......................................................... 1009 40.4. Expressions......................................................................................................... 1011 40.5. Basic Statements................................................................................................. 1011 40.5.1. Assignment ............................................................................................ 1011 40.5.2. Executing a Command With No Result ................................................. 1012 40.5.3. Executing a Query with a Single-row Result......................................... 1012 40.5.4. Executing Dynamic Commands ............................................................ 1013 40.5.5. Obtaining the Result Status.................................................................... 1016 40.5.6. Doing Nothing At All ............................................................................ 1017 40.6. Control Structures............................................................................................... 1018 40.6.1. Returning From a Function.................................................................... 1018
xxi
40.6.1.1. RETURN ...................................................................................... 1018 40.6.1.2. RETURN NEXT and RETURN QUERY ......................................... 1018 40.6.2. Conditionals ........................................................................................... 1020 40.6.2.1. IF-THEN .................................................................................... 1020 40.6.2.2. IF-THEN-ELSE ......................................................................... 1021 40.6.2.3. IF-THEN-ELSIF ....................................................................... 1021 40.6.2.4. Simple CASE .............................................................................. 1022 40.6.2.5. Searched CASE........................................................................... 1023 40.6.3. Simple Loops ......................................................................................... 1023 40.6.3.1. LOOP .......................................................................................... 1023 40.6.3.2. EXIT .......................................................................................... 1024 40.6.3.3. CONTINUE.................................................................................. 1024 40.6.3.4. WHILE ........................................................................................ 1025 40.6.3.5. FOR (Integer Variant) ................................................................. 1025 40.6.4. Looping Through Query Results ........................................................... 1026 40.6.5. Looping Through Arrays ....................................................................... 1027 40.6.6. Trapping Errors ...................................................................................... 1028 40.6.6.1. Obtaining information about an error........................................ 1030 40.7. Cursors................................................................................................................ 1031 40.7.1. Declaring Cursor Variables .................................................................... 1031 40.7.2. Opening Cursors .................................................................................... 1032 40.7.2.1. OPEN FOR query ...................................................................... 1032 40.7.2.2. OPEN FOR EXECUTE ................................................................ 1033 40.7.2.3. Opening a Bound Cursor........................................................... 1033 40.7.3. Using Cursors......................................................................................... 1034 40.7.3.1. FETCH ........................................................................................ 1034 40.7.3.2. MOVE .......................................................................................... 1034 40.7.3.3. UPDATE/DELETE WHERE CURRENT OF .................................. 1035 40.7.3.4. CLOSE ........................................................................................ 1035 40.7.3.5. Returning Cursors ..................................................................... 1035 40.7.4. Looping Through a Cursor’s Result....................................................... 1037 40.8. Errors and Messages........................................................................................... 1037 40.9. Trigger Procedures ............................................................................................. 1039 40.9.1. Triggers on data changes........................................................................ 1039 40.9.2. Triggers on events .................................................................................. 1046 40.10. PL/pgSQL Under the Hood .............................................................................. 1047 40.10.1. Variable Substitution............................................................................ 1047 40.10.2. Plan Caching ........................................................................................ 1049 40.11. Tips for Developing in PL/pgSQL.................................................................... 1050 40.11.1. Handling of Quotation Marks .............................................................. 1051 40.12. Porting from Oracle PL/SQL............................................................................ 1052 40.12.1. Porting Examples ................................................................................. 1053 40.12.2. Other Things to Watch For................................................................... 1058 40.12.2.1. Implicit Rollback after Exceptions.......................................... 1058 40.12.2.2. EXECUTE .................................................................................. 1059 40.12.2.3. Optimizing PL/pgSQL Functions............................................ 1059 40.12.3. Appendix.............................................................................................. 1059 41. PL/Tcl - Tcl Procedural Language.................................................................................. 1062 41.1. Overview ............................................................................................................ 1062 41.2. PL/Tcl Functions and Arguments....................................................................... 1062 41.3. Data Values in PL/Tcl......................................................................................... 1063 41.4. Global Data in PL/Tcl ........................................................................................ 1064
SPI_execute_plan_with_paramlist..................................................................... 1120 SPI_execp........................................................................................................... 1121 SPI_cursor_open ................................................................................................ 1122 SPI_cursor_open_with_args .............................................................................. 1124 SPI_cursor_open_with_paramlist ...................................................................... 1126 SPI_cursor_find.................................................................................................. 1127 SPI_cursor_fetch................................................................................................ 1128 SPI_cursor_move ............................................................................................... 1129 SPI_scroll_cursor_fetch..................................................................................... 1130 SPI_scroll_cursor_move .................................................................................... 1131 SPI_cursor_close................................................................................................ 1132 SPI_keepplan ..................................................................................................... 1133 SPI_saveplan...................................................................................................... 1134 44.2. Interface Support Functions ............................................................................... 1135 SPI_fname.......................................................................................................... 1135 SPI_fnumber ...................................................................................................... 1136 SPI_getvalue ...................................................................................................... 1137 SPI_getbinval ..................................................................................................... 1138 SPI_gettype ........................................................................................................ 1139 SPI_gettypeid..................................................................................................... 1140 SPI_getrelname .................................................................................................. 1141 SPI_getnspname................................................................................................. 1142 44.3. Memory Management ........................................................................................ 1143 SPI_palloc .......................................................................................................... 1143 SPI_repalloc....................................................................................................... 1145 SPI_pfree............................................................................................................ 1146 SPI_copytuple .................................................................................................... 1147 SPI_returntuple .................................................................................................. 1148 SPI_modifytuple ................................................................................................ 1149 SPI_freetuple...................................................................................................... 1151 SPI_freetuptable................................................................................................. 1152 SPI_freeplan....................................................................................................... 1153 44.4. Visibility of Data Changes.................................................................................. 1154 44.5. Examples ............................................................................................................ 1154 45. Background Worker Processes........................................................................................ 1158 VI. Reference................................................................................................................................. 1160 I. SQL Commands................................................................................................................. 1162 ABORT........................................................................................................................ 1163 ALTER AGGREGATE................................................................................................ 1165 ALTER COLLATION ................................................................................................. 1167 ALTER CONVERSION.............................................................................................. 1169 ALTER DATABASE ................................................................................................... 1171 ALTER DEFAULT PRIVILEGES .............................................................................. 1173 ALTER DOMAIN ....................................................................................................... 1176 ALTER EVENT TRIGGER ........................................................................................ 1180 ALTER EXTENSION ................................................................................................. 1181 ALTER FOREIGN DATA WRAPPER ....................................................................... 1184 ALTER FOREIGN TABLE......................................................................................... 1186 ALTER FUNCTION ................................................................................................... 1190 ALTER GROUP .......................................................................................................... 1193 ALTER INDEX ........................................................................................................... 1195
xxiv
ALTER LANGUAGE.................................................................................................. 1197 ALTER LARGE OBJECT........................................................................................... 1198 ALTER MATERIALIZED VIEW............................................................................... 1199 ALTER OPERATOR ................................................................................................... 1201 ALTER OPERATOR CLASS...................................................................................... 1203 ALTER OPERATOR FAMILY ................................................................................... 1204 ALTER ROLE ............................................................................................................. 1208 ALTER RULE ............................................................................................................. 1212 ALTER SCHEMA ....................................................................................................... 1213 ALTER SEQUENCE................................................................................................... 1214 ALTER SERVER......................................................................................................... 1217 ALTER TABLE ........................................................................................................... 1219 ALTER TABLESPACE ............................................................................................... 1230 ALTER TEXT SEARCH CONFIGURATION ........................................................... 1232 ALTER TEXT SEARCH DICTIONARY ................................................................... 1234 ALTER TEXT SEARCH PARSER............................................................................. 1236 ALTER TEXT SEARCH TEMPLATE ....................................................................... 1237 ALTER TRIGGER ...................................................................................................... 1238 ALTER TYPE.............................................................................................................. 1240 ALTER USER ............................................................................................................. 1244 ALTER USER MAPPING .......................................................................................... 1245 ALTER VIEW ............................................................................................................. 1247 ANALYZE................................................................................................................... 1249 BEGIN......................................................................................................................... 1252 CHECKPOINT............................................................................................................ 1254 CLOSE ........................................................................................................................ 1255 CLUSTER ................................................................................................................... 1257 COMMENT................................................................................................................. 1260 COMMIT..................................................................................................................... 1264 COMMIT PREPARED................................................................................................ 1265 COPY .......................................................................................................................... 1267 CREATE AGGREGATE ............................................................................................. 1277 CREATE CAST........................................................................................................... 1280 CREATE COLLATION............................................................................................... 1285 CREATE CONVERSION ........................................................................................... 1287 CREATE DATABASE................................................................................................. 1289 CREATE DOMAIN..................................................................................................... 1292 CREATE EVENT TRIGGER...................................................................................... 1295 CREATE EXTENSION............................................................................................... 1297 CREATE FOREIGN DATA WRAPPER..................................................................... 1299 CREATE FOREIGN TABLE ...................................................................................... 1301 CREATE FUNCTION................................................................................................. 1304 CREATE GROUP........................................................................................................ 1312 CREATE INDEX......................................................................................................... 1313 CREATE LANGUAGE ............................................................................................... 1320 CREATE MATERIALIZED VIEW ............................................................................ 1323 CREATE OPERATOR ................................................................................................ 1325 CREATE OPERATOR CLASS ................................................................................... 1328 CREATE OPERATOR FAMILY................................................................................. 1331 CREATE ROLE........................................................................................................... 1333 CREATE RULE........................................................................................................... 1338 CREATE SCHEMA .................................................................................................... 1341
xxv
CREATE SEQUENCE ................................................................................................ 1344 CREATE SERVER ...................................................................................................... 1348 CREATE TABLE ........................................................................................................ 1350 CREATE TABLE AS .................................................................................................. 1365 CREATE TABLESPACE............................................................................................. 1368 CREATE TEXT SEARCH CONFIGURATION......................................................... 1370 CREATE TEXT SEARCH DICTIONARY................................................................. 1372 CREATE TEXT SEARCH PARSER .......................................................................... 1374 CREATE TEXT SEARCH TEMPLATE..................................................................... 1376 CREATE TRIGGER.................................................................................................... 1378 CREATE TYPE ........................................................................................................... 1384 CREATE USER........................................................................................................... 1393 CREATE USER MAPPING........................................................................................ 1394 CREATE VIEW........................................................................................................... 1396 DEALLOCATE ........................................................................................................... 1400 DECLARE................................................................................................................... 1401 DELETE ...................................................................................................................... 1405 DISCARD.................................................................................................................... 1408 DO ............................................................................................................................... 1409 DROP AGGREGATE.................................................................................................. 1411 DROP CAST ............................................................................................................... 1413 DROP COLLATION ................................................................................................... 1415 DROP CONVERSION................................................................................................ 1416 DROP DATABASE ..................................................................................................... 1417 DROP DOMAIN ......................................................................................................... 1418 DROP EVENT TRIGGER .......................................................................................... 1419 DROP EXTENSION ................................................................................................... 1420 DROP FOREIGN DATA WRAPPER ......................................................................... 1422 DROP FOREIGN TABLE........................................................................................... 1423 DROP FUNCTION ..................................................................................................... 1424 DROP GROUP ............................................................................................................ 1426 DROP INDEX ............................................................................................................. 1427 DROP LANGUAGE.................................................................................................... 1429 DROP MATERIALIZED VIEW................................................................................. 1431 DROP OPERATOR ..................................................................................................... 1433 DROP OPERATOR CLASS........................................................................................ 1435 DROP OPERATOR FAMILY ..................................................................................... 1437 DROP OWNED........................................................................................................... 1439 DROP ROLE ............................................................................................................... 1441 DROP RULE ............................................................................................................... 1443 DROP SCHEMA ......................................................................................................... 1445 DROP SEQUENCE..................................................................................................... 1447 DROP SERVER........................................................................................................... 1448 DROP TABLE ............................................................................................................. 1449 DROP TABLESPACE ................................................................................................. 1451 DROP TEXT SEARCH CONFIGURATION ............................................................. 1453 DROP TEXT SEARCH DICTIONARY ..................................................................... 1455 DROP TEXT SEARCH PARSER............................................................................... 1456 DROP TEXT SEARCH TEMPLATE ......................................................................... 1457 DROP TRIGGER ........................................................................................................ 1458 DROP TYPE................................................................................................................ 1460 DROP USER ............................................................................................................... 1461
pg_upgrade......................................................................................................... 2850 pg_xlogdump ..................................................................................................... 2857 H. External Projects .............................................................................................................. 2859 H.1. Client Interfaces................................................................................................... 2859 H.2. Administration Tools ........................................................................................... 2859 H.3. Procedural Languages.......................................................................................... 2860 H.4. Extensions............................................................................................................ 2860 I. The Source Code Repository ............................................................................................. 2861 I.1. Getting The Source via Git ................................................................................... 2861 J. Documentation .................................................................................................................. 2862 J.1. DocBook ............................................................................................................... 2862 J.2. Tool Sets................................................................................................................ 2862 J.2.1. Linux RPM Installation ............................................................................ 2863 J.2.2. FreeBSD Installation ................................................................................ 2863 J.2.3. Debian Packages....................................................................................... 2864 J.2.4. Mac OS X................................................................................................. 2864 J.2.5. Manual Installation from Source.............................................................. 2864 J.2.5.1. Installing OpenJade ..................................................................... 2865 J.2.5.2. Installing the DocBook DTD Kit................................................. 2865 J.2.5.3. Installing the DocBook DSSSL Style Sheets .............................. 2866 J.2.5.4. Installing JadeTeX ....................................................................... 2866 J.2.6. Detection by configure ......................................................................... 2867 J.3. Building The Documentation................................................................................ 2867 J.3.1. HTML....................................................................................................... 2867 J.3.2. Manpages.................................................................................................. 2867 J.3.3. Print Output via JadeTeX ......................................................................... 2868 J.3.4. Overflow Text ........................................................................................... 2868 J.3.5. Print Output via RTF ................................................................................ 2869 J.3.6. Plain Text Files ......................................................................................... 2870 J.3.7. Syntax Check............................................................................................ 2870 J.4. Documentation Authoring .................................................................................... 2870 J.4.1. Emacs/PSGML......................................................................................... 2871 J.4.2. Other Emacs Modes ................................................................................. 2872 J.5. Style Guide............................................................................................................ 2872 J.5.1. Reference Pages ....................................................................................... 2872 K. Acronyms ......................................................................................................................... 2874 Bibliography .................................................................................................................................. 2880 Index............................................................................................................................................... 2882
lxii
Preface This book is the official documentation of PostgreSQL. It has been written by the PostgreSQL developers and other volunteers in parallel to the development of the PostgreSQL software. It describes all the functionality that the current version of PostgreSQL officially supports. To make the large amount of information about PostgreSQL manageable, this book has been organized in several parts. Each part is targeted at a different class of users, or at users in different stages of their PostgreSQL experience: •
Part I is an informal introduction for new users.
•
Part II documents the SQL query language environment, including data types and functions, as well as user-level performance tuning. Every PostgreSQL user should read this.
•
Part III describes the installation and administration of the server. Everyone who runs a PostgreSQL server, be it for private use or for others, should read this part.
•
Part IV describes the programming interfaces for PostgreSQL client programs.
•
Part V contains information for advanced users about the extensibility capabilities of the server. Topics include user-defined data types and functions.
•
Part VI contains reference information about SQL commands, client and server programs. This part supports the other parts with structured information sorted by command or program.
•
Part VII contains assorted information that might be of use to PostgreSQL developers.
1. What is PostgreSQL? PostgreSQL is an object-relational database management system (ORDBMS) based on POSTGRES, Version 4.21, developed at the University of California at Berkeley Computer Science Department. POSTGRES pioneered many concepts that only became available in some commercial database systems much later. PostgreSQL is an open-source descendant of this original Berkeley code. It supports a large part of the SQL standard and offers many modern features: • • • • • •
Also, PostgreSQL can be extended by the user in many ways, for example by adding new data types functions • operators • aggregate functions • index methods • •
1.
http://db.cs.berkeley.edu/postgres.html
lxiii
Preface •
procedural languages
And because of the liberal license, PostgreSQL can be used, modified, and distributed by anyone free of charge for any purpose, be it private, commercial, or academic.
2. A Brief History of PostgreSQL The object-relational database management system now known as PostgreSQL is derived from the POSTGRES package written at the University of California at Berkeley. With over two decades of development behind it, PostgreSQL is now the most advanced open-source database available anywhere.
2.1. The Berkeley POSTGRES Project The POSTGRES project, led by Professor Michael Stonebraker, was sponsored by the Defense Advanced Research Projects Agency (DARPA), the Army Research Office (ARO), the National Science Foundation (NSF), and ESL, Inc. The implementation of POSTGRES began in 1986. The initial concepts for the system were presented in The design of POSTGRES , and the definition of the initial data model appeared in The POSTGRES data model . The design of the rule system at that time was described in The design of the POSTGRES rules system. The rationale and architecture of the storage manager were detailed in The design of the POSTGRES storage system . POSTGRES has undergone several major releases since then. The first “demoware” system became operational in 1987 and was shown at the 1988 ACM-SIGMOD Conference. Version 1, described in The implementation of POSTGRES , was released to a few external users in June 1989. In response to a critique of the first rule system ( A commentary on the POSTGRES rules system ), the rule system was redesigned ( On Rules, Procedures, Caching and Views in Database Systems ), and Version 2 was released in June 1990 with the new rule system. Version 3 appeared in 1991 and added support for multiple storage managers, an improved query executor, and a rewritten rule system. For the most part, subsequent releases until Postgres95 (see below) focused on portability and reliability. POSTGRES has been used to implement many different research and production applications. These include: a financial data analysis system, a jet engine performance monitoring package, an asteroid tracking database, a medical information database, and several geographic information systems. POSTGRES has also been used as an educational tool at several universities. Finally, Illustra Information Technologies (later merged into Informix2, which is now owned by IBM3) picked up the code and commercialized it. In late 1992, POSTGRES became the primary data manager for the Sequoia 2000 scientific computing project4. The size of the external user community nearly doubled during 1993. It became increasingly obvious that maintenance of the prototype code and support was taking up large amounts of time that should have been devoted to database research. In an effort to reduce this support burden, the Berkeley POSTGRES project officially ended with Version 4.2.
2.2. Postgres95 In 1994, Andrew Yu and Jolly Chen added an SQL language interpreter to POSTGRES. Under a new name, Postgres95 was subsequently released to the web to find its own way in the world as an open-source descendant of the original POSTGRES Berkeley code. Postgres95 code was completely ANSI C and trimmed in size by 25%. Many internal changes improved performance and maintainability. Postgres95 release 1.0.x ran about 30-50% faster on the Wisconsin Benchmark compared to POSTGRES, Version 4.2. Apart from bug fixes, the following were the major enhancements: •
The query language PostQUEL was replaced with SQL (implemented in the server). (Interface library libpq was named after PostQUEL.) Subqueries were not supported until PostgreSQL (see below), but they could be imitated in Postgres95 with user-defined SQL functions. Aggregate functions were re-implemented. Support for the GROUP BY query clause was also added.
•
A new program (psql) was provided for interactive SQL queries, which used GNU Readline. This largely superseded the old monitor program.
•
A new front-end library, libpgtcl, supported Tcl-based clients. A sample shell, pgtclsh, provided new Tcl commands to interface Tcl programs with the Postgres95 server.
•
The large-object interface was overhauled. The inversion large objects were the only mechanism for storing large objects. (The inversion file system was removed.)
•
The instance-level rule system was removed. Rules were still available as rewrite rules.
•
A short tutorial introducing regular SQL features as well as those of Postgres95 was distributed with the source code
•
GNU make (instead of BSD make) was used for the build. Also, Postgres95 could be compiled with an unpatched GCC (data alignment of doubles was fixed).
2.3. PostgreSQL By 1996, it became clear that the name “Postgres95” would not stand the test of time. We chose a new name, PostgreSQL, to reflect the relationship between the original POSTGRES and the more recent versions with SQL capability. At the same time, we set the version numbering to start at 6.0, putting the numbers back into the sequence originally begun by the Berkeley POSTGRES project. Many people continue to refer to PostgreSQL as “Postgres” (now rarely in all capital letters) because of tradition or because it is easier to pronounce. This usage is widely accepted as a nickname or alias. The emphasis during development of Postgres95 was on identifying and understanding existing problems in the server code. With PostgreSQL, the emphasis has shifted to augmenting features and capabilities, although work continues in all areas. Details about what has happened in PostgreSQL since then can be found in Appendix E.
3. Conventions The following conventions are used in the synopsis of a command: brackets ([ and ]) indicate optional parts. (In the synopsis of a Tcl command, question marks (?) are used instead, as is usual in Tcl.)
lxv
Preface Braces ({ and }) and vertical lines (|) indicate that you must choose one alternative. Dots (...) mean that the preceding element can be repeated. Where it enhances the clarity, SQL commands are preceded by the prompt =>, and shell commands are preceded by the prompt $. Normally, prompts are not shown, though. An administrator is generally a person who is in charge of installing and running the server. A user could be anyone who is using, or wants to use, any part of the PostgreSQL system. These terms should not be interpreted too narrowly; this book does not have fixed presumptions about system administration procedures.
4. Further Information Besides the documentation, that is, this book, there are other resources about PostgreSQL: Wiki The PostgreSQL wiki5 contains the project’s FAQ6 (Frequently Asked Questions) list, TODO7 list, and detailed information about many more topics. Web Site The PostgreSQL web site8 carries details on the latest release and other information to make your work or play with PostgreSQL more productive. Mailing Lists The mailing lists are a good place to have your questions answered, to share experiences with other users, and to contact the developers. Consult the PostgreSQL web site for details. Yourself! PostgreSQL is an open-source project. As such, it depends on the user community for ongoing support. As you begin to use PostgreSQL, you will rely on others for help, either through the documentation or through the mailing lists. Consider contributing your knowledge back. Read the mailing lists and answer questions. If you learn something which is not in the documentation, write it up and contribute it. If you add features to the code, contribute them.
5. Bug Reporting Guidelines When you find a bug in PostgreSQL we want to hear about it. Your bug reports play an important part in making PostgreSQL more reliable because even the utmost care cannot guarantee that every part of PostgreSQL will work on every platform under every circumstance. The following suggestions are intended to assist you in forming bug reports that can be handled in an effective fashion. No one is required to follow them but doing so tends to be to everyone’s advantage. We cannot promise to fix every bug right away. If the bug is obvious, critical, or affects a lot of users, chances are good that someone will look into it. It could also happen that we tell you to update to a newer version to see if the bug happens there. Or we might decide that the bug cannot be fixed before 5. 6. 7. 8.
Preface some major rewrite we might be planning is done. Or perhaps it is simply too hard and there are more important things on the agenda. If you need help immediately, consider obtaining a commercial support contract.
5.1. Identifying Bugs Before you report a bug, please read and re-read the documentation to verify that you can really do whatever it is you are trying. If it is not clear from the documentation whether you can do something or not, please report that too; it is a bug in the documentation. If it turns out that a program does something different from what the documentation says, that is a bug. That might include, but is not limited to, the following circumstances:
•
A program terminates with a fatal signal or an operating system error message that would point to a problem in the program. (A counterexample might be a “disk full” message, since you have to fix that yourself.)
•
A program produces the wrong output for any given input.
•
A program refuses to accept valid input (as defined in the documentation).
•
A program accepts invalid input without a notice or error message. But keep in mind that your idea of invalid input might be our idea of an extension or compatibility with traditional practice.
•
PostgreSQL fails to compile, build, or install according to the instructions on supported platforms.
Here “program” refers to any executable, not only the backend process. Being slow or resource-hogging is not necessarily a bug. Read the documentation or ask on one of the mailing lists for help in tuning your applications. Failing to comply to the SQL standard is not necessarily a bug either, unless compliance for the specific feature is explicitly claimed. Before you continue, check on the TODO list and in the FAQ to see if your bug is already known. If you cannot decode the information on the TODO list, report your problem. The least we can do is make the TODO list clearer.
5.2. What to Report The most important thing to remember about bug reporting is to state all the facts and only facts. Do not speculate what you think went wrong, what “it seemed to do”, or which part of the program has a fault. If you are not familiar with the implementation you would probably guess wrong and not help us a bit. And even if you are, educated explanations are a great supplement to but no substitute for facts. If we are going to fix the bug we still have to see it happen for ourselves first. Reporting the bare facts is relatively straightforward (you can probably copy and paste them from the screen) but all too often important details are left out because someone thought it does not matter or the report would be understood anyway. The following items should be contained in every bug report: •
The exact sequence of steps from program start-up necessary to reproduce the problem. This should be self-contained; it is not enough to send in a bare SELECT statement without the preceding CREATE TABLE and INSERT statements, if the output should depend on the data in the tables. We do not have the time to reverse-engineer your database schema, and if we are supposed to make up our own data we would probably miss the problem.
lxvii
Preface The best format for a test case for SQL-related problems is a file that can be run through the psql frontend that shows the problem. (Be sure to not have anything in your ~/.psqlrc start-up file.) An easy way to create this file is to use pg_dump to dump out the table declarations and data needed to set the scene, then add the problem query. You are encouraged to minimize the size of your example, but this is not absolutely necessary. If the bug is reproducible, we will find it either way. If your application uses some other client interface, such as PHP, then please try to isolate the offending queries. We will probably not set up a web server to reproduce your problem. In any case remember to provide the exact input files; do not guess that the problem happens for “large files” or “midsize databases”, etc. since this information is too inexact to be of use. •
The output you got. Please do not say that it “didn’t work” or “crashed”. If there is an error message, show it, even if you do not understand it. If the program terminates with an operating system error, say which. If nothing at all happens, say so. Even if the result of your test case is a program crash or otherwise obvious it might not happen on our platform. The easiest thing is to copy the output from the terminal, if possible. Note: If you are reporting an error message, please obtain the most verbose form of the message. In psql, say \set VERBOSITY verbose beforehand. If you are extracting the message from the server log, set the run-time parameter log_error_verbosity to verbose so that all details are logged.
Note: In case of fatal errors, the error message reported by the client might not contain all the information available. Please also look at the log output of the database server. If you do not keep your server’s log output, this would be a good time to start doing so.
•
The output you expected is very important to state. If you just write “This command gives me that output.” or “This is not what I expected.”, we might run it ourselves, scan the output, and think it looks OK and is exactly what we expected. We should not have to spend the time to decode the exact semantics behind your commands. Especially refrain from merely saying that “This is not what SQL says/Oracle does.” Digging out the correct behavior from SQL is not a fun undertaking, nor do we all know how all the other relational databases out there behave. (If your problem is a program crash, you can obviously omit this item.)
•
Any command line options and other start-up options, including any relevant environment variables or configuration files that you changed from the default. Again, please provide exact information. If you are using a prepackaged distribution that starts the database server at boot time, you should try to find out how that is done.
•
Anything you did at all differently from the installation instructions.
•
The PostgreSQL version. You can run the command SELECT version(); to find out the version of the server you are connected to. Most executable programs also support a --version option; at least postgres --version and psql --version should work. If the function or the options do not exist then your version is more than old enough to warrant an upgrade. If you run a prepackaged version, such as RPMs, say so, including any subversion the package might have. If you are talking about a Git snapshot, mention that, including the commit hash. If your version is older than 9.3.19 we will almost certainly tell you to upgrade. There are many bug fixes and improvements in each new release, so it is quite possible that a bug you have encountered
lxviii
Preface in an older release of PostgreSQL has already been fixed. We can only provide limited support for sites using older releases of PostgreSQL; if you require more than we can provide, consider acquiring a commercial support contract. •
Platform information. This includes the kernel name and version, C library, processor, memory information, and so on. In most cases it is sufficient to report the vendor and version, but do not assume everyone knows what exactly “Debian” contains or that everyone runs on i386s. If you have installation problems then information about the toolchain on your machine (compiler, make, and so on) is also necessary.
Do not be afraid if your bug report becomes rather lengthy. That is a fact of life. It is better to report everything the first time than us having to squeeze the facts out of you. On the other hand, if your input files are huge, it is fair to ask first whether somebody is interested in looking into it. Here is an article9 that outlines some more tips on reporting bugs. Do not spend all your time to figure out which changes in the input make the problem go away. This will probably not help solving it. If it turns out that the bug cannot be fixed right away, you will still have time to find and share your work-around. Also, once again, do not waste your time guessing why the bug exists. We will find that out soon enough. When writing a bug report, please avoid confusing terminology. The software package in total is called “PostgreSQL”, sometimes “Postgres” for short. If you are specifically talking about the backend process, mention that, do not just say “PostgreSQL crashes”. A crash of a single backend process is quite different from crash of the parent “postgres” process; please don’t say “the server crashed” when you mean a single backend process went down, nor vice versa. Also, client programs such as the interactive frontend “psql” are completely separate from the backend. Please try to be specific about whether the problem is on the client or server side.
5.3. Where to Report Bugs In general, send bug reports to the bug report mailing list at . You are requested to use a descriptive subject for your email message, perhaps parts of the error message. Another method is to fill in the bug report web-form available at the project’s web site10. Entering a bug report this way causes it to be mailed to the mailing list. If your bug report has security implications and you’d prefer that it not become immediately visible in public archives, don’t send it to pgsql-bugs. Security issues can be reported privately to . Do not send bug reports to any of the user mailing lists, such as or . These mailing lists are for answering user questions, and their subscribers normally do not wish to receive bug reports. More importantly, they are unlikely to fix them. Also, please do not send reports to the developers’ mailing list . This list is for discussing the development of PostgreSQL, and it would be nice if we could keep the bug reports separate. We might choose to take up a discussion about your bug report on pgsql-hackers, if the problem needs more review. If you have a problem with the documentation, the best place to report it is the documentation mailing list . Please be specific about what part of the documentation you are unhappy with. 9. http://www.chiark.greenend.org.uk/~sgtatham/bugs.html 10. https://www.postgresql.org/
lxix
Preface If your bug is a portability problem on a non-supported platform, send mail to , so we (and you) can work on porting PostgreSQL to your platform. Note: Due to the unfortunate amount of spam going around, all of the above email addresses are closed mailing lists. That is, you need to be subscribed to a list to be allowed to post on it. (You need not be subscribed to use the bug-report web form, however.) If you would like to send mail but do not want to receive list traffic, you can subscribe and set your subscription option to nomail. For more information send mail to with the single word help in the body of the message.
lxx
I. Tutorial Welcome to the PostgreSQL Tutorial. The following few chapters are intended to give a simple introduction to PostgreSQL, relational database concepts, and the SQL language to those who are new to any one of these aspects. We only assume some general knowledge about how to use computers. No particular Unix or programming experience is required. This part is mainly intended to give you some hands-on experience with important aspects of the PostgreSQL system. It makes no attempt to be a complete or thorough treatment of the topics it covers. After you have worked through this tutorial you might want to move on to reading Part II to gain a more formal knowledge of the SQL language, or Part IV for information about developing applications for PostgreSQL. Those who set up and manage their own server should also read Part III.
Chapter 1. Getting Started 1.1. Installation Before you can use PostgreSQL you need to install it, of course. It is possible that PostgreSQL is already installed at your site, either because it was included in your operating system distribution or because the system administrator already installed it. If that is the case, you should obtain information from the operating system documentation or your system administrator about how to access PostgreSQL. If you are not sure whether PostgreSQL is already available or whether you can use it for your experimentation then you can install it yourself. Doing so is not hard and it can be a good exercise. PostgreSQL can be installed by any unprivileged user; no superuser (root) access is required. If you are installing PostgreSQL yourself, then refer to Chapter 15 for instructions on installation, and return to this guide when the installation is complete. Be sure to follow closely the section about setting up the appropriate environment variables. If your site administrator has not set things up in the default way, you might have some more work to do. For example, if the database server machine is a remote machine, you will need to set the PGHOST environment variable to the name of the database server machine. The environment variable PGPORT might also have to be set. The bottom line is this: if you try to start an application program and it complains that it cannot connect to the database, you should consult your site administrator or, if that is you, the documentation to make sure that your environment is properly set up. If you did not understand the preceding paragraph then read the next section.
1.2. Architectural Fundamentals Before we proceed, you should understand the basic PostgreSQL system architecture. Understanding how the parts of PostgreSQL interact will make this chapter somewhat clearer. In database jargon, PostgreSQL uses a client/server model. A PostgreSQL session consists of the following cooperating processes (programs): •
A server process, which manages the database files, accepts connections to the database from client applications, and performs database actions on behalf of the clients. The database server program is called postgres.
•
The user’s client (frontend) application that wants to perform database operations. Client applications can be very diverse in nature: a client could be a text-oriented tool, a graphical application, a web server that accesses the database to display web pages, or a specialized database maintenance tool. Some client applications are supplied with the PostgreSQL distribution; most are developed by users.
As is typical of client/server applications, the client and the server can be on different hosts. In that case they communicate over a TCP/IP network connection. You should keep this in mind, because the files that can be accessed on a client machine might not be accessible (or might only be accessible using a different file name) on the database server machine. The PostgreSQL server can handle multiple concurrent connections from clients. To achieve this it starts (“forks”) a new process for each connection. From that point on, the client and the new
1
Chapter 1. Getting Started server process communicate without intervention by the original postgres process. Thus, the master server process is always running, waiting for client connections, whereas client and associated server processes come and go. (All of this is of course invisible to the user. We only mention it here for completeness.)
1.3. Creating a Database The first test to see whether you can access the database server is to try to create a database. A running PostgreSQL server can manage many databases. Typically, a separate database is used for each project or for each user. Possibly, your site administrator has already created a database for your use. He should have told you what the name of your database is. In that case you can omit this step and skip ahead to the next section. To create a new database, in this example named mydb, you use the following command: $ createdb mydb
If this produces no response then this step was successful and you can skip over the remainder of this section. If you see a message similar to: createdb: command not found
then PostgreSQL was not installed properly. Either it was not installed at all or your shell’s search path was not set to include it. Try calling the command with an absolute path instead: $ /usr/local/pgsql/bin/createdb mydb
The path at your site might be different. Contact your site administrator or check the installation instructions to correct the situation. Another response could be this:
createdb: could not connect to database postgres: could not connect to server: No such fi Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
This means that the server was not started, or it was not started where createdb expected it. Again, check the installation instructions or consult the administrator. Another response could be this: createdb: could not connect to database postgres: FATAL:
role "joe" does not exist
where your own login name is mentioned. This will happen if the administrator has not created a PostgreSQL user account for you. (PostgreSQL user accounts are distinct from operating system user accounts.) If you are the administrator, see Chapter 20 for help creating accounts. You will need to become the operating system user under which PostgreSQL was installed (usually postgres) to create the first user account. It could also be that you were assigned a PostgreSQL user name that is different from your operating system user name; in that case you need to use the -U switch or set the PGUSER environment variable to specify your PostgreSQL user name. If you have a user account but it does not have the privileges required to create a database, you will see the following:
2
Chapter 1. Getting Started createdb: database creation failed: ERROR:
permission denied to create database
Not every user has authorization to create new databases. If PostgreSQL refuses to create databases for you then the site administrator needs to grant you permission to create databases. Consult your site administrator if this occurs. If you installed PostgreSQL yourself then you should log in for the purposes of this tutorial under the user account that you started the server as. 1 You can also create databases with other names. PostgreSQL allows you to create any number of databases at a given site. Database names must have an alphabetic first character and are limited to 63 bytes in length. A convenient choice is to create a database with the same name as your current user name. Many tools assume that database name as the default, so it can save you some typing. To create that database, simply type: $ createdb
If you do not want to use your database anymore you can remove it. For example, if you are the owner (creator) of the database mydb, you can destroy it using the following command: $ dropdb mydb
(For this command, the database name does not default to the user account name. You always need to specify it.) This action physically removes all files associated with the database and cannot be undone, so this should only be done with a great deal of forethought. More about createdb and dropdb can be found in createdb and dropdb respectively.
1.4. Accessing a Database Once you have created a database, you can access it by:
Running the PostgreSQL interactive terminal program, called psql, which allows you to interactively enter, edit, and execute SQL commands. • Using an existing graphical frontend tool like pgAdmin or an office suite with ODBC or JDBC support to create and manipulate a database. These possibilities are not covered in this tutorial. • Writing a custom application, using one of the several available language bindings. These possibilities are discussed further in Part IV. •
You probably want to start up psql to try the examples in this tutorial. It can be activated for the mydb database by typing the command: $ psql mydb
If you do not supply the database name then it will default to your user account name. You already discovered this scheme in the previous section using createdb. In psql, you will be greeted with the following message: 1. As an explanation for why this works: PostgreSQL user names are separate from operating system user accounts. When you connect to a database, you can choose what PostgreSQL user name to connect as; if you don’t, it will default to the same name as your current operating system account. As it happens, there will always be a PostgreSQL user account that has the same name as the operating system user that started the server, and it also happens that that user always has permission to create databases. Instead of logging in as that user you can also specify the -U option everywhere to select a PostgreSQL user name to connect as.
3
Chapter 1. Getting Started psql (9.3.19) Type "help" for help. mydb=>
The last line could also be: mydb=#
That would mean you are a database superuser, which is most likely the case if you installed the PostgreSQL instance yourself. Being a superuser means that you are not subject to access controls. For the purposes of this tutorial that is not important. If you encounter problems starting psql then go back to the previous section. The diagnostics of createdb and psql are similar, and if the former worked the latter should work as well. The last line printed out by psql is the prompt, and it indicates that psql is listening to you and that you can type SQL queries into a work space maintained by psql. Try out these commands: mydb=> SELECT version();
version ----------------------------------------------------------------------PostgreSQL 9.3.19 on i586-pc-linux-gnu, compiled by GCC 2.96, 32-bit (1 row) mydb=> SELECT current_date;
date -----------2002-08-31 (1 row) mydb=> SELECT 2 + 2;
?column? ---------4 (1 row)
The psql program has a number of internal commands that are not SQL commands. They begin with the backslash character, “\”. For example, you can get help on the syntax of various PostgreSQL SQL commands by typing: mydb=> \h
To get out of psql, type: mydb=> \q
and psql will quit and return you to your command shell. (For more internal commands, type \? at the psql prompt.) The full capabilities of psql are documented in psql. In this tutorial we will not use these features explicitly, but you can use them yourself when it is helpful.
4
Chapter 2. The SQL Language 2.1. Introduction This chapter provides an overview of how to use SQL to perform simple operations. This tutorial is only intended to give you an introduction and is in no way a complete tutorial on SQL. Numerous books have been written on SQL, including Understanding the New SQL and A Guide to the SQL Standard. You should be aware that some PostgreSQL language features are extensions to the standard. In the examples that follow, we assume that you have created a database named mydb, as described in the previous chapter, and have been able to start psql. Examples in this manual can also be found in the PostgreSQL source distribution in the directory src/tutorial/. (Binary distributions of PostgreSQL might not compile these files.) To use those files, first change to that directory and run make: $ cd ..../src/tutorial $ make
This creates the scripts and compiles the C files containing user-defined functions and types. Then, to start the tutorial, do the following: $ cd ..../tutorial $ psql -s mydb ... mydb=> \i basics.sql
The \i command reads in commands from the specified file. psql’s -s option puts you in single step mode which pauses before sending each statement to the server. The commands used in this section are in the file basics.sql.
2.2. Concepts PostgreSQL is a relational database management system (RDBMS). That means it is a system for managing data stored in relations. Relation is essentially a mathematical term for table. The notion of storing data in tables is so commonplace today that it might seem inherently obvious, but there are a number of other ways of organizing databases. Files and directories on Unix-like operating systems form an example of a hierarchical database. A more modern development is the objectoriented database. Each table is a named collection of rows. Each row of a given table has the same set of named columns, and each column is of a specific data type. Whereas columns have a fixed order in each row, it is important to remember that SQL does not guarantee the order of the rows within the table in any way (although they can be explicitly sorted for display). Tables are grouped into databases, and a collection of databases managed by a single PostgreSQL server instance constitutes a database cluster.
5
Chapter 2. The SQL Language
2.3. Creating a New Table You can create a new table by specifying the table name, along with all column names and their types: CREATE TABLE weather ( city varchar(80), temp_lo int, temp_hi int, prcp real, date date );
-- low temperature -- high temperature -- precipitation
You can enter this into psql with the line breaks. psql will recognize that the command is not terminated until the semicolon. White space (i.e., spaces, tabs, and newlines) can be used freely in SQL commands. That means you can type the command aligned differently than above, or even all on one line. Two dashes (“--”) introduce comments. Whatever follows them is ignored up to the end of the line. SQL is case insensitive about key words and identifiers, except when identifiers are double-quoted to preserve the case (not done above). varchar(80) specifies a data type that can store arbitrary character strings up to 80 characters in length. int is the normal integer type. real is a type for storing single precision floating-point numbers. date should be self-explanatory. (Yes, the column of type date is also named date. This might
be convenient or confusing — you choose.) PostgreSQL supports the standard SQL types int, smallint, real, double precision, char(N ), varchar(N ), date, time, timestamp, and interval, as well as other types of general utility and a rich set of geometric types. PostgreSQL can be customized with an arbitrary number of user-defined data types. Consequently, type names are not key words in the syntax, except where required to support special cases in the SQL standard. The second example will store cities and their associated geographical location: CREATE TABLE cities ( name varchar(80), location point );
The point type is an example of a PostgreSQL-specific data type. Finally, it should be mentioned that if you don’t need a table any longer or want to recreate it differently you can remove it using the following command: DROP TABLE tablename;
2.4. Populating a Table With Rows The INSERT statement is used to populate a table with rows: INSERT INTO weather VALUES (’San Francisco’, 46, 50, 0.25, ’1994-11-27’);
6
Chapter 2. The SQL Language Note that all data types use rather obvious input formats. Constants that are not simple numeric values usually must be surrounded by single quotes (’), as in the example. The date type is actually quite flexible in what it accepts, but for this tutorial we will stick to the unambiguous format shown here. The point type requires a coordinate pair as input, as shown here: INSERT INTO cities VALUES (’San Francisco’, ’(-194.0, 53.0)’);
The syntax used so far requires you to remember the order of the columns. An alternative syntax allows you to list the columns explicitly: INSERT INTO weather (city, temp_lo, temp_hi, prcp, date) VALUES (’San Francisco’, 43, 57, 0.0, ’1994-11-29’);
You can list the columns in a different order if you wish or even omit some columns, e.g., if the precipitation is unknown: INSERT INTO weather (date, city, temp_hi, temp_lo) VALUES (’1994-11-29’, ’Hayward’, 54, 37);
Many developers consider explicitly listing the columns better style than relying on the order implicitly. Please enter all the commands shown above so you have some data to work with in the following sections. You could also have used COPY to load large amounts of data from flat-text files. This is usually faster because the COPY command is optimized for this application while allowing less flexibility than INSERT. An example would be: COPY weather FROM ’/home/user/weather.txt’;
where the file name for the source file must be available on the machine running the backend process, not the client, since the backend process reads the file directly. You can read more about the COPY command in COPY.
2.5. Querying a Table To retrieve data from a table, the table is queried. An SQL SELECT statement is used to do this. The statement is divided into a select list (the part that lists the columns to be returned), a table list (the part that lists the tables from which to retrieve the data), and an optional qualification (the part that specifies any restrictions). For example, to retrieve all the rows of table weather, type: SELECT * FROM weather;
Here * is a shorthand for “all columns”. 1 So the same result would be had with: SELECT city, temp_lo, temp_hi, prcp, date FROM weather;
The output should be: city
| temp_lo | temp_hi | prcp |
date
1. While SELECT * is useful for off-the-cuff queries, it is widely considered bad style in production code, since adding a column to the table would change the results.
7
Chapter 2. The SQL Language ---------------+---------+---------+------+-----------San Francisco | 46 | 50 | 0.25 | 1994-11-27 San Francisco | 43 | 57 | 0 | 1994-11-29 Hayward | 37 | 54 | | 1994-11-29 (3 rows)
You can write expressions, not just simple column references, in the select list. For example, you can do: SELECT city, (temp_hi+temp_lo)/2 AS temp_avg, date FROM weather;
This should give: city | temp_avg | date ---------------+----------+-----------San Francisco | 48 | 1994-11-27 San Francisco | 50 | 1994-11-29 Hayward | 45 | 1994-11-29 (3 rows)
Notice how the AS clause is used to relabel the output column. (The AS clause is optional.) A query can be “qualified” by adding a WHERE clause that specifies which rows are wanted. The WHERE clause contains a Boolean (truth value) expression, and only rows for which the Boolean expression is true are returned. The usual Boolean operators (AND, OR, and NOT) are allowed in the qualification. For example, the following retrieves the weather of San Francisco on rainy days: SELECT * FROM weather WHERE city = ’San Francisco’ AND prcp > 0.0;
Result: city | temp_lo | temp_hi | prcp | date ---------------+---------+---------+------+-----------San Francisco | 46 | 50 | 0.25 | 1994-11-27 (1 row)
You can request that the results of a query be returned in sorted order: SELECT * FROM weather ORDER BY city; city | temp_lo | temp_hi | prcp | date ---------------+---------+---------+------+-----------Hayward | 37 | 54 | | 1994-11-29 San Francisco | 43 | 57 | 0 | 1994-11-29 San Francisco | 46 | 50 | 0.25 | 1994-11-27
In this example, the sort order isn’t fully specified, and so you might get the San Francisco rows in either order. But you’d always get the results shown above if you do: SELECT * FROM weather ORDER BY city, temp_lo;
8
Chapter 2. The SQL Language You can request that duplicate rows be removed from the result of a query: SELECT DISTINCT city FROM weather; city --------------Hayward San Francisco (2 rows)
Here again, the result row ordering might vary. You can ensure consistent results by using DISTINCT and ORDER BY together: 2 SELECT DISTINCT city FROM weather ORDER BY city;
2.6. Joins Between Tables Thus far, our queries have only accessed one table at a time. Queries can access multiple tables at once, or access the same table in such a way that multiple rows of the table are being processed at the same time. A query that accesses multiple rows of the same or different tables at one time is called a join query. As an example, say you wish to list all the weather records together with the location of the associated city. To do that, we need to compare the city column of each row of the weather table with the name column of all rows in the cities table, and select the pairs of rows where these values match. Note: This is only a conceptual model. The join is usually performed in a more efficient manner than actually comparing each possible pair of rows, but this is invisible to the user.
This would be accomplished by the following query: SELECT * FROM weather, cities WHERE city = name; city | temp_lo | temp_hi | prcp | date | name | location ---------------+---------+---------+------+------------+---------------+----------San Francisco | 46 | 50 | 0.25 | 1994-11-27 | San Francisco | (-194,53) San Francisco | 43 | 57 | 0 | 1994-11-29 | San Francisco | (-194,53) (2 rows)
Observe two things about the result set: •
There is no result row for the city of Hayward. This is because there is no matching entry in the cities table for Hayward, so the join ignores the unmatched rows in the weather table. We will see shortly how this can be fixed.
2. In some database systems, including older versions of PostgreSQL, the implementation of DISTINCT automatically orders the rows and so ORDER BY is unnecessary. But this is not required by the SQL standard, and current PostgreSQL does not guarantee that DISTINCT causes the rows to be ordered.
9
Chapter 2. The SQL Language •
There are two columns containing the city name. This is correct because the lists of columns from the weather and cities tables are concatenated. In practice this is undesirable, though, so you will probably want to list the output columns explicitly rather than using *: SELECT city, temp_lo, temp_hi, prcp, date, location FROM weather, cities WHERE city = name;
Exercise: Attempt to determine the semantics of this query when the WHERE clause is omitted. Since the columns all had different names, the parser automatically found which table they belong to. If there were duplicate column names in the two tables you’d need to qualify the column names to show which one you meant, as in: SELECT weather.city, weather.temp_lo, weather.temp_hi, weather.prcp, weather.date, cities.location FROM weather, cities WHERE cities.name = weather.city;
It is widely considered good style to qualify all column names in a join query, so that the query won’t fail if a duplicate column name is later added to one of the tables. Join queries of the kind seen thus far can also be written in this alternative form: SELECT * FROM weather INNER JOIN cities ON (weather.city = cities.name);
This syntax is not as commonly used as the one above, but we show it here to help you understand the following topics. Now we will figure out how we can get the Hayward records back in. What we want the query to do is to scan the weather table and for each row to find the matching cities row(s). If no matching row is found we want some “empty values” to be substituted for the cities table’s columns. This kind of query is called an outer join. (The joins we have seen so far are inner joins.) The command looks like this: SELECT * FROM weather LEFT OUTER JOIN cities ON (weather.city = cities.name); city | temp_lo | temp_hi | prcp | date | name | location ---------------+---------+---------+------+------------+---------------+----------Hayward | 37 | 54 | | 1994-11-29 | | San Francisco | 46 | 50 | 0.25 | 1994-11-27 | San Francisco | (-194,53) San Francisco | 43 | 57 | 0 | 1994-11-29 | San Francisco | (-194,53) (3 rows)
This query is called a left outer join because the table mentioned on the left of the join operator will have each of its rows in the output at least once, whereas the table on the right will only have those rows output that match some row of the left table. When outputting a left-table row for which there is no right-table match, empty (null) values are substituted for the right-table columns. Exercise: There are also right outer joins and full outer joins. Try to find out what those do. We can also join a table against itself. This is called a self join. As an example, suppose we wish to find all the weather records that are in the temperature range of other weather records. So we need to compare the temp_lo and temp_hi columns of each weather row to the temp_lo and temp_hi columns of all other weather rows. We can do this with the following query:
10
Chapter 2. The SQL Language SELECT W1.city, W1.temp_lo AS low, W1.temp_hi AS high, W2.city, W2.temp_lo AS low, W2.temp_hi AS high FROM weather W1, weather W2 WHERE W1.temp_lo < W2.temp_lo AND W1.temp_hi > W2.temp_hi; city | low | high | city | low | high ---------------+-----+------+---------------+-----+-----San Francisco | 43 | 57 | San Francisco | 46 | 50 Hayward | 37 | 54 | San Francisco | 46 | 50 (2 rows)
Here we have relabeled the weather table as W1 and W2 to be able to distinguish the left and right side of the join. You can also use these kinds of aliases in other queries to save some typing, e.g.: SELECT * FROM weather w, cities c WHERE w.city = c.name;
You will encounter this style of abbreviating quite frequently.
2.7. Aggregate Functions Like most other relational database products, PostgreSQL supports aggregate functions. An aggregate function computes a single result from multiple input rows. For example, there are aggregates to compute the count, sum, avg (average), max (maximum) and min (minimum) over a set of rows. As an example, we can find the highest low-temperature reading anywhere with: SELECT max(temp_lo) FROM weather; max ----46 (1 row)
If we wanted to know what city (or cities) that reading occurred in, we might try: SELECT city FROM weather WHERE temp_lo = max(temp_lo);
WRONG
but this will not work since the aggregate max cannot be used in the WHERE clause. (This restriction exists because the WHERE clause determines which rows will be included in the aggregate calculation; so obviously it has to be evaluated before aggregate functions are computed.) However, as is often the case the query can be restated to accomplish the desired result, here by using a subquery: SELECT city FROM weather WHERE temp_lo = (SELECT max(temp_lo) FROM weather); city --------------San Francisco (1 row)
11
Chapter 2. The SQL Language This is OK because the subquery is an independent computation that computes its own aggregate separately from what is happening in the outer query. Aggregates are also very useful in combination with GROUP BY clauses. For example, we can get the maximum low temperature observed in each city with: SELECT city, max(temp_lo) FROM weather GROUP BY city; city | max ---------------+----Hayward | 37 San Francisco | 46 (2 rows)
which gives us one output row per city. Each aggregate result is computed over the table rows matching that city. We can filter these grouped rows using HAVING: SELECT city, max(temp_lo) FROM weather GROUP BY city HAVING max(temp_lo) < 40; city | max ---------+----Hayward | 37 (1 row)
which gives us the same results for only the cities that have all temp_lo values below 40. Finally, if we only care about cities whose names begin with “S”, we might do: SELECT city, max(temp_lo) FROM weather WHERE city LIKE ’S%’Ê GROUP BY city HAVING max(temp_lo) < 40;
Ê
The LIKE operator does pattern matching and is explained in Section 9.7.
It is important to understand the interaction between aggregates and SQL’s WHERE and HAVING clauses. The fundamental difference between WHERE and HAVING is this: WHERE selects input rows before groups and aggregates are computed (thus, it controls which rows go into the aggregate computation), whereas HAVING selects group rows after groups and aggregates are computed. Thus, the WHERE clause must not contain aggregate functions; it makes no sense to try to use an aggregate to determine which rows will be inputs to the aggregates. On the other hand, the HAVING clause always contains aggregate functions. (Strictly speaking, you are allowed to write a HAVING clause that doesn’t use aggregates, but it’s seldom useful. The same condition could be used more efficiently at the WHERE stage.) In the previous example, we can apply the city name restriction in WHERE, since it needs no aggregate. This is more efficient than adding the restriction to HAVING, because we avoid doing the grouping and aggregate calculations for all rows that fail the WHERE check.
12
Chapter 2. The SQL Language
2.8. Updates You can update existing rows using the UPDATE command. Suppose you discover the temperature readings are all off by 2 degrees after November 28. You can correct the data as follows: UPDATE weather SET temp_hi = temp_hi - 2, WHERE date > ’1994-11-28’;
temp_lo = temp_lo - 2
Look at the new state of the data: SELECT * FROM weather; city | temp_lo | temp_hi | prcp | date ---------------+---------+---------+------+-----------San Francisco | 46 | 50 | 0.25 | 1994-11-27 San Francisco | 41 | 55 | 0 | 1994-11-29 Hayward | 35 | 52 | | 1994-11-29 (3 rows)
2.9. Deletions Rows can be removed from a table using the DELETE command. Suppose you are no longer interested in the weather of Hayward. Then you can do the following to delete those rows from the table: DELETE FROM weather WHERE city = ’Hayward’;
All weather records belonging to Hayward are removed. SELECT * FROM weather; city | temp_lo | temp_hi | prcp | date ---------------+---------+---------+------+-----------San Francisco | 46 | 50 | 0.25 | 1994-11-27 San Francisco | 41 | 55 | 0 | 1994-11-29 (2 rows)
One should be wary of statements of the form DELETE FROM tablename;
Without a qualification, DELETE will remove all rows from the given table, leaving it empty. The system will not request confirmation before doing this!
13
Chapter 3. Advanced Features 3.1. Introduction In the previous chapter we have covered the basics of using SQL to store and access your data in PostgreSQL. We will now discuss some more advanced features of SQL that simplify management and prevent loss or corruption of your data. Finally, we will look at some PostgreSQL extensions. This chapter will on occasion refer to examples found in Chapter 2 to change or improve them, so it will be useful to have read that chapter. Some examples from this chapter can also be found in advanced.sql in the tutorial directory. This file also contains some sample data to load, which is not repeated here. (Refer to Section 2.1 for how to use the file.)
3.2. Views Refer back to the queries in Section 2.6. Suppose the combined listing of weather records and city location is of particular interest to your application, but you do not want to type the query each time you need it. You can create a view over the query, which gives a name to the query that you can refer to like an ordinary table: CREATE VIEW myview AS SELECT city, temp_lo, temp_hi, prcp, date, location FROM weather, cities WHERE city = name; SELECT * FROM myview;
Making liberal use of views is a key aspect of good SQL database design. Views allow you to encapsulate the details of the structure of your tables, which might change as your application evolves, behind consistent interfaces. Views can be used in almost any place a real table can be used. Building views upon other views is not uncommon.
3.3. Foreign Keys Recall the weather and cities tables from Chapter 2. Consider the following problem: You want to make sure that no one can insert rows in the weather table that do not have a matching entry in the cities table. This is called maintaining the referential integrity of your data. In simplistic database systems this would be implemented (if at all) by first looking at the cities table to check if a matching record exists, and then inserting or rejecting the new weather records. This approach has a number of problems and is very inconvenient, so PostgreSQL can do this for you. The new declaration of the tables would look like this: CREATE TABLE cities ( city varchar(80) primary key, location point );
14
Chapter 3. Advanced Features
CREATE TABLE weather ( city varchar(80) references cities(city), temp_lo int, temp_hi int, prcp real, date date );
Now try inserting an invalid record: INSERT INTO weather VALUES (’Berkeley’, 45, 53, 0.0, ’1994-11-28’);
ERROR: insert or update on table "weather" violates foreign key constraint "weather_city DETAIL: Key (city)=(Berkeley) is not present in table "cities".
The behavior of foreign keys can be finely tuned to your application. We will not go beyond this simple example in this tutorial, but just refer you to Chapter 5 for more information. Making correct use of foreign keys will definitely improve the quality of your database applications, so you are strongly encouraged to learn about them.
3.4. Transactions Transactions are a fundamental concept of all database systems. The essential point of a transaction is that it bundles multiple steps into a single, all-or-nothing operation. The intermediate states between the steps are not visible to other concurrent transactions, and if some failure occurs that prevents the transaction from completing, then none of the steps affect the database at all. For example, consider a bank database that contains balances for various customer accounts, as well as total deposit balances for branches. Suppose that we want to record a payment of $100.00 from Alice’s account to Bob’s account. Simplifying outrageously, the SQL commands for this might look like: UPDATE accounts SET balance = balance - 100.00 WHERE name = ’Alice’; UPDATE branches SET balance = balance - 100.00 WHERE name = (SELECT branch_name FROM accounts WHERE name = ’Alice’); UPDATE accounts SET balance = balance + 100.00 WHERE name = ’Bob’; UPDATE branches SET balance = balance + 100.00 WHERE name = (SELECT branch_name FROM accounts WHERE name = ’Bob’);
The details of these commands are not important here; the important point is that there are several separate updates involved to accomplish this rather simple operation. Our bank’s officers will want to be assured that either all these updates happen, or none of them happen. It would certainly not do for a system failure to result in Bob receiving $100.00 that was not debited from Alice. Nor would Alice long remain a happy customer if she was debited without Bob being credited. We need a guarantee that if something goes wrong partway through the operation, none of the steps executed so far will take effect. Grouping the updates into a transaction gives us this guarantee. A transaction is said to be atomic: from the point of view of other transactions, it either happens completely or not at all.
15
Chapter 3. Advanced Features We also want a guarantee that once a transaction is completed and acknowledged by the database system, it has indeed been permanently recorded and won’t be lost even if a crash ensues shortly thereafter. For example, if we are recording a cash withdrawal by Bob, we do not want any chance that the debit to his account will disappear in a crash just after he walks out the bank door. A transactional database guarantees that all the updates made by a transaction are logged in permanent storage (i.e., on disk) before the transaction is reported complete. Another important property of transactional databases is closely related to the notion of atomic updates: when multiple transactions are running concurrently, each one should not be able to see the incomplete changes made by others. For example, if one transaction is busy totalling all the branch balances, it would not do for it to include the debit from Alice’s branch but not the credit to Bob’s branch, nor vice versa. So transactions must be all-or-nothing not only in terms of their permanent effect on the database, but also in terms of their visibility as they happen. The updates made so far by an open transaction are invisible to other transactions until the transaction completes, whereupon all the updates become visible simultaneously. In PostgreSQL, a transaction is set up by surrounding the SQL commands of the transaction with BEGIN and COMMIT commands. So our banking transaction would actually look like: BEGIN; UPDATE accounts SET balance = balance - 100.00 WHERE name = ’Alice’; -- etc etc COMMIT;
If, partway through the transaction, we decide we do not want to commit (perhaps we just noticed that Alice’s balance went negative), we can issue the command ROLLBACK instead of COMMIT, and all our updates so far will be canceled. PostgreSQL actually treats every SQL statement as being executed within a transaction. If you do not issue a BEGIN command, then each individual statement has an implicit BEGIN and (if successful) COMMIT wrapped around it. A group of statements surrounded by BEGIN and COMMIT is sometimes called a transaction block. Note: Some client libraries issue BEGIN and COMMIT commands automatically, so that you might get the effect of transaction blocks without asking. Check the documentation for the interface you are using.
It’s possible to control the statements in a transaction in a more granular fashion through the use of savepoints. Savepoints allow you to selectively discard parts of the transaction, while committing the rest. After defining a savepoint with SAVEPOINT, you can if needed roll back to the savepoint with ROLLBACK TO. All the transaction’s database changes between defining the savepoint and rolling back to it are discarded, but changes earlier than the savepoint are kept. After rolling back to a savepoint, it continues to be defined, so you can roll back to it several times. Conversely, if you are sure you won’t need to roll back to a particular savepoint again, it can be released, so the system can free some resources. Keep in mind that either releasing or rolling back to a savepoint will automatically release all savepoints that were defined after it. All this is happening within the transaction block, so none of it is visible to other database sessions. When and if you commit the transaction block, the committed actions become visible as a unit to other sessions, while the rolled-back actions never become visible at all.
16
Chapter 3. Advanced Features Remembering the bank database, suppose we debit $100.00 from Alice’s account, and credit Bob’s account, only to find later that we should have credited Wally’s account. We could do it using savepoints like this: BEGIN; UPDATE accounts SET balance WHERE name = ’Alice’; SAVEPOINT my_savepoint; UPDATE accounts SET balance WHERE name = ’Bob’; -- oops ... forget that and ROLLBACK TO my_savepoint; UPDATE accounts SET balance WHERE name = ’Wally’; COMMIT;
This example is, of course, oversimplified, but there’s a lot of control possible in a transaction block through the use of savepoints. Moreover, ROLLBACK TO is the only way to regain control of a transaction block that was put in aborted state by the system due to an error, short of rolling it back completely and starting again.
3.5. Window Functions A window function performs a calculation across a set of table rows that are somehow related to the current row. This is comparable to the type of calculation that can be done with an aggregate function. But unlike regular aggregate functions, use of a window function does not cause rows to become grouped into a single output row — the rows retain their separate identities. Behind the scenes, the window function is able to access more than just the current row of the query result. Here is an example that shows how to compare each employee’s salary with the average salary in his or her department: SELECT depname, empno, salary, avg(salary) OVER (PARTITION BY depname) FROM empsalary; depname | empno | salary | avg -----------+-------+--------+----------------------develop | 11 | 5200 | 5020.0000000000000000 develop | 7 | 4200 | 5020.0000000000000000 develop | 9 | 4500 | 5020.0000000000000000 develop | 8 | 6000 | 5020.0000000000000000 develop | 10 | 5200 | 5020.0000000000000000 personnel | 5 | 3500 | 3700.0000000000000000 personnel | 2 | 3900 | 3700.0000000000000000 sales | 3 | 4800 | 4866.6666666666666667 sales | 1 | 5000 | 4866.6666666666666667 sales | 4 | 4800 | 4866.6666666666666667 (10 rows)
The first three output columns come directly from the table empsalary, and there is one output row for each row in the table. The fourth column represents an average taken across all the table rows that have the same depname value as the current row. (This actually is the same function as the regular avg aggregate function, but the OVER clause causes it to be treated as a window function and computed across an appropriate set of rows.)
17
Chapter 3. Advanced Features A window function call always contains an OVER clause directly following the window function’s name and argument(s). This is what syntactically distinguishes it from a regular function or aggregate function. The OVER clause determines exactly how the rows of the query are split up for processing by the window function. The PARTITION BY list within OVER specifies dividing the rows into groups, or partitions, that share the same values of the PARTITION BY expression(s). For each row, the window function is computed across the rows that fall into the same partition as the current row. You can also control the order in which rows are processed by window functions using ORDER BY within OVER. (The window ORDER BY does not even have to match the order in which the rows are output.) Here is an example:
As shown here, the rank function produces a numerical rank within the current row’s partition for each distinct ORDER BY value, in the order defined by the ORDER BY clause. rank needs no explicit parameter, because its behavior is entirely determined by the OVER clause. The rows considered by a window function are those of the “virtual table” produced by the query’s FROM clause as filtered by its WHERE, GROUP BY, and HAVING clauses if any. For example, a row removed because it does not meet the WHERE condition is not seen by any window function. A query can contain multiple window functions that slice up the data in different ways by means of different OVER clauses, but they all act on the same collection of rows defined by this virtual table. We already saw that ORDER BY can be omitted if the ordering of rows is not important. It is also possible to omit PARTITION BY, in which case there is just one partition containing all the rows. There is another important concept associated with window functions: for each row, there is a set of rows within its partition called its window frame. Many (but not all) window functions act only on the rows of the window frame, rather than of the whole partition. By default, if ORDER BY is supplied then the frame consists of all rows from the start of the partition up through the current row, plus any following rows that are equal to the current row according to the ORDER BY clause. When ORDER BY is omitted the default frame consists of all rows in the partition. 1 Here is an example using sum: SELECT salary, sum(salary) OVER () FROM empsalary; salary | sum --------+------5200 | 47100 5000 | 47100 3500 | 47100 4800 | 47100 1. There are options to define the window frame in other ways, but this tutorial does not cover them. See Section 4.2.8 for details.
Above, since there is no ORDER BY in the OVER clause, the window frame is the same as the partition, which for lack of PARTITION BY is the whole table; in other words each sum is taken over the whole table and so we get the same result for each output row. But if we add an ORDER BY clause, we get very different results: SELECT salary, sum(salary) OVER (ORDER BY salary) FROM empsalary; salary | sum --------+------3500 | 3500 3900 | 7400 4200 | 11600 4500 | 16100 4800 | 25700 4800 | 25700 5000 | 30700 5200 | 41100 5200 | 41100 6000 | 47100 (10 rows)
Here the sum is taken from the first (lowest) salary up through the current one, including any duplicates of the current one (notice the results for the duplicated salaries). Window functions are permitted only in the SELECT list and the ORDER BY clause of the query. They are forbidden elsewhere, such as in GROUP BY, HAVING and WHERE clauses. This is because they logically execute after the processing of those clauses. Also, window functions execute after regular aggregate functions. This means it is valid to include an aggregate function call in the arguments of a window function, but not vice versa. If there is a need to filter or group rows after the window calculations are performed, you can use a sub-select. For example: SELECT depname, empno, salary, enroll_date FROM (SELECT depname, empno, salary, enroll_date, rank() OVER (PARTITION BY depname ORDER BY salary DESC, empno) AS pos FROM empsalary ) AS ss WHERE pos < 3;
The above query only shows the rows from the inner query having rank less than 3. When a query involves multiple window functions, it is possible to write out each one with a separate OVER clause, but this is duplicative and error-prone if the same windowing behavior is wanted for several functions. Instead, each windowing behavior can be named in a WINDOW clause and then referenced in OVER. For example: SELECT sum(salary) OVER w, avg(salary) OVER w FROM empsalary
19
Chapter 3. Advanced Features WINDOW w AS (PARTITION BY depname ORDER BY salary DESC);
More details about window functions can be found in Section 4.2.8, Section 9.21, Section 7.2.4, and the SELECT reference page.
3.6. Inheritance Inheritance is a concept from object-oriented databases. It opens up interesting new possibilities of database design. Let’s create two tables: A table cities and a table capitals. Naturally, capitals are also cities, so you want some way to show the capitals implicitly when you list all cities. If you’re really clever you might invent some scheme like this: CREATE TABLE name population altitude state );
capitals ( text, real, int, -- (in ft) char(2)
CREATE TABLE name population altitude );
non_capitals ( text, real, int -- (in ft)
CREATE VIEW cities AS SELECT name, population, altitude FROM capitals UNION SELECT name, population, altitude FROM non_capitals;
This works OK as far as querying goes, but it gets ugly when you need to update several rows, for one thing. A better solution is this: CREATE TABLE name population altitude );
cities ( text, real, int -- (in ft)
CREATE TABLE capitals ( state char(2) ) INHERITS (cities);
In this case, a row of capitals inherits all columns (name, population, and altitude) from its parent, cities. The type of the column name is text, a native PostgreSQL type for variable length character strings. State capitals have an extra column, state, that shows their state. In PostgreSQL, a table can inherit from zero or more other tables.
20
Chapter 3. Advanced Features For example, the following query finds the names of all cities, including state capitals, that are located at an altitude over 500 feet: SELECT name, altitude FROM cities WHERE altitude > 500;
which returns: name | altitude -----------+---------Las Vegas | 2174 Mariposa | 1953 Madison | 845 (3 rows)
On the other hand, the following query finds all the cities that are not state capitals and are situated at an altitude of 500 feet or higher: SELECT name, altitude FROM ONLY cities WHERE altitude > 500; name | altitude -----------+---------Las Vegas | 2174 Mariposa | 1953 (2 rows)
Here the ONLY before cities indicates that the query should be run over only the cities table, and not tables below cities in the inheritance hierarchy. Many of the commands that we have already discussed — SELECT, UPDATE, and DELETE — support this ONLY notation. Note: Although inheritance is frequently useful, it has not been integrated with unique constraints or foreign keys, which limits its usefulness. See Section 5.8 for more detail.
3.7. Conclusion PostgreSQL has many features not touched upon in this tutorial introduction, which has been oriented toward newer users of SQL. These features are discussed in more detail in the remainder of this book. If you feel you need more introductory material, please visit the PostgreSQL web site2 for links to more resources.
2.
https://www.postgresql.org
21
II. The SQL Language This part describes the use of the SQL language in PostgreSQL. We start with describing the general syntax of SQL, then explain how to create the structures to hold data, how to populate the database, and how to query it. The middle part lists the available data types and functions for use in SQL commands. The rest treats several aspects that are important for tuning a database for optimal performance. The information in this part is arranged so that a novice user can follow it start to end to gain a full understanding of the topics without having to refer forward too many times. The chapters are intended to be self-contained, so that advanced users can read the chapters individually as they choose. The information in this part is presented in a narrative fashion in topical units. Readers looking for a complete description of a particular command should see Part VI. Readers of this part should know how to connect to a PostgreSQL database and issue SQL commands. Readers that are unfamiliar with these issues are encouraged to read Part I first. SQL commands are typically entered using the PostgreSQL interactive terminal psql, but other programs that have similar functionality can be used as well.
Chapter 4. SQL Syntax This chapter describes the syntax of SQL. It forms the foundation for understanding the following chapters which will go into detail about how SQL commands are applied to define and modify data. We also advise users who are already familiar with SQL to read this chapter carefully because it contains several rules and concepts that are implemented inconsistently among SQL databases or that are specific to PostgreSQL.
4.1. Lexical Structure SQL input consists of a sequence of commands. A command is composed of a sequence of tokens, terminated by a semicolon (“;”). The end of the input stream also terminates a command. Which tokens are valid depends on the syntax of the particular command. A token can be a key word, an identifier, a quoted identifier, a literal (or constant), or a special character symbol. Tokens are normally separated by whitespace (space, tab, newline), but need not be if there is no ambiguity (which is generally only the case if a special character is adjacent to some other token type). For example, the following is (syntactically) valid SQL input: SELECT * FROM MY_TABLE; UPDATE MY_TABLE SET A = 5; INSERT INTO MY_TABLE VALUES (3, ’hi there’);
This is a sequence of three commands, one per line (although this is not required; more than one command can be on a line, and commands can usefully be split across lines). Additionally, comments can occur in SQL input. They are not tokens, they are effectively equivalent to whitespace. The SQL syntax is not very consistent regarding what tokens identify commands and which are operands or parameters. The first few tokens are generally the command name, so in the above example we would usually speak of a “SELECT”, an “UPDATE”, and an “INSERT” command. But for instance the UPDATE command always requires a SET token to appear in a certain position, and this particular variation of INSERT also requires a VALUES in order to be complete. The precise syntax rules for each command are described in Part VI.
4.1.1. Identifiers and Key Words Tokens such as SELECT, UPDATE, or VALUES in the example above are examples of key words, that is, words that have a fixed meaning in the SQL language. The tokens MY_TABLE and A are examples of identifiers. They identify names of tables, columns, or other database objects, depending on the command they are used in. Therefore they are sometimes simply called “names”. Key words and identifiers have the same lexical structure, meaning that one cannot know whether a token is an identifier or a key word without knowing the language. A complete list of key words can be found in Appendix C. SQL identifiers and key words must begin with a letter (a-z, but also letters with diacritical marks and non-Latin letters) or an underscore (_). Subsequent characters in an identifier or key word can be letters, underscores, digits (0-9), or dollar signs ($). Note that dollar signs are not allowed in identifiers according to the letter of the SQL standard, so their use might render applications less portable. The
24
Chapter 4. SQL Syntax SQL standard will not define a key word that contains digits or starts or ends with an underscore, so identifiers of this form are safe against possible conflict with future extensions of the standard. The system uses no more than NAMEDATALEN-1 bytes of an identifier; longer names can be written in commands, but they will be truncated. By default, NAMEDATALEN is 64 so the maximum identifier length is 63 bytes. If this limit is problematic, it can be raised by changing the NAMEDATALEN constant in src/include/pg_config_manual.h. Key words and unquoted identifiers are case insensitive. Therefore: UPDATE MY_TABLE SET A = 5;
can equivalently be written as: uPDaTE my_TabLE SeT a = 5;
A convention often used is to write key words in upper case and names in lower case, e.g.: UPDATE my_table SET a = 5;
There is a second kind of identifier: the delimited identifier or quoted identifier. It is formed by enclosing an arbitrary sequence of characters in double-quotes ("). A delimited identifier is always an identifier, never a key word. So "select" could be used to refer to a column or table named “select”, whereas an unquoted select would be taken as a key word and would therefore provoke a parse error when used where a table or column name is expected. The example can be written with quoted identifiers like this: UPDATE "my_table" SET "a" = 5;
Quoted identifiers can contain any character, except the character with code zero. (To include a double quote, write two double quotes.) This allows constructing table or column names that would otherwise not be possible, such as ones containing spaces or ampersands. The length limitation still applies. A variant of quoted identifiers allows including escaped Unicode characters identified by their code points. This variant starts with U& (upper or lower case U followed by ampersand) immediately before the opening double quote, without any spaces in between, for example U&"foo". (Note that this creates an ambiguity with the operator &. Use spaces around the operator to avoid this problem.) Inside the quotes, Unicode characters can be specified in escaped form by writing a backslash followed by the four-digit hexadecimal code point number or alternatively a backslash followed by a plus sign followed by a six-digit hexadecimal code point number. For example, the identifier "data" could be written as U&"d\0061t\+000061"
The following less trivial example writes the Russian word “slon” (elephant) in Cyrillic letters: U&"\0441\043B\043E\043D"
If a different escape character than backslash is desired, it can be specified using the UESCAPE clause after the string, for example: U&"d!0061t!+000061" UESCAPE ’!’
25
Chapter 4. SQL Syntax The escape character can be any single character other than a hexadecimal digit, the plus sign, a single quote, a double quote, or a whitespace character. Note that the escape character is written in single quotes, not double quotes. To include the escape character in the identifier literally, write it twice. The Unicode escape syntax works only when the server encoding is UTF8. When other server encodings are used, only code points in the ASCII range (up to \007F) can be specified. Both the 4-digit and the 6-digit form can be used to specify UTF-16 surrogate pairs to compose characters with code points larger than U+FFFF, although the availability of the 6-digit form technically makes this unnecessary. (Surrogate pairs are not stored directly, but combined into a single code point that is then encoded in UTF-8.) Quoting an identifier also makes it case-sensitive, whereas unquoted names are always folded to lower case. For example, the identifiers FOO, foo, and "foo" are considered the same by PostgreSQL, but "Foo" and "FOO" are different from these three and each other. (The folding of unquoted names to lower case in PostgreSQL is incompatible with the SQL standard, which says that unquoted names should be folded to upper case. Thus, foo should be equivalent to "FOO" not "foo" according to the standard. If you want to write portable applications you are advised to always quote a particular name or never quote it.)
4.1.2. Constants There are three kinds of implicitly-typed constants in PostgreSQL: strings, bit strings, and numbers. Constants can also be specified with explicit types, which can enable more accurate representation and more efficient handling by the system. These alternatives are discussed in the following subsections.
4.1.2.1. String Constants A string constant in SQL is an arbitrary sequence of characters bounded by single quotes (’), for example ’This is a string’. To include a single-quote character within a string constant, write two adjacent single quotes, e.g., ’Dianne”s horse’. Note that this is not the same as a double-quote character ("). Two string constants that are only separated by whitespace with at least one newline are concatenated and effectively treated as if the string had been written as one constant. For example: SELECT ’foo’ ’bar’;
is equivalent to: SELECT ’foobar’;
but: SELECT ’foo’
’bar’;
is not valid syntax. (This slightly bizarre behavior is specified by SQL; PostgreSQL is following the standard.)
26
Chapter 4. SQL Syntax
4.1.2.2. String Constants with C-style Escapes PostgreSQL also accepts “escape” string constants, which are an extension to the SQL standard. An escape string constant is specified by writing the letter E (upper or lower case) just before the opening single quote, e.g., E’foo’. (When continuing an escape string constant across lines, write E only before the first opening quote.) Within an escape string, a backslash character (\) begins a C-like backslash escape sequence, in which the combination of backslash and following character(s) represent a special byte value, as shown in Table 4-1. Table 4-1. Backslash Escape Sequences Backslash Escape Sequence
Interpretation
\b
backspace
\f
form feed
\n
newline
\r
carriage return
\t
tab
\o, \oo, \ooo (o = 0 - 7)
octal byte value
\xh, \xhh (h = 0 - 9, A - F)
hexadecimal byte value
\uxxxx , \Uxxxxxxxx (x = 0 - 9, A - F)
16 or 32-bit hexadecimal Unicode character value
Any other character following a backslash is taken literally. Thus, to include a backslash character, write two backslashes (\\). Also, a single quote can be included in an escape string by writing \’, in addition to the normal way of ”. It is your responsibility that the byte sequences you create, especially when using the octal or hexadecimal escapes, compose valid characters in the server character set encoding. When the server encoding is UTF-8, then the Unicode escapes or the alternative Unicode escape syntax, explained in Section 4.1.2.3, should be used instead. (The alternative would be doing the UTF-8 encoding by hand and writing out the bytes, which would be very cumbersome.) The Unicode escape syntax works fully only when the server encoding is UTF8. When other server encodings are used, only code points in the ASCII range (up to \u007F) can be specified. Both the 4-digit and the 8-digit form can be used to specify UTF-16 surrogate pairs to compose characters with code points larger than U+FFFF, although the availability of the 8-digit form technically makes this unnecessary. (When surrogate pairs are used when the server encoding is UTF8, they are first combined into a single code point that is then encoded in UTF-8.)
Caution If the configuration parameter standard_conforming_strings is off, then PostgreSQL recognizes backslash escapes in both regular and escape string constants. However, as of PostgreSQL 9.1, the default is on, meaning that backslash escapes are recognized only in escape string constants. This behavior is more standards-compliant, but might break applications which rely on the historical behavior, where backslash escapes were always recognized. As a workaround, you can set this parameter to off, but it is better to migrate away from using backslash escapes. If you need to use a backslash escape to represent a special character, write the string constant with an E. In addition to standard_conforming_strings, the configuration parameters escape_string_warning and backslash_quote govern treatment of backslashes in string constants.
27
Chapter 4. SQL Syntax The character with the code zero cannot be in a string constant.
4.1.2.3. String Constants with Unicode Escapes PostgreSQL also supports another type of escape syntax for strings that allows specifying arbitrary Unicode characters by code point. A Unicode escape string constant starts with U& (upper or lower case letter U followed by ampersand) immediately before the opening quote, without any spaces in between, for example U&’foo’. (Note that this creates an ambiguity with the operator &. Use spaces around the operator to avoid this problem.) Inside the quotes, Unicode characters can be specified in escaped form by writing a backslash followed by the four-digit hexadecimal code point number or alternatively a backslash followed by a plus sign followed by a six-digit hexadecimal code point number. For example, the string ’data’ could be written as U&’d\0061t\+000061’
The following less trivial example writes the Russian word “slon” (elephant) in Cyrillic letters: U&’\0441\043B\043E\043D’
If a different escape character than backslash is desired, it can be specified using the UESCAPE clause after the string, for example: U&’d!0061t!+000061’ UESCAPE ’!’
The escape character can be any single character other than a hexadecimal digit, the plus sign, a single quote, a double quote, or a whitespace character. The Unicode escape syntax works only when the server encoding is UTF8. When other server encodings are used, only code points in the ASCII range (up to \007F) can be specified. Both the 4-digit and the 6-digit form can be used to specify UTF-16 surrogate pairs to compose characters with code points larger than U+FFFF, although the availability of the 6-digit form technically makes this unnecessary. (When surrogate pairs are used when the server encoding is UTF8, they are first combined into a single code point that is then encoded in UTF-8.) Also, the Unicode escape syntax for string constants only works when the configuration parameter standard_conforming_strings is turned on. This is because otherwise this syntax could confuse clients that parse the SQL statements to the point that it could lead to SQL injections and similar security issues. If the parameter is set to off, this syntax will be rejected with an error message. To include the escape character in the string literally, write it twice.
4.1.2.4. Dollar-quoted String Constants While the standard syntax for specifying string constants is usually convenient, it can be difficult to understand when the desired string contains many single quotes or backslashes, since each of those must be doubled. To allow more readable queries in such situations, PostgreSQL provides another way, called “dollar quoting”, to write string constants. A dollar-quoted string constant consists of a dollar sign ($), an optional “tag” of zero or more characters, another dollar sign, an arbitrary sequence of characters that makes up the string content, a dollar sign, the same tag that began this dollar quote, and a dollar sign. For example, here are two different ways to specify the string “Dianne’s horse” using dollar quoting: $$Dianne’s horse$$
Notice that inside the dollar-quoted string, single quotes can be used without needing to be escaped. Indeed, no characters inside a dollar-quoted string are ever escaped: the string content is always written literally. Backslashes are not special, and neither are dollar signs, unless they are part of a sequence matching the opening tag. It is possible to nest dollar-quoted string constants by choosing different tags at each nesting level. This is most commonly used in writing function definitions. For example: $function$ BEGIN RETURN ($1 ~ $q$[\t\r\n\v\\]$q$); END; $function$
Here, the sequence $q$[\t\r\n\v\\]$q$ represents a dollar-quoted literal string [\t\r\n\v\\], which will be recognized when the function body is executed by PostgreSQL. But since the sequence does not match the outer dollar quoting delimiter $function$, it is just some more characters within the constant so far as the outer string is concerned. The tag, if any, of a dollar-quoted string follows the same rules as an unquoted identifier, except that it cannot contain a dollar sign. Tags are case sensitive, so $tag$String content$tag$ is correct, but $TAG$String content$tag$ is not. A dollar-quoted string that follows a keyword or identifier must be separated from it by whitespace; otherwise the dollar quoting delimiter would be taken as part of the preceding identifier. Dollar quoting is not part of the SQL standard, but it is often a more convenient way to write complicated string literals than the standard-compliant single quote syntax. It is particularly useful when representing string constants inside other constants, as is often needed in procedural function definitions. With single-quote syntax, each backslash in the above example would have to be written as four backslashes, which would be reduced to two backslashes in parsing the original string constant, and then to one when the inner string constant is re-parsed during function execution.
4.1.2.5. Bit-string Constants Bit-string constants look like regular string constants with a B (upper or lower case) immediately before the opening quote (no intervening whitespace), e.g., B’1001’. The only characters allowed within bit-string constants are 0 and 1. Alternatively, bit-string constants can be specified in hexadecimal notation, using a leading X (upper or lower case), e.g., X’1FF’. This notation is equivalent to a bit-string constant with four binary digits for each hexadecimal digit. Both forms of bit-string constant can be continued across lines in the same way as regular string constants. Dollar quoting cannot be used in a bit-string constant.
4.1.2.6. Numeric Constants Numeric constants are accepted in these general forms: digits digits.[digits][e[+-]digits] [digits].digits[e[+-]digits] digitse[+-]digits
29
Chapter 4. SQL Syntax where digits is one or more decimal digits (0 through 9). At least one digit must be before or after the decimal point, if one is used. At least one digit must follow the exponent marker (e), if one is present. There cannot be any spaces or other characters embedded in the constant. Note that any leading plus or minus sign is not actually considered part of the constant; it is an operator applied to the constant. These are some examples of valid numeric constants: 42 3.5 4. .001 5e2 1.925e-3
A numeric constant that contains neither a decimal point nor an exponent is initially presumed to be type integer if its value fits in type integer (32 bits); otherwise it is presumed to be type bigint if its value fits in type bigint (64 bits); otherwise it is taken to be type numeric. Constants that contain decimal points and/or exponents are always initially presumed to be type numeric. The initially assigned data type of a numeric constant is just a starting point for the type resolution algorithms. In most cases the constant will be automatically coerced to the most appropriate type depending on context. When necessary, you can force a numeric value to be interpreted as a specific data type by casting it. For example, you can force a numeric value to be treated as type real (float4) by writing: REAL ’1.23’ 1.23::REAL
-- string style -- PostgreSQL (historical) style
These are actually just special cases of the general casting notations discussed next.
4.1.2.7. Constants of Other Types A constant of an arbitrary type can be entered using any one of the following notations: type ’string ’ ’string ’::type CAST ( ’string ’ AS type )
The string constant’s text is passed to the input conversion routine for the type called type. The result is a constant of the indicated type. The explicit type cast can be omitted if there is no ambiguity as to the type the constant must be (for example, when it is assigned directly to a table column), in which case it is automatically coerced. The string constant can be written using either regular SQL notation or dollar-quoting. It is also possible to specify a type coercion using a function-like syntax: typename ( ’string ’ )
but not all type names can be used in this way; see Section 4.2.9 for details. The ::, CAST(), and function-call syntaxes can also be used to specify run-time type conversions of arbitrary expressions, as discussed in Section 4.2.9. To avoid syntactic ambiguity, the type ’string ’ syntax can only be used to specify the type of a simple literal constant. Another restriction on the type
30
Chapter 4. SQL Syntax ’string ’ syntax is that it does not work for array types; use :: or CAST() to specify the type of an
array constant. The CAST() syntax conforms to SQL. The type ’string ’ syntax is a generalization of the standard: SQL specifies this syntax only for a few data types, but PostgreSQL allows it for all types. The syntax with :: is historical PostgreSQL usage, as is the function-call syntax.
4.1.3. Operators An operator name is a sequence of up to NAMEDATALEN-1 (63 by default) characters from the following list: +-*/=~!@#%^&|‘? There are a few restrictions on operator names, however:
and /* cannot appear anywhere in an operator name, since they will be taken as the start of a comment.
• --
•
A multiple-character operator name cannot end in + or -, unless the name also contains at least one of these characters: ~!@#%^&|‘? For example, @- is an allowed operator name, but *- is not. This restriction allows PostgreSQL to parse SQL-compliant queries without requiring spaces between tokens.
When working with non-SQL-standard operator names, you will usually need to separate adjacent operators with spaces to avoid ambiguity. For example, if you have defined a left unary operator named @, you cannot write X*@Y; you must write X* @Y to ensure that PostgreSQL reads it as two operator names not one.
4.1.4. Special Characters Some characters that are not alphanumeric have a special meaning that is different from being an operator. Details on the usage can be found at the location where the respective syntax element is described. This section only exists to advise the existence and summarize the purposes of these characters.
•
A dollar sign ($) followed by digits is used to represent a positional parameter in the body of a function definition or a prepared statement. In other contexts the dollar sign can be part of an identifier or a dollar-quoted string constant.
•
Parentheses (()) have their usual meaning to group expressions and enforce precedence. In some cases parentheses are required as part of the fixed syntax of a particular SQL command.
•
Brackets ([]) are used to select the elements of an array. See Section 8.15 for more information on arrays.
•
Commas (,) are used in some syntactical constructs to separate the elements of a list.
31
Chapter 4. SQL Syntax •
The semicolon (;) terminates an SQL command. It cannot appear anywhere within a command, except within a string constant or quoted identifier.
•
The colon (:) is used to select “slices” from arrays. (See Section 8.15.) In certain SQL dialects (such as Embedded SQL), the colon is used to prefix variable names.
•
The asterisk (*) is used in some contexts to denote all the fields of a table row or composite value. It also has a special meaning when used as the argument of an aggregate function, namely that the aggregate does not require any explicit parameter.
•
The period (.) is used in numeric constants, and to separate schema, table, and column names.
4.1.5. Comments A comment is a sequence of characters beginning with double dashes and extending to the end of the line, e.g.: -- This is a standard SQL comment
Alternatively, C-style block comments can be used: /* multiline comment * with nesting: /* nested block comment */ */
where the comment begins with /* and extends to the matching occurrence of */. These block comments nest, as specified in the SQL standard but unlike C, so that one can comment out larger blocks of code that might contain existing block comments. A comment is removed from the input stream before further syntax analysis and is effectively replaced by whitespace.
4.1.6. Operator Precedence Table 4-2 shows the precedence and associativity of the operators in PostgreSQL. Most operators have the same precedence and are left-associative. The precedence and associativity of the operators is hard-wired into the parser. This can lead to non-intuitive behavior; for example the Boolean operators < and > have a different precedence than the Boolean operators =. Also, you will sometimes need to add parentheses when using combinations of binary and unary operators. For instance: SELECT 5 ! - 6;
will be parsed as: SELECT 5 ! (- 6);
because the parser has no idea — until it is too late — that ! is defined as a postfix operator, not an infix one. To get the desired behavior in this case, you must write: SELECT (5 !) - 6;
Note that the operator precedence rules also apply to user-defined operators that have the same names as the built-in operators mentioned above. For example, if you define a “+” operator for some custom data type it will have the same precedence as the built-in “+” operator, no matter what yours does. When a schema-qualified operator name is used in the OPERATOR syntax, as for example in: SELECT 3 OPERATOR(pg_catalog.+) 4;
the OPERATOR construct is taken to have the default precedence shown in Table 4-2 for “any other” operator. This is true no matter which specific operator appears inside OPERATOR().
4.2. Value Expressions Value expressions are used in a variety of contexts, such as in the target list of the SELECT command, as new column values in INSERT or UPDATE, or in search conditions in a number of commands. The result of a value expression is sometimes called a scalar, to distinguish it from the result of a table expression (which is a table). Value expressions are therefore also called scalar expressions (or even simply expressions). The expression syntax allows the calculation of values from primitive parts using arithmetic, logical, set, and other operations.
33
Chapter 4. SQL Syntax A value expression is one of the following: •
A constant or literal value
•
A column reference
•
A positional parameter reference, in the body of a function definition or prepared statement
•
A subscripted expression
•
A field selection expression
•
An operator invocation
•
A function call
•
An aggregate expression
•
A window function call
•
A type cast
•
A collation expression
•
A scalar subquery
•
An array constructor
•
A row constructor
•
Another value expression in parentheses (used to group subexpressions and override precedence)
In addition to this list, there are a number of constructs that can be classified as an expression but do not follow any general syntax rules. These generally have the semantics of a function or operator and are explained in the appropriate location in Chapter 9. An example is the IS NULL clause. We have already discussed constants in Section 4.1.2. The following sections discuss the remaining options.
4.2.1. Column References A column can be referenced in the form: correlation.columnname
correlation is the name of a table (possibly qualified with a schema name), or an alias for a table defined by means of a FROM clause. The correlation name and separating dot can be omitted if the column name is unique across all the tables being used in the current query. (See also Chapter 7.)
4.2.2. Positional Parameters A positional parameter reference is used to indicate a value that is supplied externally to an SQL statement. Parameters are used in SQL function definitions and in prepared queries. Some client libraries also support specifying data values separately from the SQL command string, in which case parameters are used to refer to the out-of-line data values. The form of a parameter reference is: $number
34
Chapter 4. SQL Syntax
For example, consider the definition of a function, dept, as: CREATE FUNCTION dept(text) RETURNS dept AS $$ SELECT * FROM dept WHERE name = $1 $$ LANGUAGE SQL;
Here the $1 references the value of the first function argument whenever the function is invoked.
4.2.3. Subscripts If an expression yields a value of an array type, then a specific element of the array value can be extracted by writing expression[subscript]
or multiple adjacent elements (an “array slice”) can be extracted by writing expression[lower_subscript:upper_subscript]
(Here, the brackets [ ] are meant to appear literally.) Each subscript is itself an expression, which must yield an integer value. In general the array expression must be parenthesized, but the parentheses can be omitted when the expression to be subscripted is just a column reference or positional parameter. Also, multiple subscripts can be concatenated when the original array is multidimensional. For example: mytable.arraycolumn[4] mytable.two_d_column[17][34] $1[10:42] (arrayfunction(a,b))[42]
The parentheses in the last example are required. See Section 8.15 for more about arrays.
4.2.4. Field Selection If an expression yields a value of a composite type (row type), then a specific field of the row can be extracted by writing expression.fieldname
In general the row expression must be parenthesized, but the parentheses can be omitted when the expression to be selected from is just a table reference or positional parameter. For example: mytable.mycolumn $1.somecolumn (rowfunction(a,b)).col3
(Thus, a qualified column reference is actually just a special case of the field selection syntax.) An important special case is extracting a field from a table column that is of a composite type: (compositecol).somefield (mytable.compositecol).somefield
35
Chapter 4. SQL Syntax The parentheses are required here to show that compositecol is a column name not a table name, or that mytable is a table name not a schema name in the second case. You can ask for all fields of a composite value by writing .*: (compositecol).*
This notation behaves differently depending on context; see Section 8.16.5 for details.
4.2.5. Operator Invocations There are three possible syntaxes for an operator invocation: expression operator expression (binary infix operator) operator expression (unary prefix operator) expression operator (unary postfix operator)
where the operator token follows the syntax rules of Section 4.1.3, or is one of the key words AND, OR, and NOT, or is a qualified operator name in the form: OPERATOR(schema.operatorname)
Which particular operators exist and whether they are unary or binary depends on what operators have been defined by the system or the user. Chapter 9 describes the built-in operators.
4.2.6. Function Calls The syntax for a function call is the name of a function (possibly qualified with a schema name), followed by its argument list enclosed in parentheses: function_name ([expression [, expression ... ]] )
For example, the following computes the square root of 2: sqrt(2)
The list of built-in functions is in Chapter 9. Other functions can be added by the user. The arguments can optionally have names attached. See Section 4.3 for details. Note: A function that takes a single argument of composite type can optionally be called using field-selection syntax, and conversely field selection can be written in functional style. That is, the notations col(table) and table.col are interchangeable. This behavior is not SQL-standard but is provided in PostgreSQL because it allows use of functions to emulate “computed fields”. For more information see Section 8.16.5.
36
Chapter 4. SQL Syntax
4.2.7. Aggregate Expressions An aggregate expression represents the application of an aggregate function across the rows selected by a query. An aggregate function reduces multiple inputs to a single output value, such as the sum or average of the inputs. The syntax of an aggregate expression is one of the following: aggregate_name aggregate_name aggregate_name aggregate_name
where aggregate_name is a previously defined aggregate (possibly qualified with a schema name), expression is any value expression that does not itself contain an aggregate expression or a window function call, and order_by_clause is a optional ORDER BY clause as described below. The first form of aggregate expression invokes the aggregate once for each input row. The second form is the same as the first, since ALL is the default. The third form invokes the aggregate once for each distinct value of the expression (or distinct set of values, for multiple expressions) found in the input rows. The last form invokes the aggregate once for each input row; since no particular input value is specified, it is generally only useful for the count(*) aggregate function. Most aggregate functions ignore null inputs, so that rows in which one or more of the expression(s) yield null are discarded. This can be assumed to be true, unless otherwise specified, for all built-in aggregates. For example, count(*) yields the total number of input rows; count(f1) yields the number of input rows in which f1 is non-null, since count ignores nulls; and count(distinct f1) yields the number of distinct non-null values of f1. Ordinarily, the input rows are fed to the aggregate function in an unspecified order. In many cases this does not matter; for example, min produces the same result no matter what order it receives the inputs in. However, some aggregate functions (such as array_agg and string_agg) produce results that depend on the ordering of the input rows. When using such an aggregate, the optional order_by_clause can be used to specify the desired ordering. The order_by_clause has the same syntax as for a query-level ORDER BY clause, as described in Section 7.5, except that its expressions are always just expressions and cannot be output-column names or numbers. For example: SELECT array_agg(a ORDER BY b DESC) FROM table;
When dealing with multiple-argument aggregate functions, note that the ORDER BY clause goes after all the aggregate arguments. For example, write this: SELECT string_agg(a, ’,’ ORDER BY a) FROM table;
not this: SELECT string_agg(a ORDER BY a, ’,’) FROM table;
-- incorrect
The latter is syntactically valid, but it represents a call of a single-argument aggregate function with two ORDER BY keys (the second one being rather useless since it’s a constant). If DISTINCT is specified in addition to an order_by_clause, then all the ORDER BY expressions must match regular arguments of the aggregate; that is, you cannot sort on an expression that is not included in the DISTINCT list.
37
Chapter 4. SQL Syntax Note: The ability to specify both DISTINCT and ORDER BY in an aggregate function is a PostgreSQL extension.
The predefined aggregate functions are described in Section 9.20. Other aggregate functions can be added by the user. An aggregate expression can only appear in the result list or HAVING clause of a SELECT command. It is forbidden in other clauses, such as WHERE, because those clauses are logically evaluated before the results of aggregates are formed. When an aggregate expression appears in a subquery (see Section 4.2.11 and Section 9.22), the aggregate is normally evaluated over the rows of the subquery. But an exception occurs if the aggregate’s arguments contain only outer-level variables: the aggregate then belongs to the nearest such outer level, and is evaluated over the rows of that query. The aggregate expression as a whole is then an outer reference for the subquery it appears in, and acts as a constant over any one evaluation of that subquery. The restriction about appearing only in the result list or HAVING clause applies with respect to the query level that the aggregate belongs to.
4.2.8. Window Function Calls A window function call represents the application of an aggregate-like function over some portion of the rows selected by a query. Unlike regular aggregate function calls, this is not tied to grouping of the selected rows into a single output row — each row remains separate in the query output. However the window function is able to scan all the rows that would be part of the current row’s group according to the grouping specification (PARTITION BY list) of the window function call. The syntax of a window function call is one of the following: function_name function_name function_name function_name
([expression [, expression ... ]]) OVER window_name ([expression [, expression ... ]]) OVER ( window_definition ) ( * ) OVER window_name ( * ) OVER ( window_definition )
where window_definition has the syntax [ [ [ [
existing_window_name ] PARTITION BY expression [, ...] ] ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...] ] frame_clause ]
and the optional frame_clause can be one of { RANGE | ROWS } frame_start { RANGE | ROWS } BETWEEN frame_start AND frame_end
where frame_start and frame_end can be one of UNBOUNDED PRECEDING value PRECEDING CURRENT ROW value FOLLOWING UNBOUNDED FOLLOWING
Here, expression represents any value expression that does not itself contain window function calls.
38
Chapter 4. SQL Syntax window_name is a reference to a named window specification defined in the query’s WINDOW clause. Alternatively, a full window_definition can be given within parentheses, using the same syntax as for defining a named window in the WINDOW clause; see the SELECT reference page for details. It’s worth pointing out that OVER wname is not exactly equivalent to OVER (wname); the latter im-
plies copying and modifying the window definition, and will be rejected if the referenced window specification includes a frame clause. The PARTITION BY option groups the rows of the query into partitions, which are processed separately by the window function. PARTITION BY works similarly to a query-level GROUP BY clause, except that its expressions are always just expressions and cannot be output-column names or numbers. Without PARTITION BY, all rows produced by the query are treated as a single partition. The ORDER BY option determines the order in which the rows of a partition are processed by the window function. It works similarly to a query-level ORDER BY clause, but likewise cannot use output-column names or numbers. Without ORDER BY, rows are processed in an unspecified order. The frame_clause specifies the set of rows constituting the window frame, which is a subset of the current partition, for those window functions that act on the frame instead of the whole partition. The frame can be specified in either RANGE or ROWS mode; in either case, it runs from the frame_start to the frame_end . If frame_end is omitted, it defaults to CURRENT ROW. A frame_start of UNBOUNDED PRECEDING means that the frame starts with the first row of the partition, and similarly a frame_end of UNBOUNDED FOLLOWING means that the frame ends with the last row of the partition. In RANGE mode, a frame_start of CURRENT ROW means the frame starts with the current row’s first peer row (a row that ORDER BY considers equivalent to the current row), while a frame_end of CURRENT ROW means the frame ends with the last equivalent peer. In ROWS mode, CURRENT ROW simply means the current row. The value PRECEDING and value FOLLOWING cases are currently only allowed in ROWS mode. They indicate that the frame starts or ends the specified number of rows before or after the current row. value must be an integer expression not containing any variables, aggregate functions, or window functions. The value must not be null or negative; but it can be zero, which just selects the current row. The default framing option is RANGE UNBOUNDED PRECEDING, which is the same as RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW. With ORDER BY, this sets the frame to be all rows from the partition start up through the current row’s last peer. Without ORDER BY, all rows of the partition are included in the window frame, since all rows become peers of the current row. Restrictions are that frame_start cannot be UNBOUNDED FOLLOWING, frame_end cannot be UNBOUNDED PRECEDING, and the frame_end choice cannot appear earlier in the above list than the frame_start choice — for example RANGE BETWEEN CURRENT ROW AND value PRECEDING is not allowed. The built-in window functions are described in Table 9-49. Other window functions can be added by the user. Also, any built-in or user-defined aggregate function can be used as a window function. The syntaxes using * are used for calling parameter-less aggregate functions as window functions, for example count(*) OVER (PARTITION BY x ORDER BY y). The asterisk (*) is customarily not used for non-aggregate window functions. Aggregate window functions, unlike normal aggregate functions, do not allow DISTINCT or ORDER BY to be used within the function argument list. Window function calls are permitted only in the SELECT list and the ORDER BY clause of the query. More information about window functions can be found in Section 3.5, Section 9.21, Section 7.2.4.
39
Chapter 4. SQL Syntax
4.2.9. Type Casts A type cast specifies a conversion from one data type to another. PostgreSQL accepts two equivalent syntaxes for type casts: CAST ( expression AS type ) expression::type
The CAST syntax conforms to SQL; the syntax with :: is historical PostgreSQL usage. When a cast is applied to a value expression of a known type, it represents a run-time type conversion. The cast will succeed only if a suitable type conversion operation has been defined. Notice that this is subtly different from the use of casts with constants, as shown in Section 4.1.2.7. A cast applied to an unadorned string literal represents the initial assignment of a type to a literal constant value, and so it will succeed for any type (if the contents of the string literal are acceptable input syntax for the data type). An explicit type cast can usually be omitted if there is no ambiguity as to the type that a value expression must produce (for example, when it is assigned to a table column); the system will automatically apply a type cast in such cases. However, automatic casting is only done for casts that are marked “OK to apply implicitly” in the system catalogs. Other casts must be invoked with explicit casting syntax. This restriction is intended to prevent surprising conversions from being applied silently. It is also possible to specify a type cast using a function-like syntax: typename ( expression )
However, this only works for types whose names are also valid as function names. For example, double precision cannot be used this way, but the equivalent float8 can. Also, the names interval, time, and timestamp can only be used in this fashion if they are double-quoted, because of syntactic conflicts. Therefore, the use of the function-like cast syntax leads to inconsistencies and should probably be avoided. Note: The function-like syntax is in fact just a function call. When one of the two standard cast syntaxes is used to do a run-time conversion, it will internally invoke a registered function to perform the conversion. By convention, these conversion functions have the same name as their output type, and thus the “function-like syntax” is nothing more than a direct invocation of the underlying conversion function. Obviously, this is not something that a portable application should rely on. For further details see CREATE CAST.
4.2.10. Collation Expressions The COLLATE clause overrides the collation of an expression. It is appended to the expression it applies to: expr COLLATE collation
where collation is a possibly schema-qualified identifier. The COLLATE clause binds tighter than operators; parentheses can be used when necessary. If no collation is explicitly specified, the database system either derives a collation from the columns involved in the expression, or it defaults to the default collation of the database if no column is involved in the expression.
40
Chapter 4. SQL Syntax The two common uses of the COLLATE clause are overriding the sort order in an ORDER BY clause, for example: SELECT a, b, c FROM tbl WHERE ... ORDER BY a COLLATE "C";
and overriding the collation of a function or operator call that has locale-sensitive results, for example: SELECT * FROM tbl WHERE a > ’foo’ COLLATE "C";
Note that in the latter case the COLLATE clause is attached to an input argument of the operator we wish to affect. It doesn’t matter which argument of the operator or function call the COLLATE clause is attached to, because the collation that is applied by the operator or function is derived by considering all arguments, and an explicit COLLATE clause will override the collations of all other arguments. (Attaching non-matching COLLATE clauses to more than one argument, however, is an error. For more details see Section 22.2.) Thus, this gives the same result as the previous example: SELECT * FROM tbl WHERE a COLLATE "C" > ’foo’;
But this is an error: SELECT * FROM tbl WHERE (a > ’foo’) COLLATE "C";
because it attempts to apply a collation to the result of the > operator, which is of the non-collatable data type boolean.
4.2.11. Scalar Subqueries A scalar subquery is an ordinary SELECT query in parentheses that returns exactly one row with one column. (See Chapter 7 for information about writing queries.) The SELECT query is executed and the single returned value is used in the surrounding value expression. It is an error to use a query that returns more than one row or more than one column as a scalar subquery. (But if, during a particular execution, the subquery returns no rows, there is no error; the scalar result is taken to be null.) The subquery can refer to variables from the surrounding query, which will act as constants during any one evaluation of the subquery. See also Section 9.22 for other expressions involving subqueries. For example, the following finds the largest city population in each state: SELECT name, (SELECT max(pop) FROM cities WHERE cities.state = states.name) FROM states;
4.2.12. Array Constructors An array constructor is an expression that builds an array value using values for its member elements. A simple array constructor consists of the key word ARRAY, a left square bracket [, a list of expressions (separated by commas) for the array element values, and finally a right square bracket ]. For example: SELECT ARRAY[1,2,3+4]; array --------{1,2,7} (1 row)
41
Chapter 4. SQL Syntax By default, the array element type is the common type of the member expressions, determined using the same rules as for UNION or CASE constructs (see Section 10.5). You can override this by explicitly casting the array constructor to the desired type, for example: SELECT ARRAY[1,2,22.7]::integer[]; array ---------{1,2,23} (1 row)
This has the same effect as casting each expression to the array element type individually. For more on casting, see Section 4.2.9. Multidimensional array values can be built by nesting array constructors. In the inner constructors, the key word ARRAY can be omitted. For example, these produce the same result: SELECT ARRAY[ARRAY[1,2], ARRAY[3,4]]; array --------------{{1,2},{3,4}} (1 row) SELECT ARRAY[[1,2],[3,4]]; array --------------{{1,2},{3,4}} (1 row)
Since multidimensional arrays must be rectangular, inner constructors at the same level must produce sub-arrays of identical dimensions. Any cast applied to the outer ARRAY constructor propagates automatically to all the inner constructors. Multidimensional array constructor elements can be anything yielding an array of the proper kind, not only a sub-ARRAY construct. For example: CREATE TABLE arr(f1 int[], f2 int[]); INSERT INTO arr VALUES (ARRAY[[1,2],[3,4]], ARRAY[[5,6],[7,8]]); SELECT ARRAY[f1, f2, ’{{9,10},{11,12}}’::int[]] FROM arr; array -----------------------------------------------{{{1,2},{3,4}},{{5,6},{7,8}},{{9,10},{11,12}}} (1 row)
You can construct an empty array, but since it’s impossible to have an array with no type, you must explicitly cast your empty array to the desired type. For example: SELECT ARRAY[]::integer[]; array ------{} (1 row)
42
Chapter 4. SQL Syntax It is also possible to construct an array from the results of a subquery. In this form, the array constructor is written with the key word ARRAY followed by a parenthesized (not bracketed) subquery. For example: SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE ’bytea%’); array ----------------------------------------------------------------------{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413} (1 row)
The subquery must return a single column. The resulting one-dimensional array will have an element for each row in the subquery result, with an element type matching that of the subquery’s output column. The subscripts of an array value built with ARRAY always begin with one. For more information about arrays, see Section 8.15.
4.2.13. Row Constructors A row constructor is an expression that builds a row value (also called a composite value) using values for its member fields. A row constructor consists of the key word ROW, a left parenthesis, zero or more expressions (separated by commas) for the row field values, and finally a right parenthesis. For example: SELECT ROW(1,2.5,’this is a test’);
The key word ROW is optional when there is more than one expression in the list. A row constructor can include the syntax rowvalue.*, which will be expanded to a list of the elements of the row value, just as occurs when the .* syntax is used at the top level of a SELECT list (see Section 8.16.5). For example, if table t has columns f1 and f2, these are the same: SELECT ROW(t.*, 42) FROM t; SELECT ROW(t.f1, t.f2, 42) FROM t;
Note: Before PostgreSQL 8.2, the .* syntax was not expanded in row constructors, so that writing ROW(t.*, 42) created a two-field row whose first field was another row value. The new behavior is usually more useful. If you need the old behavior of nested row values, write the inner row value without .*, for instance ROW(t, 42).
By default, the value created by a ROW expression is of an anonymous record type. If necessary, it can be cast to a named composite type — either the row type of a table, or a composite type created with CREATE TYPE AS. An explicit cast might be needed to avoid ambiguity. For example: CREATE TABLE mytable(f1 int, f2 float, f3 text); CREATE FUNCTION getf1(mytable) RETURNS int AS ’SELECT $1.f1’ LANGUAGE SQL; -- No cast needed since only one getf1() exists SELECT getf1(ROW(1,2.5,’this is a test’)); getf1 -------
43
Chapter 4. SQL Syntax 1 (1 row) CREATE TYPE myrowtype AS (f1 int, f2 text, f3 numeric); CREATE FUNCTION getf1(myrowtype) RETURNS int AS ’SELECT $1.f1’ LANGUAGE SQL; -- Now we need a cast to indicate which function to call: SELECT getf1(ROW(1,2.5,’this is a test’)); ERROR: function getf1(record) is not unique SELECT getf1(ROW(1,2.5,’this is a test’)::mytable); getf1 ------1 (1 row) SELECT getf1(CAST(ROW(11,’this is a test’,2.5) AS myrowtype)); getf1 ------11 (1 row)
Row constructors can be used to build composite values to be stored in a composite-type table column, or to be passed to a function that accepts a composite parameter. Also, it is possible to compare two row values or test a row with IS NULL or IS NOT NULL, for example: SELECT ROW(1,2.5,’this is a test’) = ROW(1, 3, ’not the same’); SELECT ROW(table.*) IS NULL FROM table;
-- detect all-null rows
For more detail see Section 9.23. Row constructors can also be used in connection with subqueries, as discussed in Section 9.22.
4.2.14. Expression Evaluation Rules The order of evaluation of subexpressions is not defined. In particular, the inputs of an operator or function are not necessarily evaluated left-to-right or in any other fixed order. Furthermore, if the result of an expression can be determined by evaluating only some parts of it, then other subexpressions might not be evaluated at all. For instance, if one wrote: SELECT true OR somefunc();
then somefunc() would (probably) not be called at all. The same would be the case if one wrote: SELECT somefunc() OR true;
Note that this is not the same as the left-to-right “short-circuiting” of Boolean operators that is found in some programming languages. As a consequence, it is unwise to use functions with side effects as part of complex expressions. It is particularly dangerous to rely on side effects or evaluation order in WHERE and HAVING clauses, since those clauses are extensively reprocessed as part of developing an execution plan. Boolean expressions
44
Chapter 4. SQL Syntax (AND/OR/NOT combinations) in those clauses can be reorganized in any manner allowed by the laws of Boolean algebra. When it is essential to force evaluation order, a CASE construct (see Section 9.17) can be used. For example, this is an untrustworthy way of trying to avoid division by zero in a WHERE clause: SELECT ... WHERE x > 0 AND y/x > 1.5;
But this is safe: SELECT ... WHERE CASE WHEN x > 0 THEN y/x > 1.5 ELSE false END;
A CASE construct used in this fashion will defeat optimization attempts, so it should only be done when necessary. (In this particular example, it would be better to sidestep the problem by writing y > 1.5*x instead.) CASE is not a cure-all for such issues, however. One limitation of the technique illustrated above is that it does not prevent early evaluation of constant subexpressions. As described in Section 35.6, functions and operators marked IMMUTABLE can be evaluated when the query is planned rather than when it is executed. Thus for example SELECT CASE WHEN x > 0 THEN x ELSE 1/0 END FROM tab;
is likely to result in a division-by-zero failure due to the planner trying to simplify the constant subexpression, even if every row in the table has x > 0 so that the ELSE arm would never be entered at run time. While that particular example might seem silly, related cases that don’t obviously involve constants can occur in queries executed within functions, since the values of function arguments and local variables can be inserted into queries as constants for planning purposes. Within PL/pgSQL functions, for example, using an IF-THEN-ELSE statement to protect a risky computation is much safer than just nesting it in a CASE expression. Another limitation of the same kind is that a CASE cannot prevent evaluation of an aggregate expression contained within it, because aggregate expressions are computed before other expressions in a SELECT list or HAVING clause are considered. For example, the following query can cause a division-by-zero error despite seemingly having protected against it: SELECT CASE WHEN min(employees) > 0 THEN avg(expenses / employees) END FROM departments;
The min() and avg() aggregates are computed concurrently over all the input rows, so if any row has employees equal to zero, the division-by-zero error will occur before there is any opportunity to test the result of min(). Instead, use a WHERE clause to prevent problematic input rows from reaching an aggregate function in the first place.
4.3. Calling Functions PostgreSQL allows functions that have named parameters to be called using either positional or named notation. Named notation is especially useful for functions that have a large number of parameters, since it makes the associations between parameters and actual arguments more explicit and reliable. In positional notation, a function call is written with its argument values in the same order as they
45
Chapter 4. SQL Syntax are defined in the function declaration. In named notation, the arguments are matched to the function parameters by name and can be written in any order. In either notation, parameters that have default values given in the function declaration need not be written in the call at all. But this is particularly useful in named notation, since any combination of parameters can be omitted; while in positional notation parameters can only be omitted from right to left. PostgreSQL also supports mixed notation, which combines positional and named notation. In this case, positional parameters are written first and named parameters appear after them. The following examples will illustrate the usage of all three notations, using the following function definition: CREATE FUNCTION concat_lower_or_upper(a text, b text, uppercase boolean DEFAULT false) RETURNS text AS $$ SELECT CASE WHEN $3 THEN UPPER($1 || ’ ’ || $2) ELSE LOWER($1 || ’ ’ || $2) END; $$ LANGUAGE SQL IMMUTABLE STRICT;
Function concat_lower_or_upper has two mandatory parameters, a and b. Additionally there is one optional parameter uppercase which defaults to false. The a and b inputs will be concatenated, and forced to either upper or lower case depending on the uppercase parameter. The remaining details of this function definition are not important here (see Chapter 35 for more information).
4.3.1. Using Positional Notation Positional notation is the traditional mechanism for passing arguments to functions in PostgreSQL. An example is: SELECT concat_lower_or_upper(’Hello’, ’World’, true); concat_lower_or_upper ----------------------HELLO WORLD (1 row)
All arguments are specified in order. The result is upper case since uppercase is specified as true. Another example is: SELECT concat_lower_or_upper(’Hello’, ’World’); concat_lower_or_upper ----------------------hello world (1 row)
Here, the uppercase parameter is omitted, so it receives its default value of false, resulting in lower case output. In positional notation, arguments can be omitted from right to left so long as they have defaults.
46
Chapter 4. SQL Syntax
4.3.2. Using Named Notation In named notation, each argument’s name is specified using := to separate it from the argument expression. For example: SELECT concat_lower_or_upper(a := ’Hello’, b := ’World’); concat_lower_or_upper ----------------------hello world (1 row)
Again, the argument uppercase was omitted so it is set to false implicitly. One advantage of using named notation is that the arguments may be specified in any order, for example: SELECT concat_lower_or_upper(a := ’Hello’, b := ’World’, uppercase := true); concat_lower_or_upper ----------------------HELLO WORLD (1 row) SELECT concat_lower_or_upper(a := ’Hello’, uppercase := true, b := ’World’); concat_lower_or_upper ----------------------HELLO WORLD (1 row)
4.3.3. Using Mixed Notation The mixed notation combines positional and named notation. However, as already mentioned, named arguments cannot precede positional arguments. For example: SELECT concat_lower_or_upper(’Hello’, ’World’, uppercase := true); concat_lower_or_upper ----------------------HELLO WORLD (1 row)
In the above query, the arguments a and b are specified positionally, while uppercase is specified by name. In this example, that adds little except documentation. With a more complex function having numerous parameters that have default values, named or mixed notation can save a great deal of writing and reduce chances for error. Note: Named and mixed call notations currently cannot be used when calling an aggregate function (but they do work when an aggregate function is used as a window function).
47
Chapter 5. Data Definition This chapter covers how one creates the database structures that will hold one’s data. In a relational database, the raw data is stored in tables, so the majority of this chapter is devoted to explaining how tables are created and modified and what features are available to control what data is stored in the tables. Subsequently, we discuss how tables can be organized into schemas, and how privileges can be assigned to tables. Finally, we will briefly look at other features that affect the data storage, such as inheritance, views, functions, and triggers.
5.1. Table Basics A table in a relational database is much like a table on paper: It consists of rows and columns. The number and order of the columns is fixed, and each column has a name. The number of rows is variable — it reflects how much data is stored at a given moment. SQL does not make any guarantees about the order of the rows in a table. When a table is read, the rows will appear in an unspecified order, unless sorting is explicitly requested. This is covered in Chapter 7. Furthermore, SQL does not assign unique identifiers to rows, so it is possible to have several completely identical rows in a table. This is a consequence of the mathematical model that underlies SQL but is usually not desirable. Later in this chapter we will see how to deal with this issue. Each column has a data type. The data type constrains the set of possible values that can be assigned to a column and assigns semantics to the data stored in the column so that it can be used for computations. For instance, a column declared to be of a numerical type will not accept arbitrary text strings, and the data stored in such a column can be used for mathematical computations. By contrast, a column declared to be of a character string type will accept almost any kind of data but it does not lend itself to mathematical calculations, although other operations such as string concatenation are available. PostgreSQL includes a sizable set of built-in data types that fit many applications. Users can also define their own data types. Most built-in data types have obvious names and semantics, so we defer a detailed explanation to Chapter 8. Some of the frequently used data types are integer for whole numbers, numeric for possibly fractional numbers, text for character strings, date for dates, time for time-of-day values, and timestamp for values containing both date and time. To create a table, you use the aptly named CREATE TABLE command. In this command you specify at least a name for the new table, the names of the columns and the data type of each column. For example: CREATE TABLE my_first_table ( first_column text, second_column integer );
This creates a table named my_first_table with two columns. The first column is named first_column and has a data type of text; the second column has the name second_column and the type integer. The table and column names follow the identifier syntax explained in Section 4.1.1. The type names are usually also identifiers, but there are some exceptions. Note that the column list is comma-separated and surrounded by parentheses. Of course, the previous example was heavily contrived. Normally, you would give names to your tables and columns that convey what kind of data they store. So let’s look at a more realistic example: CREATE TABLE products (
48
Chapter 5. Data Definition product_no integer, name text, price numeric );
(The numeric type can store fractional components, as would be typical of monetary amounts.) Tip: When you create many interrelated tables it is wise to choose a consistent naming pattern for the tables and columns. For instance, there is a choice of using singular or plural nouns for table names, both of which are favored by some theorist or other.
There is a limit on how many columns a table can contain. Depending on the column types, it is between 250 and 1600. However, defining a table with anywhere near this many columns is highly unusual and often a questionable design. If you no longer need a table, you can remove it using the DROP TABLE command. For example: DROP TABLE my_first_table; DROP TABLE products;
Attempting to drop a table that does not exist is an error. Nevertheless, it is common in SQL script files to unconditionally try to drop each table before creating it, ignoring any error messages, so that the script works whether or not the table exists. (If you like, you can use the DROP TABLE IF EXISTS variant to avoid the error messages, but this is not standard SQL.) If you need to modify a table that already exists, see Section 5.5 later in this chapter. With the tools discussed so far you can create fully functional tables. The remainder of this chapter is concerned with adding features to the table definition to ensure data integrity, security, or convenience. If you are eager to fill your tables with data now you can skip ahead to Chapter 6 and read the rest of this chapter later.
5.2. Default Values A column can be assigned a default value. When a new row is created and no values are specified for some of the columns, those columns will be filled with their respective default values. A data manipulation command can also request explicitly that a column be set to its default value, without having to know what that value is. (Details about data manipulation commands are in Chapter 6.) If no default value is declared explicitly, the default value is the null value. This usually makes sense because a null value can be considered to represent unknown data. In a table definition, default values are listed after the column data type. For example: CREATE TABLE products ( product_no integer, name text, price numeric DEFAULT 9.99 );
The default value can be an expression, which will be evaluated whenever the default value is inserted (not when the table is created). A common example is for a timestamp column to have a default of
49
Chapter 5. Data Definition CURRENT_TIMESTAMP, so that it gets set to the time of row insertion. Another common example is generating a “serial number” for each row. In PostgreSQL this is typically done by something like: CREATE TABLE products ( product_no integer DEFAULT nextval(’products_product_no_seq’), ... );
where the nextval() function supplies successive values from a sequence object (see Section 9.16). This arrangement is sufficiently common that there’s a special shorthand for it: CREATE TABLE products ( product_no SERIAL, ... );
The SERIAL shorthand is discussed further in Section 8.1.4.
5.3. Constraints Data types are a way to limit the kind of data that can be stored in a table. For many applications, however, the constraint they provide is too coarse. For example, a column containing a product price should probably only accept positive values. But there is no standard data type that accepts only positive numbers. Another issue is that you might want to constrain column data with respect to other columns or rows. For example, in a table containing product information, there should be only one row for each product number. To that end, SQL allows you to define constraints on columns and tables. Constraints give you as much control over the data in your tables as you wish. If a user attempts to store data in a column that would violate a constraint, an error is raised. This applies even if the value came from the default value definition.
5.3.1. Check Constraints A check constraint is the most generic constraint type. It allows you to specify that the value in a certain column must satisfy a Boolean (truth-value) expression. For instance, to require positive product prices, you could use: CREATE TABLE products ( product_no integer, name text, price numeric CHECK (price > 0) );
As you see, the constraint definition comes after the data type, just like default value definitions. Default values and constraints can be listed in any order. A check constraint consists of the key word CHECK followed by an expression in parentheses. The check constraint expression should involve the column thus constrained, otherwise the constraint would not make too much sense. You can also give the constraint a separate name. This clarifies error messages and allows you to refer to the constraint when you need to change it. The syntax is:
So, to specify a named constraint, use the key word CONSTRAINT followed by an identifier followed by the constraint definition. (If you don’t specify a constraint name in this way, the system chooses a name for you.) A check constraint can also refer to several columns. Say you store a regular price and a discounted price, and you want to ensure that the discounted price is lower than the regular price: CREATE TABLE products ( product_no integer, name text, price numeric CHECK (price > 0), discounted_price numeric CHECK (discounted_price > 0), CHECK (price > discounted_price) );
The first two constraints should look familiar. The third one uses a new syntax. It is not attached to a particular column, instead it appears as a separate item in the comma-separated column list. Column definitions and these constraint definitions can be listed in mixed order. We say that the first two constraints are column constraints, whereas the third one is a table constraint because it is written separately from any one column definition. Column constraints can also be written as table constraints, while the reverse is not necessarily possible, since a column constraint is supposed to refer to only the column it is attached to. (PostgreSQL doesn’t enforce that rule, but you should follow it if you want your table definitions to work with other database systems.) The above example could also be written as: CREATE TABLE products ( product_no integer, name text, price numeric, CHECK (price > 0), discounted_price numeric, CHECK (discounted_price > 0), CHECK (price > discounted_price) );
or even: CREATE TABLE products ( product_no integer, name text, price numeric CHECK (price > 0), discounted_price numeric, CHECK (discounted_price > 0 AND price > discounted_price) );
It’s a matter of taste. Names can be assigned to table constraints in the same way as column constraints: CREATE TABLE products (
It should be noted that a check constraint is satisfied if the check expression evaluates to true or the null value. Since most expressions will evaluate to the null value if any operand is null, they will not prevent null values in the constrained columns. To ensure that a column does not contain null values, the not-null constraint described in the next section can be used.
5.3.2. Not-Null Constraints A not-null constraint simply specifies that a column must not assume the null value. A syntax example: CREATE TABLE products ( product_no integer NOT NULL, name text NOT NULL, price numeric );
A not-null constraint is always written as a column constraint. A not-null constraint is functionally equivalent to creating a check constraint CHECK (column_name IS NOT NULL), but in PostgreSQL creating an explicit not-null constraint is more efficient. The drawback is that you cannot give explicit names to not-null constraints created this way. Of course, a column can have more than one constraint. Just write the constraints one after another: CREATE TABLE products ( product_no integer NOT NULL, name text NOT NULL, price numeric NOT NULL CHECK (price > 0) );
The order doesn’t matter. It does not necessarily determine in which order the constraints are checked. The NOT NULL constraint has an inverse: the NULL constraint. This does not mean that the column must be null, which would surely be useless. Instead, this simply selects the default behavior that the column might be null. The NULL constraint is not present in the SQL standard and should not be used in portable applications. (It was only added to PostgreSQL to be compatible with some other database systems.) Some users, however, like it because it makes it easy to toggle the constraint in a script file. For example, you could start with: CREATE TABLE products ( product_no integer NULL, name text NULL, price numeric NULL );
and then insert the NOT key word where desired.
52
Chapter 5. Data Definition Tip: In most database designs the majority of columns should be marked not null.
5.3.3. Unique Constraints Unique constraints ensure that the data contained in a column, or a group of columns, is unique among all the rows in the table. The syntax is: CREATE TABLE products ( product_no integer UNIQUE, name text, price numeric );
when written as a column constraint, and: CREATE TABLE products ( product_no integer, name text, price numeric, UNIQUE (product_no) );
when written as a table constraint. To define a unique constraint for a group of columns, write it as a table constraint with the column names separated by commas: CREATE TABLE example ( a integer, b integer, c integer, UNIQUE (a, c) );
This specifies that the combination of values in the indicated columns is unique across the whole table, though any one of the columns need not be (and ordinarily isn’t) unique. You can assign your own name for a unique constraint, in the usual way: CREATE TABLE products ( product_no integer CONSTRAINT must_be_different UNIQUE, name text, price numeric );
Adding a unique constraint will automatically create a unique B-tree index on the column or group of columns listed in the constraint. A uniqueness restriction covering only some rows cannot be written as a unique constraint, but it is possible to enforce such a restriction by creating a unique partial index. In general, a unique constraint is violated if there is more than one row in the table where the values of all of the columns included in the constraint are equal. However, two null values are never considered equal in this comparison. That means even in the presence of a unique constraint it is possible to store duplicate rows that contain a null value in at least one of the constrained columns. This behavior
53
Chapter 5. Data Definition conforms to the SQL standard, but we have heard that other SQL databases might not follow this rule. So be careful when developing applications that are intended to be portable.
5.3.4. Primary Keys A primary key constraint indicates that a column, or group of columns, can be used as a unique identifier for rows in the table. This requires that the values be both unique and not null. So, the following two table definitions accept the same data: CREATE TABLE products ( product_no integer UNIQUE NOT NULL, name text, price numeric ); CREATE TABLE products ( product_no integer PRIMARY KEY, name text, price numeric );
Primary keys can span more than one column; the syntax is similar to unique constraints: CREATE TABLE example ( a integer, b integer, c integer, PRIMARY KEY (a, c) );
Adding a primary key will automatically create a unique B-tree index on the column or group of columns listed in the primary key, and will force the column(s) to be marked NOT NULL. A table can have at most one primary key. (There can be any number of unique and not-null constraints, which are functionally almost the same thing, but only one can be identified as the primary key.) Relational database theory dictates that every table must have a primary key. This rule is not enforced by PostgreSQL, but it is usually best to follow it. Primary keys are useful both for documentation purposes and for client applications. For example, a GUI application that allows modifying row values probably needs to know the primary key of a table to be able to identify rows uniquely. There are also various ways in which the database system makes use of a primary key if one has been declared; for example, the primary key defines the default target column(s) for foreign keys referencing its table.
5.3.5. Foreign Keys A foreign key constraint specifies that the values in a column (or a group of columns) must match the values appearing in some row of another table. We say this maintains the referential integrity between two related tables. Say you have the product table that we have used several times already:
54
Chapter 5. Data Definition CREATE TABLE products ( product_no integer PRIMARY KEY, name text, price numeric );
Let’s also assume you have a table storing orders of those products. We want to ensure that the orders table only contains orders of products that actually exist. So we define a foreign key constraint in the orders table that references the products table: CREATE TABLE orders ( order_id integer PRIMARY KEY, product_no integer REFERENCES products (product_no), quantity integer );
Now it is impossible to create orders with non-NULL product_no entries that do not appear in the products table. We say that in this situation the orders table is the referencing table and the products table is the referenced table. Similarly, there are referencing and referenced columns. You can also shorten the above command to: CREATE TABLE orders ( order_id integer PRIMARY KEY, product_no integer REFERENCES products, quantity integer );
because in absence of a column list the primary key of the referenced table is used as the referenced column(s). A foreign key can also constrain and reference a group of columns. As usual, it then needs to be written in table constraint form. Here is a contrived syntax example: CREATE TABLE t1 ( a integer PRIMARY KEY, b integer, c integer, FOREIGN KEY (b, c) REFERENCES other_table (c1, c2) );
Of course, the number and type of the constrained columns need to match the number and type of the referenced columns. You can assign your own name for a foreign key constraint, in the usual way. A table can have more than one foreign key constraint. This is used to implement many-to-many relationships between tables. Say you have tables about products and orders, but now you want to allow one order to contain possibly many products (which the structure above did not allow). You could use this table structure: CREATE TABLE products ( product_no integer PRIMARY KEY, name text, price numeric );
Notice that the primary key overlaps with the foreign keys in the last table. We know that the foreign keys disallow creation of orders that do not relate to any products. But what if a product is removed after an order is created that references it? SQL allows you to handle that as well. Intuitively, we have a few options: Disallow deleting a referenced product • Delete the orders as well • Something else? •
To illustrate this, let’s implement the following policy on the many-to-many relationship example above: when someone wants to remove a product that is still referenced by an order (via order_items), we disallow it. If someone removes an order, the order items are removed as well: CREATE TABLE products ( product_no integer PRIMARY KEY, name text, price numeric ); CREATE TABLE orders ( order_id integer PRIMARY KEY, shipping_address text, ... ); CREATE TABLE order_items ( product_no integer REFERENCES products ON DELETE RESTRICT, order_id integer REFERENCES orders ON DELETE CASCADE, quantity integer, PRIMARY KEY (product_no, order_id) );
Restricting and cascading deletes are the two most common options. RESTRICT prevents deletion of a referenced row. NO ACTION means that if any referencing rows still exist when the constraint is checked, an error is raised; this is the default behavior if you do not specify anything. (The essential difference between these two choices is that NO ACTION allows the check to be deferred until later in the transaction, whereas RESTRICT does not.) CASCADE specifies that when a referenced row is deleted, row(s) referencing it should be automatically deleted as well. There are two other options: SET NULL and SET DEFAULT. These cause the referencing column(s) in the referencing row(s) to be
56
Chapter 5. Data Definition set to nulls or their default values, respectively, when the referenced row is deleted. Note that these do not excuse you from observing any constraints. For example, if an action specifies SET DEFAULT but the default value would not satisfy the foreign key constraint, the operation will fail. Analogous to ON DELETE there is also ON UPDATE which is invoked when a referenced column is changed (updated). The possible actions are the same. In this case, CASCADE means that the updated values of the referenced column(s) should be copied into the referencing row(s). Normally, a referencing row need not satisfy the foreign key constraint if any of its referencing columns are null. If MATCH FULL is added to the foreign key declaration, a referencing row escapes satisfying the constraint only if all its referencing columns are null (so a mix of null and non-null values is guaranteed to fail a MATCH FULL constraint). If you don’t want referencing rows to be able to avoid satisfying the foreign key constraint, declare the referencing column(s) as NOT NULL. A foreign key must reference columns that either are a primary key or form a unique constraint. This means that the referenced columns always have an index (the one underlying the primary key or unique constraint); so checks on whether a referencing row has a match will be efficient. Since a DELETE of a row from the referenced table or an UPDATE of a referenced column will require a scan of the referencing table for rows matching the old value, it is often a good idea to index the referencing columns too. Because this is not always needed, and there are many choices available on how to index, declaration of a foreign key constraint does not automatically create an index on the referencing columns. More information about updating and deleting data is in Chapter 6. Also see the description of foreign key constraint syntax in the reference documentation for CREATE TABLE.
5.3.6. Exclusion Constraints Exclusion constraints ensure that if any two rows are compared on the specified columns or expressions using the specified operators, at least one of these operator comparisons will return false or null. The syntax is: CREATE TABLE circles ( c circle, EXCLUDE USING gist (c WITH &&) );
See also CREATE TABLE ... CONSTRAINT ... EXCLUDE for details. Adding an exclusion constraint will automatically create an index of the type specified in the constraint declaration.
5.4. System Columns Every table has several system columns that are implicitly defined by the system. Therefore, these names cannot be used as names of user-defined columns. (Note that these restrictions are separate from whether the name is a key word or not; quoting a name will not allow you to escape these restrictions.) You do not really need to be concerned about these columns; just know they exist.
57
Chapter 5. Data Definition oid
The object identifier (object ID) of a row. This column is only present if the table was created using WITH OIDS, or if the default_with_oids configuration variable was set at the time. This column is of type oid (same name as the column); see Section 8.18 for more information about the type. tableoid
The OID of the table containing this row. This column is particularly handy for queries that select from inheritance hierarchies (see Section 5.8), since without it, it’s difficult to tell which individual table a row came from. The tableoid can be joined against the oid column of pg_class to obtain the table name. xmin
The identity (transaction ID) of the inserting transaction for this row version. (A row version is an individual state of a row; each update of a row creates a new row version for the same logical row.) cmin
The command identifier (starting at zero) within the inserting transaction. xmax
The identity (transaction ID) of the deleting transaction, or zero for an undeleted row version. It is possible for this column to be nonzero in a visible row version. That usually indicates that the deleting transaction hasn’t committed yet, or that an attempted deletion was rolled back. cmax
The command identifier within the deleting transaction, or zero. ctid
The physical location of the row version within its table. Note that although the ctid can be used to locate the row version very quickly, a row’s ctid will change if it is updated or moved by VACUUM FULL. Therefore ctid is useless as a long-term row identifier. The OID, or even better a user-defined serial number, should be used to identify logical rows. OIDs are 32-bit quantities and are assigned from a single cluster-wide counter. In a large or long-lived database, it is possible for the counter to wrap around. Hence, it is bad practice to assume that OIDs are unique, unless you take steps to ensure that this is the case. If you need to identify the rows in a table, using a sequence generator is strongly recommended. However, OIDs can be used as well, provided that a few additional precautions are taken: •
A unique constraint should be created on the OID column of each table for which the OID will be used to identify rows. When such a unique constraint (or unique index) exists, the system takes care not to generate an OID matching an already-existing row. (Of course, this is only possible if the table contains fewer than 232 (4 billion) rows, and in practice the table size had better be much less than that, or performance might suffer.)
•
OIDs should never be assumed to be unique across tables; use the combination of tableoid and row OID if you need a database-wide identifier.
•
Of course, the tables in question must be created WITH OIDS. As of PostgreSQL 8.1, WITHOUT OIDS is the default.
58
Chapter 5. Data Definition Transaction identifiers are also 32-bit quantities. In a long-lived database it is possible for transaction IDs to wrap around. This is not a fatal problem given appropriate maintenance procedures; see Chapter 23 for details. It is unwise, however, to depend on the uniqueness of transaction IDs over the long term (more than one billion transactions). Command identifiers are also 32-bit quantities. This creates a hard limit of 232 (4 billion) SQL commands within a single transaction. In practice this limit is not a problem — note that the limit is on the number of SQL commands, not the number of rows processed. Also, as of PostgreSQL 8.3, only commands that actually modify the database contents will consume a command identifier.
5.5. Modifying Tables When you create a table and you realize that you made a mistake, or the requirements of the application change, you can drop the table and create it again. But this is not a convenient option if the table is already filled with data, or if the table is referenced by other database objects (for instance a foreign key constraint). Therefore PostgreSQL provides a family of commands to make modifications to existing tables. Note that this is conceptually distinct from altering the data contained in the table: here we are interested in altering the definition, or structure, of the table. You can: • • • • • • • •
All these actions are performed using the ALTER TABLE command, whose reference page contains details beyond those given here.
5.5.1. Adding a Column To add a column, use a command like: ALTER TABLE products ADD COLUMN description text;
The new column is initially filled with whatever default value is given (null if you don’t specify a DEFAULT clause). You can also define constraints on the column at the same time, using the usual syntax: ALTER TABLE products ADD COLUMN description text CHECK (description ”);
In fact all the options that can be applied to a column description in CREATE TABLE can be used here. Keep in mind however that the default value must satisfy the given constraints, or the ADD will fail. Alternatively, you can add constraints later (see below) after you’ve filled in the new column correctly. Tip: Adding a column with a default requires updating each row of the table (to store the new column value). However, if no default is specified, PostgreSQL is able to avoid the physical update. So if you intend to fill the column with mostly nondefault values, it’s best to add the column with
59
Chapter 5. Data Definition no default, insert the correct values using UPDATE, and then add any desired default as described below.
5.5.2. Removing a Column To remove a column, use a command like: ALTER TABLE products DROP COLUMN description;
Whatever data was in the column disappears. Table constraints involving the column are dropped, too. However, if the column is referenced by a foreign key constraint of another table, PostgreSQL will not silently drop that constraint. You can authorize dropping everything that depends on the column by adding CASCADE: ALTER TABLE products DROP COLUMN description CASCADE;
See Section 5.12 for a description of the general mechanism behind this.
5.5.3. Adding a Constraint To add a constraint, the table constraint syntax is used. For example: ALTER TABLE products ADD CHECK (name ”); ALTER TABLE products ADD CONSTRAINT some_name UNIQUE (product_no); ALTER TABLE products ADD FOREIGN KEY (product_group_id) REFERENCES product_groups;
To add a not-null constraint, which cannot be written as a table constraint, use this syntax: ALTER TABLE products ALTER COLUMN product_no SET NOT NULL;
The constraint will be checked immediately, so the table data must satisfy the constraint before it can be added.
5.5.4. Removing a Constraint To remove a constraint you need to know its name. If you gave it a name then that’s easy. Otherwise the system assigned a generated name, which you need to find out. The psql command \d tablename can be helpful here; other interfaces might also provide a way to inspect table details. Then the command is: ALTER TABLE products DROP CONSTRAINT some_name;
(If you are dealing with a generated constraint name like $2, don’t forget that you’ll need to doublequote it to make it a valid identifier.) As with dropping a column, you need to add CASCADE if you want to drop a constraint that something else depends on. An example is that a foreign key constraint depends on a unique or primary key constraint on the referenced column(s).
60
Chapter 5. Data Definition This works the same for all constraint types except not-null constraints. To drop a not null constraint use: ALTER TABLE products ALTER COLUMN product_no DROP NOT NULL;
(Recall that not-null constraints do not have names.)
5.5.5. Changing a Column’s Default Value To set a new default for a column, use a command like: ALTER TABLE products ALTER COLUMN price SET DEFAULT 7.77;
Note that this doesn’t affect any existing rows in the table, it just changes the default for future INSERT commands. To remove any default value, use: ALTER TABLE products ALTER COLUMN price DROP DEFAULT;
This is effectively the same as setting the default to null. As a consequence, it is not an error to drop a default where one hadn’t been defined, because the default is implicitly the null value.
5.5.6. Changing a Column’s Data Type To convert a column to a different data type, use a command like: ALTER TABLE products ALTER COLUMN price TYPE numeric(10,2);
This will succeed only if each existing entry in the column can be converted to the new type by an implicit cast. If a more complex conversion is needed, you can add a USING clause that specifies how to compute the new values from the old. PostgreSQL will attempt to convert the column’s default value (if any) to the new type, as well as any constraints that involve the column. But these conversions might fail, or might produce surprising results. It’s often best to drop any constraints on the column before altering its type, and then add back suitably modified constraints afterwards.
5.5.7. Renaming a Column To rename a column: ALTER TABLE products RENAME COLUMN product_no TO product_number;
5.5.8. Renaming a Table To rename a table: ALTER TABLE products RENAME TO items;
61
Chapter 5. Data Definition
5.6. Privileges When an object is created, it is assigned an owner. The owner is normally the role that executed the creation statement. For most kinds of objects, the initial state is that only the owner (or a superuser) can do anything with the object. To allow other roles to use it, privileges must be granted. There are different kinds of privileges: SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER, CREATE, CONNECT, TEMPORARY, EXECUTE, and USAGE. The privileges applicable to a particular object vary depending on the object’s type (table, function, etc). For complete information on the different types of privileges supported by PostgreSQL, refer to the GRANT reference page. The following sections and chapters will also show you how those privileges are used. The right to modify or destroy an object is always the privilege of the owner only. An object can be assigned to a new owner with an ALTER command of the appropriate kind for the object, e.g. ALTER TABLE. Superusers can always do this; ordinary roles can only do it if they are both the current owner of the object (or a member of the owning role) and a member of the new owning role. To assign privileges, the GRANT command is used. For example, if joe is an existing user, and accounts is an existing table, the privilege to update the table can be granted with: GRANT UPDATE ON accounts TO joe;
Writing ALL in place of a specific privilege grants all privileges that are relevant for the object type. The special “user” name PUBLIC can be used to grant a privilege to every user on the system. Also, “group” roles can be set up to help manage privileges when there are many users of a database — for details see Chapter 20. To revoke a privilege, use the fittingly named REVOKE command: REVOKE ALL ON accounts FROM PUBLIC;
The special privileges of the object owner (i.e., the right to do DROP, GRANT, REVOKE, etc.) are always implicit in being the owner, and cannot be granted or revoked. But the object owner can choose to revoke his own ordinary privileges, for example to make a table read-only for himself as well as others. Ordinarily, only the object’s owner (or a superuser) can grant or revoke privileges on an object. However, it is possible to grant a privilege “with grant option”, which gives the recipient the right to grant it in turn to others. If the grant option is subsequently revoked then all who received the privilege from that recipient (directly or through a chain of grants) will lose the privilege. For details see the GRANT and REVOKE reference pages.
5.7. Schemas A PostgreSQL database cluster contains one or more named databases. Users and groups of users are shared across the entire cluster, but no other data is shared across databases. Any given client con-
62
Chapter 5. Data Definition nection to the server can access only the data in a single database, the one specified in the connection request. Note: Users of a cluster do not necessarily have the privilege to access every database in the cluster. Sharing of user names means that there cannot be different users named, say, joe in two databases in the same cluster; but the system can be configured to allow joe access to only some of the databases.
A database contains one or more named schemas, which in turn contain tables. Schemas also contain other kinds of named objects, including data types, functions, and operators. The same object name can be used in different schemas without conflict; for example, both schema1 and myschema can contain tables named mytable. Unlike databases, schemas are not rigidly separated: a user can access objects in any of the schemas in the database he is connected to, if he has privileges to do so. There are several reasons why one might want to use schemas: •
To allow many users to use one database without interfering with each other.
•
To organize database objects into logical groups to make them more manageable.
•
Third-party applications can be put into separate schemas so they do not collide with the names of other objects.
Schemas are analogous to directories at the operating system level, except that schemas cannot be nested.
5.7.1. Creating a Schema To create a schema, use the CREATE SCHEMA command. Give the schema a name of your choice. For example: CREATE SCHEMA myschema;
To create or access objects in a schema, write a qualified name consisting of the schema name and table name separated by a dot: schema.table
This works anywhere a table name is expected, including the table modification commands and the data access commands discussed in the following chapters. (For brevity we will speak of tables only, but the same ideas apply to other kinds of named objects, such as types and functions.) Actually, the even more general syntax database.schema.table
can be used too, but at present this is just for pro forma compliance with the SQL standard. If you write a database name, it must be the same as the database you are connected to. So to create a table in the new schema, use: CREATE TABLE myschema.mytable ( ... );
63
Chapter 5. Data Definition
To drop a schema if it’s empty (all objects in it have been dropped), use: DROP SCHEMA myschema;
To drop a schema including all contained objects, use: DROP SCHEMA myschema CASCADE;
See Section 5.12 for a description of the general mechanism behind this. Often you will want to create a schema owned by someone else (since this is one of the ways to restrict the activities of your users to well-defined namespaces). The syntax for that is: CREATE SCHEMA schemaname AUTHORIZATION username;
You can even omit the schema name, in which case the schema name will be the same as the user name. See Section 5.7.6 for how this can be useful. Schema names beginning with pg_ are reserved for system purposes and cannot be created by users.
5.7.2. The Public Schema In the previous sections we created tables without specifying any schema names. By default such tables (and other objects) are automatically put into a schema named “public”. Every new database contains such a schema. Thus, the following are equivalent: CREATE TABLE products ( ... );
and: CREATE TABLE public.products ( ... );
5.7.3. The Schema Search Path Qualified names are tedious to write, and it’s often best not to wire a particular schema name into applications anyway. Therefore tables are often referred to by unqualified names, which consist of just the table name. The system determines which table is meant by following a search path, which is a list of schemas to look in. The first matching table in the search path is taken to be the one wanted. If there is no match in the search path, an error is reported, even if matching table names exist in other schemas in the database. The first schema named in the search path is called the current schema. Aside from being the first schema searched, it is also the schema in which new tables will be created if the CREATE TABLE command does not specify a schema name. To show the current search path, use the following command: SHOW search_path;
In the default setup this returns: search_path
64
Chapter 5. Data Definition -------------"$user",public
The first element specifies that a schema with the same name as the current user is to be searched. If no such schema exists, the entry is ignored. The second element refers to the public schema that we have seen already. The first schema in the search path that exists is the default location for creating new objects. That is the reason that by default objects are created in the public schema. When objects are referenced in any other context without schema qualification (table modification, data modification, or query commands) the search path is traversed until a matching object is found. Therefore, in the default configuration, any unqualified access again can only refer to the public schema. To put our new schema in the path, we use: SET search_path TO myschema,public;
(We omit the $user here because we have no immediate need for it.) And then we can access the table without schema qualification: DROP TABLE mytable;
Also, since myschema is the first element in the path, new objects would by default be created in it. We could also have written: SET search_path TO myschema;
Then we no longer have access to the public schema without explicit qualification. There is nothing special about the public schema except that it exists by default. It can be dropped, too. See also Section 9.25 for other ways to manipulate the schema search path. The search path works in the same way for data type names, function names, and operator names as it does for table names. Data type and function names can be qualified in exactly the same way as table names. If you need to write a qualified operator name in an expression, there is a special provision: you must write OPERATOR(schema.operator)
This is needed to avoid syntactic ambiguity. An example is: SELECT 3 OPERATOR(pg_catalog.+) 4;
In practice one usually relies on the search path for operators, so as not to have to write anything so ugly as that.
5.7.4. Schemas and Privileges By default, users cannot access any objects in schemas they do not own. To allow that, the owner of the schema must grant the USAGE privilege on the schema. To allow users to make use of the objects in the schema, additional privileges might need to be granted, as appropriate for the object. A user can also be allowed to create objects in someone else’s schema. To allow that, the CREATE privilege on the schema needs to be granted. Note that by default, everyone has CREATE and USAGE privileges on the schema public. This allows all users that are able to connect to a given database to create objects in its public schema. If you do not want to allow that, you can revoke that privilege:
65
Chapter 5. Data Definition REVOKE CREATE ON SCHEMA public FROM PUBLIC;
(The first “public” is the schema, the second “public” means “every user”. In the first sense it is an identifier, in the second sense it is a key word, hence the different capitalization; recall the guidelines from Section 4.1.1.)
5.7.5. The System Catalog Schema In addition to public and user-created schemas, each database contains a pg_catalog schema, which contains the system tables and all the built-in data types, functions, and operators. pg_catalog is always effectively part of the search path. If it is not named explicitly in the path then it is implicitly searched before searching the path’s schemas. This ensures that built-in names will always be findable. However, you can explicitly place pg_catalog at the end of your search path if you prefer to have user-defined names override built-in names. In PostgreSQL versions before 7.3, table names beginning with pg_ were reserved. This is no longer true: you can create such a table name if you wish, in any non-system schema. However, it’s best to continue to avoid such names, to ensure that you won’t suffer a conflict if some future version defines a system table named the same as your table. (With the default search path, an unqualified reference to your table name would then be resolved as the system table instead.) System tables will continue to follow the convention of having names beginning with pg_, so that they will not conflict with unqualified user-table names so long as users avoid the pg_ prefix.
5.7.6. Usage Patterns Schemas can be used to organize your data in many ways. There are a few usage patterns that are recommended and are easily supported by the default configuration:
•
If you do not create any schemas then all users access the public schema implicitly. This simulates the situation where schemas are not available at all. This setup is mainly recommended when there is only a single user or a few cooperating users in a database. This setup also allows smooth transition from the non-schema-aware world.
•
You can create a schema for each user with the same name as that user. Recall that the default search path starts with $user, which resolves to the user name. Therefore, if each user has a separate schema, they access their own schemas by default. If you use this setup then you might also want to revoke access to the public schema (or drop it altogether), so users are truly constrained to their own schemas.
•
To install shared applications (tables to be used by everyone, additional functions provided by third parties, etc.), put them into separate schemas. Remember to grant appropriate privileges to allow the other users to access them. Users can then refer to these additional objects by qualifying the names with a schema name, or they can put the additional schemas into their search path, as they choose.
66
Chapter 5. Data Definition
5.7.7. Portability In the SQL standard, the notion of objects in the same schema being owned by different users does not exist. Moreover, some implementations do not allow you to create schemas that have a different name than their owner. In fact, the concepts of schema and user are nearly equivalent in a database system that implements only the basic schema support specified in the standard. Therefore, many users consider qualified names to really consist of username.tablename. This is how PostgreSQL will effectively behave if you create a per-user schema for every user. Also, there is no concept of a public schema in the SQL standard. For maximum conformance to the standard, you should not use (perhaps even remove) the public schema. Of course, some SQL database systems might not implement schemas at all, or provide namespace support by allowing (possibly limited) cross-database access. If you need to work with those systems, then maximum portability would be achieved by not using schemas at all.
5.8. Inheritance PostgreSQL implements table inheritance, which can be a useful tool for database designers. (SQL:1999 and later define a type inheritance feature, which differs in many respects from the features described here.) Let’s start with an example: suppose we are trying to build a data model for cities. Each state has many cities, but only one capital. We want to be able to quickly retrieve the capital city for any particular state. This can be done by creating two tables, one for state capitals and one for cities that are not capitals. However, what happens when we want to ask for data about a city, regardless of whether it is a capital or not? The inheritance feature can help to resolve this problem. We define the capitals table so that it inherits from cities: CREATE TABLE cities name population altitude );
( text, float, int
-- in feet
CREATE TABLE capitals ( state char(2) ) INHERITS (cities);
In this case, the capitals table inherits all the columns of its parent table, cities. State capitals also have an extra column, state, that shows their state. In PostgreSQL, a table can inherit from zero or more other tables, and a query can reference either all rows of a table or all rows of a table plus all of its descendant tables. The latter behavior is the default. For example, the following query finds the names of all cities, including state capitals, that are located at an altitude over 500 feet: SELECT name, altitude FROM cities WHERE altitude > 500;
Given the sample data from the PostgreSQL tutorial (see Section 2.1), this returns: name | altitude -----------+----------
67
Chapter 5. Data Definition Las Vegas | Mariposa | Madison |
2174 1953 845
On the other hand, the following query finds all the cities that are not state capitals and are situated at an altitude over 500 feet: SELECT name, altitude FROM ONLY cities WHERE altitude > 500; name | altitude -----------+---------Las Vegas | 2174 Mariposa | 1953
Here the ONLY keyword indicates that the query should apply only to cities, and not any tables below cities in the inheritance hierarchy. Many of the commands that we have already discussed — SELECT, UPDATE and DELETE — support the ONLY keyword. You can also write the table name with a trailing * to explicitly specify that descendant tables are included: SELECT name, altitude FROM cities* WHERE altitude > 500;
Writing * is not necessary, since this behavior is the default (unless you have changed the setting of the sql_inheritance configuration option). However writing * might be useful to emphasize that additional tables will be searched. In some cases you might wish to know which table a particular row originated from. There is a system column called tableoid in each table which can tell you the originating table: SELECT c.tableoid, c.name, c.altitude FROM cities c WHERE c.altitude > 500;
which returns: tableoid | name | altitude ----------+-----------+---------139793 | Las Vegas | 2174 139793 | Mariposa | 1953 139798 | Madison | 845
(If you try to reproduce this example, you will probably get different numeric OIDs.) By doing a join with pg_class you can see the actual table names: SELECT p.relname, c.name, c.altitude FROM cities c, pg_class p WHERE c.altitude > 500 AND c.tableoid = p.oid;
which returns:
68
Chapter 5. Data Definition relname | name | altitude ----------+-----------+---------cities | Las Vegas | 2174 cities | Mariposa | 1953 capitals | Madison | 845
Inheritance does not automatically propagate data from INSERT or COPY commands to other tables in the inheritance hierarchy. In our example, the following INSERT statement will fail: INSERT INTO cities (name, population, altitude, state) VALUES (’New York’, NULL, NULL, ’NY’);
We might hope that the data would somehow be routed to the capitals table, but this does not happen: INSERT always inserts into exactly the table specified. In some cases it is possible to redirect the insertion using a rule (see Chapter 38). However that does not help for the above case because the cities table does not contain the column state, and so the command will be rejected before the rule can be applied. All check constraints and not-null constraints on a parent table are automatically inherited by its children. Other types of constraints (unique, primary key, and foreign key constraints) are not inherited. A table can inherit from more than one parent table, in which case it has the union of the columns defined by the parent tables. Any columns declared in the child table’s definition are added to these. If the same column name appears in multiple parent tables, or in both a parent table and the child’s definition, then these columns are “merged” so that there is only one such column in the child table. To be merged, columns must have the same data types, else an error is raised. The merged column will have copies of all the check constraints coming from any one of the column definitions it came from, and will be marked not-null if any of them are. Table inheritance is typically established when the child table is created, using the INHERITS clause of the CREATE TABLE statement. Alternatively, a table which is already defined in a compatible way can have a new parent relationship added, using the INHERIT variant of ALTER TABLE. To do this the new child table must already include columns with the same names and types as the columns of the parent. It must also include check constraints with the same names and check expressions as those of the parent. Similarly an inheritance link can be removed from a child using the NO INHERIT variant of ALTER TABLE. Dynamically adding and removing inheritance links like this can be useful when the inheritance relationship is being used for table partitioning (see Section 5.9). One convenient way to create a compatible table that will later be made a new child is to use the LIKE clause in CREATE TABLE. This creates a new table with the same columns as the source table. If there are any CHECK constraints defined on the source table, the INCLUDING CONSTRAINTS option to LIKE should be specified, as the new child must have constraints matching the parent to be considered compatible. A parent table cannot be dropped while any of its children remain. Neither can columns or check constraints of child tables be dropped or altered if they are inherited from any parent tables. If you wish to remove a table and all of its descendants, one easy way is to drop the parent table with the CASCADE option. ALTER TABLE will propagate any changes in column data definitions and check constraints down the inheritance hierarchy. Again, dropping columns that are depended on by other tables is only possible when using the CASCADE option. ALTER TABLE follows the same rules for duplicate column merging and rejection that apply during CREATE TABLE. Note how table access permissions are handled. Querying a parent table can automatically access data in child tables without further access privilege checking. This preserves the appearance that the data
69
Chapter 5. Data Definition is (also) in the parent table. Accessing the child tables directly is, however, not automatically allowed and would require further privileges to be granted.
5.8.1. Caveats Note that not all SQL commands are able to work on inheritance hierarchies. Commands that are used for data querying, data modification, or schema modification (e.g., SELECT, UPDATE, DELETE, most variants of ALTER TABLE, but not INSERT or ALTER TABLE ... RENAME) typically default to including child tables and support the ONLY notation to exclude them. Commands that do database maintenance and tuning (e.g., REINDEX, VACUUM) typically only work on individual, physical tables and do not support recursing over inheritance hierarchies. The respective behavior of each individual command is documented in its reference page (Reference I, SQL Commands). A serious limitation of the inheritance feature is that indexes (including unique constraints) and foreign key constraints only apply to single tables, not to their inheritance children. This is true on both the referencing and referenced sides of a foreign key constraint. Thus, in the terms of the above example: •
If we declared cities.name to be UNIQUE or a PRIMARY KEY, this would not stop the capitals table from having rows with names duplicating rows in cities. And those duplicate rows would by default show up in queries from cities. In fact, by default capitals would have no unique constraint at all, and so could contain multiple rows with the same name. You could add a unique constraint to capitals, but this would not prevent duplication compared to cities.
•
Similarly, if we were to specify that cities.name REFERENCES some other table, this constraint would not automatically propagate to capitals. In this case you could work around it by manually adding the same REFERENCES constraint to capitals.
•
Specifying that another table’s column REFERENCES cities(name) would allow the other table to contain city names, but not capital names. There is no good workaround for this case.
These deficiencies will probably be fixed in some future release, but in the meantime considerable care is needed in deciding whether inheritance is useful for your application.
5.9. Partitioning PostgreSQL supports basic table partitioning. This section describes why and how to implement partitioning as part of your database design.
5.9.1. Overview Partitioning refers to splitting what is logically one large table into smaller physical pieces. Partitioning can provide several benefits:
•
Query performance can be improved dramatically in certain situations, particularly when most of the heavily accessed rows of the table are in a single partition or a small number of partitions. The partitioning substitutes for leading columns of indexes, reducing index size and making it more likely that the heavily-used parts of the indexes fit in memory.
70
Chapter 5. Data Definition •
When queries or updates access a large percentage of a single partition, performance can be improved by taking advantage of sequential scan of that partition instead of using an index and random access reads scattered across the whole table.
•
Bulk loads and deletes can be accomplished by adding or removing partitions, if that requirement is planned into the partitioning design. ALTER TABLE NO INHERIT and DROP TABLE are both far faster than a bulk operation. These commands also entirely avoid the VACUUM overhead caused by a bulk DELETE.
•
Seldom-used data can be migrated to cheaper and slower storage media.
The benefits will normally be worthwhile only when a table would otherwise be very large. The exact point at which a table will benefit from partitioning depends on the application, although a rule of thumb is that the size of the table should exceed the physical memory of the database server. Currently, PostgreSQL supports partitioning via table inheritance. Each partition must be created as a child table of a single parent table. The parent table itself is normally empty; it exists just to represent the entire data set. You should be familiar with inheritance (see Section 5.8) before attempting to set up partitioning. The following forms of partitioning can be implemented in PostgreSQL: Range Partitioning The table is partitioned into “ranges” defined by a key column or set of columns, with no overlap between the ranges of values assigned to different partitions. For example one might partition by date ranges, or by ranges of identifiers for particular business objects. List Partitioning The table is partitioned by explicitly listing which key values appear in each partition.
5.9.2. Implementing Partitioning To set up a partitioned table, do the following:
1. Create the “master” table, from which all of the partitions will inherit. This table will contain no data. Do not define any check constraints on this table, unless you intend them to be applied equally to all partitions. There is no point in defining any indexes or unique constraints on it, either. 2. Create several “child” tables that each inherit from the master table. Normally, these tables will not add any columns to the set inherited from the master. We will refer to the child tables as partitions, though they are in every way normal PostgreSQL tables. 3. Add table constraints to the partition tables to define the allowed key values in each partition. Typical examples would be: CHECK ( x = 1 ) CHECK ( county IN ( ’Oxfordshire’, ’Buckinghamshire’, ’Warwickshire’ )) CHECK ( outletID >= 100 AND outletID < 200 )
Ensure that the constraints guarantee that there is no overlap between the key values permitted in different partitions. A common mistake is to set up range constraints like: CHECK ( outletID BETWEEN 100 AND 200 )
71
Chapter 5. Data Definition CHECK ( outletID BETWEEN 200 AND 300 )
This is wrong since it is not clear which partition the key value 200 belongs in. Note that there is no difference in syntax between range and list partitioning; those terms are descriptive only. 4. For each partition, create an index on the key column(s), as well as any other indexes you might want. (The key index is not strictly necessary, but in most scenarios it is helpful. If you intend the key values to be unique then you should always create a unique or primary-key constraint for each partition.) 5. Optionally, define a trigger or rule to redirect data inserted into the master table to the appropriate partition. 6. Ensure that the constraint_exclusion configuration parameter is not disabled in postgresql.conf. If it is, queries will not be optimized as desired.
For example, suppose we are constructing a database for a large ice cream company. The company measures peak temperatures every day as well as ice cream sales in each region. Conceptually, we want a table like: CREATE TABLE measurement ( city_id int not null, logdate date not null, peaktemp int, unitsales int );
We know that most queries will access just the last week’s, month’s or quarter’s data, since the main use of this table will be to prepare online reports for management. To reduce the amount of old data that needs to be stored, we decide to only keep the most recent 3 years worth of data. At the beginning of each month we will remove the oldest month’s data. In this situation we can use partitioning to help us meet all of our different requirements for the measurements table. Following the steps outlined above, partitioning can be set up as follows: 1. The master table is the measurement table, declared exactly as above. 2. Next we create one partition for each active month: CREATE CREATE ... CREATE CREATE CREATE
Each of the partitions are complete tables in their own right, but they inherit their definitions from the measurement table. This solves one of our problems: deleting old data. Each month, all we will need to do is perform a DROP TABLE on the oldest child table and create a new child table for the new month’s data. 3. We must provide non-overlapping table constraints. Rather than just creating the partition tables as above, the table creation script should really be: CREATE TABLE measurement_y2006m02 ( CHECK ( logdate >= DATE ’2006-02-01’ AND logdate < DATE ’2006-03-01’ ) ) INHERITS (measurement); CREATE TABLE measurement_y2006m03 ( CHECK ( logdate >= DATE ’2006-03-01’ AND logdate < DATE ’2006-04-01’ ) ) INHERITS (measurement);
72
Chapter 5. Data Definition ... CREATE TABLE measurement_y2007m11 ( CHECK ( logdate >= DATE ’2007-11-01’ AND logdate < DATE ’2007-12-01’ ) ) INHERITS (measurement); CREATE TABLE measurement_y2007m12 ( CHECK ( logdate >= DATE ’2007-12-01’ AND logdate < DATE ’2008-01-01’ ) ) INHERITS (measurement); CREATE TABLE measurement_y2008m01 ( CHECK ( logdate >= DATE ’2008-01-01’ AND logdate < DATE ’2008-02-01’ ) ) INHERITS (measurement);
4. We probably need indexes on the key columns too: CREATE CREATE ... CREATE CREATE CREATE
INDEX measurement_y2006m02_logdate ON measurement_y2006m02 (logdate); INDEX measurement_y2006m03_logdate ON measurement_y2006m03 (logdate); INDEX measurement_y2007m11_logdate ON measurement_y2007m11 (logdate); INDEX measurement_y2007m12_logdate ON measurement_y2007m12 (logdate); INDEX measurement_y2008m01_logdate ON measurement_y2008m01 (logdate);
We choose not to add further indexes at this time. 5. We want our application to be able to say INSERT INTO measurement ... and have the data be redirected into the appropriate partition table. We can arrange that by attaching a suitable trigger function to the master table. If data will be added only to the latest partition, we can use a very simple trigger function: CREATE OR REPLACE FUNCTION measurement_insert_trigger() RETURNS TRIGGER AS $$ BEGIN INSERT INTO measurement_y2008m01 VALUES (NEW.*); RETURN NULL; END; $$ LANGUAGE plpgsql;
After creating the function, we create a trigger which calls the trigger function: CREATE TRIGGER insert_measurement_trigger BEFORE INSERT ON measurement FOR EACH ROW EXECUTE PROCEDURE measurement_insert_trigger();
We must redefine the trigger function each month so that it always points to the current partition. The trigger definition does not need to be updated, however. We might want to insert data and have the server automatically locate the partition into which the row should be added. We could do this with a more complex trigger function, for example:
CREATE OR REPLACE FUNCTION measurement_insert_trigger() RETURNS TRIGGER AS $$ BEGIN IF ( NEW.logdate >= DATE ’2006-02-01’ AND NEW.logdate < DATE ’2006-03-01’ ) THEN INSERT INTO measurement_y2006m02 VALUES (NEW.*); ELSIF ( NEW.logdate >= DATE ’2006-03-01’ AND NEW.logdate < DATE ’2006-04-01’ ) THEN INSERT INTO measurement_y2006m03 VALUES (NEW.*); ... ELSIF ( NEW.logdate >= DATE ’2008-01-01’ AND NEW.logdate < DATE ’2008-02-01’ ) THEN INSERT INTO measurement_y2008m01 VALUES (NEW.*); ELSE RAISE EXCEPTION ’Date out of range. Fix the measurement_insert_trigger() fun END IF;
73
Chapter 5. Data Definition RETURN NULL; END; $$ LANGUAGE plpgsql;
The trigger definition is the same as before. Note that each IF test must exactly match the CHECK constraint for its partition. While this function is more complex than the single-month case, it doesn’t need to be updated as often, since branches can be added in advance of being needed. Note: In practice it might be best to check the newest partition first, if most inserts go into that partition. For simplicity we have shown the trigger’s tests in the same order as in other parts of this example.
As we can see, a complex partitioning scheme could require a substantial amount of DDL. In the above example we would be creating a new partition each month, so it might be wise to write a script that generates the required DDL automatically.
5.9.3. Managing Partitions Normally the set of partitions established when initially defining the table are not intended to remain static. It is common to want to remove old partitions of data and periodically add new partitions for new data. One of the most important advantages of partitioning is precisely that it allows this otherwise painful task to be executed nearly instantaneously by manipulating the partition structure, rather than physically moving large amounts of data around. The simplest option for removing old data is simply to drop the partition that is no longer necessary: DROP TABLE measurement_y2006m02;
This can very quickly delete millions of records because it doesn’t have to individually delete every record. Another option that is often preferable is to remove the partition from the partitioned table but retain access to it as a table in its own right: ALTER TABLE measurement_y2006m02 NO INHERIT measurement;
This allows further operations to be performed on the data before it is dropped. For example, this is often a useful time to back up the data using COPY, pg_dump, or similar tools. It might also be a useful time to aggregate data into smaller formats, perform other data manipulations, or run reports. Similarly we can add a new partition to handle new data. We can create an empty partition in the partitioned table just as the original partitions were created above: CREATE TABLE measurement_y2008m02 ( CHECK ( logdate >= DATE ’2008-02-01’ AND logdate < DATE ’2008-03-01’ ) ) INHERITS (measurement);
74
Chapter 5. Data Definition As an alternative, it is sometimes more convenient to create the new table outside the partition structure, and make it a proper partition later. This allows the data to be loaded, checked, and transformed prior to it appearing in the partitioned table: CREATE TABLE measurement_y2008m02 (LIKE measurement INCLUDING DEFAULTS INCLUDING CONSTRAINTS); ALTER TABLE measurement_y2008m02 ADD CONSTRAINT y2008m02 CHECK ( logdate >= DATE ’2008-02-01’ AND logdate < DATE ’2008-03-01’ ); \copy measurement_y2008m02 from ’measurement_y2008m02’ -- possibly some other data preparation work ALTER TABLE measurement_y2008m02 INHERIT measurement;
5.9.4. Partitioning and Constraint Exclusion Constraint exclusion is a query optimization technique that improves performance for partitioned tables defined in the fashion described above. As an example: SET constraint_exclusion = on; SELECT count(*) FROM measurement WHERE logdate >= DATE ’2008-01-01’;
Without constraint exclusion, the above query would scan each of the partitions of the measurement table. With constraint exclusion enabled, the planner will examine the constraints of each partition and try to prove that the partition need not be scanned because it could not contain any rows meeting the query’s WHERE clause. When the planner can prove this, it excludes the partition from the query plan. You can use the EXPLAIN command to show the difference between a plan with constraint_exclusion on and a plan with it off. A typical unoptimized plan for this type of table setup is: SET constraint_exclusion = off; EXPLAIN SELECT count(*) FROM measurement WHERE logdate >= DATE ’2008-01-01’;
QUERY PLAN ----------------------------------------------------------------------------------------Aggregate (cost=158.66..158.68 rows=1 width=0) -> Append (cost=0.00..151.88 rows=2715 width=0) -> Seq Scan on measurement (cost=0.00..30.38 rows=543 width=0) Filter: (logdate >= ’2008-01-01’::date) -> Seq Scan on measurement_y2006m02 measurement (cost=0.00..30.38 rows=543 wi Filter: (logdate >= ’2008-01-01’::date) -> Seq Scan on measurement_y2006m03 measurement (cost=0.00..30.38 rows=543 wi Filter: (logdate >= ’2008-01-01’::date) ... -> Seq Scan on measurement_y2007m12 measurement (cost=0.00..30.38 rows=543 wi Filter: (logdate >= ’2008-01-01’::date) -> Seq Scan on measurement_y2008m01 measurement (cost=0.00..30.38 rows=543 wi Filter: (logdate >= ’2008-01-01’::date)
Some or all of the partitions might use index scans instead of full-table sequential scans, but the point here is that there is no need to scan the older partitions at all to answer this query. When we enable constraint exclusion, we get a significantly cheaper plan that will deliver the same answer: SET constraint_exclusion = on;
75
Chapter 5. Data Definition
EXPLAIN SELECT count(*) FROM measurement WHERE logdate >= DATE ’2008-01-01’; QUERY PLAN ----------------------------------------------------------------------------------------Aggregate (cost=63.47..63.48 rows=1 width=0) -> Append (cost=0.00..60.75 rows=1086 width=0) -> Seq Scan on measurement (cost=0.00..30.38 rows=543 width=0) Filter: (logdate >= ’2008-01-01’::date) -> Seq Scan on measurement_y2008m01 measurement (cost=0.00..30.38 rows=543 wi Filter: (logdate >= ’2008-01-01’::date)
Note that constraint exclusion is driven only by CHECK constraints, not by the presence of indexes. Therefore it isn’t necessary to define indexes on the key columns. Whether an index needs to be created for a given partition depends on whether you expect that queries that scan the partition will generally scan a large part of the partition or just a small part. An index will be helpful in the latter case but not the former. The default (and recommended) setting of constraint_exclusion is actually neither on nor off, but an intermediate setting called partition, which causes the technique to be applied only to queries that are likely to be working on partitioned tables. The on setting causes the planner to examine CHECK constraints in all queries, even simple ones that are unlikely to benefit.
5.9.5. Alternative Partitioning Methods A different approach to redirecting inserts into the appropriate partition table is to set up rules, instead of a trigger, on the master table. For example: CREATE RULE measurement_insert_y2006m02 AS ON INSERT TO measurement WHERE ( logdate >= DATE ’2006-02-01’ AND logdate < DATE ’2006-03-01’ ) DO INSTEAD INSERT INTO measurement_y2006m02 VALUES (NEW.*); ... CREATE RULE measurement_insert_y2008m01 AS ON INSERT TO measurement WHERE ( logdate >= DATE ’2008-01-01’ AND logdate < DATE ’2008-02-01’ ) DO INSTEAD INSERT INTO measurement_y2008m01 VALUES (NEW.*);
A rule has significantly more overhead than a trigger, but the overhead is paid once per query rather than once per row, so this method might be advantageous for bulk-insert situations. In most cases, however, the trigger method will offer better performance. Be aware that COPY ignores rules. If you want to use COPY to insert data, you’ll need to copy into the correct partition table rather than into the master. COPY does fire triggers, so you can use it normally if you use the trigger approach. Another disadvantage of the rule approach is that there is no simple way to force an error if the set of rules doesn’t cover the insertion date; the data will silently go into the master table instead. Partitioning can also be arranged using a UNION ALL view, instead of table inheritance. For example, CREATE VIEW measurement AS SELECT * FROM measurement_y2006m02 UNION ALL SELECT * FROM measurement_y2006m03 ...
76
Chapter 5. Data Definition UNION ALL SELECT * FROM measurement_y2007m11 UNION ALL SELECT * FROM measurement_y2007m12 UNION ALL SELECT * FROM measurement_y2008m01;
However, the need to recreate the view adds an extra step to adding and dropping individual partitions of the data set. In practice this method has little to recommend it compared to using inheritance.
5.9.6. Caveats The following caveats apply to partitioned tables:
•
There is no automatic way to verify that all of the CHECK constraints are mutually exclusive. It is safer to create code that generates partitions and creates and/or modifies associated objects than to write each by hand.
•
The schemes shown here assume that the partition key column(s) of a row never change, or at least do not change enough to require it to move to another partition. An UPDATE that attempts to do that will fail because of the CHECK constraints. If you need to handle such cases, you can put suitable update triggers on the partition tables, but it makes management of the structure much more complicated.
•
If you are using manual VACUUM or ANALYZE commands, don’t forget that you need to run them on each partition individually. A command like: ANALYZE measurement;
will only process the master table.
The following caveats apply to constraint exclusion: •
Constraint exclusion only works when the query’s WHERE clause contains constants (or externally supplied parameters). For example, a comparison against a non-immutable function such as CURRENT_TIMESTAMP cannot be optimized, since the planner cannot know which partition the function value might fall into at run time.
•
Keep the partitioning constraints simple, else the planner may not be able to prove that partitions don’t need to be visited. Use simple equality conditions for list partitioning, or simple range tests for range partitioning, as illustrated in the preceding examples. A good rule of thumb is that partitioning constraints should contain only comparisons of the partitioning column(s) to constants using Btree-indexable operators.
•
All constraints on all partitions of the master table are examined during constraint exclusion, so large numbers of partitions are likely to increase query planning time considerably. Partitioning using these techniques will work well with up to perhaps a hundred partitions; don’t try to use many thousands of partitions.
5.10. Foreign Data PostgreSQL implements portions of the SQL/MED specification, allowing you to access data that resides outside PostgreSQL using regular SQL queries. Such data is referred to as foreign data. (Note
77
Chapter 5. Data Definition that this usage is not to be confused with foreign keys, which are a type of constraint within the database.) Foreign data is accessed with help from a foreign data wrapper. A foreign data wrapper is a library that can communicate with an external data source, hiding the details of connecting to the data source and obtaining data from it. There are some foreign data wrappers available as contrib modules; see Appendix F. Other kinds of foreign data wrappers might be found as third party products. If none of the existing foreign data wrappers suit your needs, you can write your own; see Chapter 52. To access foreign data, you need to create a foreign server object, which defines how to connect to a particular external data source according to the set of options used by its supporting foreign data wrapper. Then you need to create one or more foreign tables, which define the structure of the remote data. A foreign table can be used in queries just like a normal table, but a foreign table has no storage in the PostgreSQL server. Whenever it is used, PostgreSQL asks the foreign data wrapper to fetch data from the external source, or transmit data to the external source in the case of update commands. Accessing remote data may require authenticating to the external data source. This information can be provided by a user mapping, which can provide additional data such as user names and passwords based on the current PostgreSQL role. For additional information, see CREATE FOREIGN DATA WRAPPER, CREATE SERVER, CREATE USER MAPPING, and CREATE FOREIGN TABLE.
5.11. Other Database Objects Tables are the central objects in a relational database structure, because they hold your data. But they are not the only objects that exist in a database. Many other kinds of objects can be created to make the use and management of the data more efficient or convenient. They are not discussed in this chapter, but we give you a list here so that you are aware of what is possible: •
Views
•
Functions and operators
•
Data types and domains
•
Triggers and rewrite rules
Detailed information on these topics appears in Part V.
5.12. Dependency Tracking When you create complex database structures involving many tables with foreign key constraints, views, triggers, functions, etc. you implicitly create a net of dependencies between the objects. For instance, a table with a foreign key constraint depends on the table it references. To ensure the integrity of the entire database structure, PostgreSQL makes sure that you cannot drop objects that other objects still depend on. For example, attempting to drop the products table we considered in Section 5.3.5, with the orders table depending on it, would result in an error message like this: DROP TABLE products; ERROR:
cannot drop table products because other objects depend on it
78
Chapter 5. Data Definition DETAIL: constraint orders_product_no_fkey on table orders depends on table products HINT: Use DROP ... CASCADE to drop the dependent objects too.
The error message contains a useful hint: if you do not want to bother deleting all the dependent objects individually, you can run: DROP TABLE products CASCADE;
and all the dependent objects will be removed. In this case, it doesn’t remove the orders table, it only removes the foreign key constraint. (If you want to check what DROP ... CASCADE will do, run DROP without CASCADE and read the DETAIL output.) All DROP commands in PostgreSQL support specifying CASCADE. Of course, the nature of the possible dependencies varies with the type of the object. You can also write RESTRICT instead of CASCADE to get the default behavior, which is to prevent the dropping of objects that other objects depend on. Note: According to the SQL standard, specifying either RESTRICT or CASCADE is required in a DROP command. No database system actually enforces that rule, but whether the default behavior is RESTRICT or CASCADE varies across systems.
For user-defined functions, PostgreSQL tracks dependencies associated with a function’s externallyvisible properties, such as its argument and result types, but not dependencies that could only be known by examining the function body. As an example, consider this situation: CREATE TYPE rainbow AS ENUM (’red’, ’orange’, ’yellow’, ’green’, ’blue’, ’purple’); CREATE TABLE my_colors (color rainbow, note text); CREATE FUNCTION get_color_note (rainbow) RETURNS text AS ’SELECT note FROM my_colors WHERE color = $1’ LANGUAGE SQL;
(See Section 35.4 for an explanation of SQL-language functions.) PostgreSQL will be aware that the get_color_note function depends on the rainbow type: dropping the type would force dropping the function, because its argument type would no longer be defined. But PostgreSQL will not consider get_color_note to depend on the my_colors table, and so will not drop the function if the table is dropped. While there are disadvantages to this approach, there are also benefits. The function is still valid in some sense if the table is missing, though executing it would cause an error; creating a new table of the same name would allow the function to work again.
79
Chapter 6. Data Manipulation The previous chapter discussed how to create tables and other structures to hold your data. Now it is time to fill the tables with data. This chapter covers how to insert, update, and delete table data. The chapter after this will finally explain how to extract your long-lost data from the database.
6.1. Inserting Data When a table is created, it contains no data. The first thing to do before a database can be of much use is to insert data. Data is conceptually inserted one row at a time. Of course you can also insert more than one row, but there is no way to insert less than one row. Even if you know only some column values, a complete row must be created. To create a new row, use the INSERT command. The command requires the table name and column values. For example, consider the products table from Chapter 5: CREATE TABLE products ( product_no integer, name text, price numeric );
An example command to insert a row would be: INSERT INTO products VALUES (1, ’Cheese’, 9.99);
The data values are listed in the order in which the columns appear in the table, separated by commas. Usually, the data values will be literals (constants), but scalar expressions are also allowed. The above syntax has the drawback that you need to know the order of the columns in the table. To avoid this you can also list the columns explicitly. For example, both of the following commands have the same effect as the one above: INSERT INTO products (product_no, name, price) VALUES (1, ’Cheese’, 9.99); INSERT INTO products (name, price, product_no) VALUES (’Cheese’, 9.99, 1);
Many users consider it good practice to always list the column names. If you don’t have values for all the columns, you can omit some of them. In that case, the columns will be filled with their default values. For example: INSERT INTO products (product_no, name) VALUES (1, ’Cheese’); INSERT INTO products VALUES (1, ’Cheese’);
The second form is a PostgreSQL extension. It fills the columns from the left with as many values as are given, and the rest will be defaulted. For clarity, you can also request default values explicitly, for individual columns or for the entire row: INSERT INTO products (product_no, name, price) VALUES (1, ’Cheese’, DEFAULT); INSERT INTO products DEFAULT VALUES;
You can insert multiple rows in a single command: INSERT INTO products (product_no, name, price) VALUES
It is also possible to insert the result of a query (which might be no rows, one row, or many rows): INSERT INTO products (product_no, name, price) SELECT product_no, name, price FROM new_products WHERE release_date = ’today’;
This provides the full power of the SQL query mechanism (Chapter 7) for computing the rows to be inserted. Tip: When inserting a lot of data at the same time, considering using the COPY command. It is not as flexible as the INSERT command, but is more efficient. Refer to Section 14.4 for more information on improving bulk loading performance.
6.2. Updating Data The modification of data that is already in the database is referred to as updating. You can update individual rows, all the rows in a table, or a subset of all rows. Each column can be updated separately; the other columns are not affected. To update existing rows, use the UPDATE command. This requires three pieces of information: 1. The name of the table and column to update 2. The new value of the column 3. Which row(s) to update
Recall from Chapter 5 that SQL does not, in general, provide a unique identifier for rows. Therefore it is not always possible to directly specify which row to update. Instead, you specify which conditions a row must meet in order to be updated. Only if you have a primary key in the table (independent of whether you declared it or not) can you reliably address individual rows by choosing a condition that matches the primary key. Graphical database access tools rely on this fact to allow you to update rows individually. For example, this command updates all products that have a price of 5 to have a price of 10: UPDATE products SET price = 10 WHERE price = 5;
This might cause zero, one, or many rows to be updated. It is not an error to attempt an update that does not match any rows. Let’s look at that command in detail. First is the key word UPDATE followed by the table name. As usual, the table name can be schema-qualified, otherwise it is looked up in the path. Next is the key word SET followed by the column name, an equal sign, and the new column value. The new column value can be any scalar expression, not just a constant. For example, if you want to raise the price of all products by 10% you could use: UPDATE products SET price = price * 1.10;
81
Chapter 6. Data Manipulation As you see, the expression for the new value can refer to the existing value(s) in the row. We also left out the WHERE clause. If it is omitted, it means that all rows in the table are updated. If it is present, only those rows that match the WHERE condition are updated. Note that the equals sign in the SET clause is an assignment while the one in the WHERE clause is a comparison, but this does not create any ambiguity. Of course, the WHERE condition does not have to be an equality test. Many other operators are available (see Chapter 9). But the expression needs to evaluate to a Boolean result. You can update more than one column in an UPDATE command by listing more than one assignment in the SET clause. For example: UPDATE mytable SET a = 5, b = 3, c = 1 WHERE a > 0;
6.3. Deleting Data So far we have explained how to add data to tables and how to change data. What remains is to discuss how to remove data that is no longer needed. Just as adding data is only possible in whole rows, you can only remove entire rows from a table. In the previous section we explained that SQL does not provide a way to directly address individual rows. Therefore, removing rows can only be done by specifying conditions that the rows to be removed have to match. If you have a primary key in the table then you can specify the exact row. But you can also remove groups of rows matching a condition, or you can remove all rows in the table at once. You use the DELETE command to remove rows; the syntax is very similar to the UPDATE command. For instance, to remove all rows from the products table that have a price of 10, use: DELETE FROM products WHERE price = 10;
If you simply write: DELETE FROM products;
then all rows in the table will be deleted! Caveat programmer.
6.4. Returning Data From Modified Rows Sometimes it is useful to obtain data from modified rows while they are being manipulated. The INSERT, UPDATE, and DELETE commands all have an optional RETURNING clause that supports this. Use of RETURNING avoids performing an extra database query to collect the data, and is especially valuable when it would otherwise be difficult to identify the modified rows reliably. The allowed contents of a RETURNING clause are the same as a SELECT command’s output list (see Section 7.3). It can contain column names of the command’s target table, or value expressions using those columns. A common shorthand is RETURNING *, which selects all columns of the target table in order. In an INSERT, the data available to RETURNING is the row as it was inserted. This is not so useful in trivial inserts, since it would just repeat the data provided by the client. But it can be very handy when relying on computed default values. For example, when using a serial column to provide unique identifiers, RETURNING can return the ID assigned to a new row:
82
Chapter 6. Data Manipulation CREATE TABLE users (firstname text, lastname text, id serial primary key); INSERT INTO users (firstname, lastname) VALUES (’Joe’, ’Cool’) RETURNING id;
The RETURNING clause is also very useful with INSERT ... SELECT. In an UPDATE, the data available to RETURNING is the new content of the modified row. For example: UPDATE products SET price = price * 1.10 WHERE price SELECT * FROM t1 CROSS JOIN t2;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 1 | a | 3 | yyy 1 | a | 5 | zzz 2 | b | 1 | xxx 2 | b | 3 | yyy 2 | b | 5 | zzz 3 | c | 1 | xxx 3 | c | 3 | yyy 3 | c | 5 | zzz (9 rows) => SELECT * FROM t1 INNER JOIN t2 ON t1.num = t2.num;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 3 | c | 3 | yyy (2 rows) => SELECT * FROM t1 INNER JOIN t2 USING (num);
num | name | value -----+------+------1 | a | xxx 3 | c | yyy (2 rows)
num | name | value -----+------+------1 | a | xxx 3 | c | yyy (2 rows) => SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 2 | b | | 3 | c | 3 | yyy (3 rows) => SELECT * FROM t1 LEFT JOIN t2 USING (num);
num | name | value -----+------+------1 | a | xxx 2 | b | 3 | c | yyy (3 rows) => SELECT * FROM t1 RIGHT JOIN t2 ON t1.num = t2.num;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 3 | c | 3 | yyy | | 5 | zzz (3 rows) => SELECT * FROM t1 FULL JOIN t2 ON t1.num = t2.num;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 2 | b | | 3 | c | 3 | yyy | | 5 | zzz (4 rows)
The join condition specified with ON can also contain conditions that do not relate directly to the join. This can prove useful for some queries but needs to be thought out carefully. For example: => SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num AND t2.value = ’xxx’;
num | name | num | value -----+------+-----+------1 | a | 1 | xxx 2 | b | | 3 | c | | (3 rows)
Notice that placing the restriction in the WHERE clause produces a different result: => SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num WHERE t2.value = ’xxx’;
num | name | num | value -----+------+-----+-------
88
Chapter 7. Queries 1 | a (1 row)
|
1 | xxx
This is because a restriction placed in the ON clause is processed before the join, while a restriction placed in the WHERE clause is processed after the join. That does not matter with inner joins, but it matters a lot with outer joins.
7.2.1.2. Table and Column Aliases A temporary name can be given to tables and complex table references to be used for references to the derived table in the rest of the query. This is called a table alias. To create a table alias, write FROM table_reference AS alias
or FROM table_reference alias
The AS key word is optional noise. alias can be any identifier. A typical application of table aliases is to assign short identifiers to long table names to keep the join clauses readable. For example:
SELECT * FROM some_very_long_table_name s JOIN another_fairly_long_name a ON s.id = a.num
The alias becomes the new name of the table reference so far as the current query is concerned — it is not allowed to refer to the table by the original name elsewhere in the query. Thus, this is not valid: SELECT * FROM my_table AS m WHERE my_table.a > 5;
-- wrong
Table aliases are mainly for notational convenience, but it is necessary to use them when joining a table to itself, e.g.: SELECT * FROM people AS mother JOIN people AS child ON mother.id = child.mother_id;
Additionally, an alias is required if the table reference is a subquery (see Section 7.2.1.3). Parentheses are used to resolve ambiguities. In the following example, the first statement assigns the alias b to the second instance of my_table, but the second statement assigns the alias to the result of the join: SELECT * FROM my_table AS a CROSS JOIN my_table AS b ... SELECT * FROM (my_table AS a CROSS JOIN my_table) AS b ...
Another form of table aliasing gives temporary names to the columns of the table, as well as the table itself: FROM table_reference [AS] alias ( column1 [, column2 [, ...]] )
If fewer column aliases are specified than the actual table has columns, the remaining columns are not renamed. This syntax is especially useful for self-joins or subqueries.
89
Chapter 7. Queries When an alias is applied to the output of a JOIN clause, the alias hides the original name(s) within the JOIN. For example: SELECT a.* FROM my_table AS a JOIN your_table AS b ON ...
is valid SQL, but: SELECT a.* FROM (my_table AS a JOIN your_table AS b ON ...) AS c
is not valid; the table alias a is not visible outside the alias c.
7.2.1.3. Subqueries Subqueries specifying a derived table must be enclosed in parentheses and must be assigned a table alias name (as in Section 7.2.1.2). For example: FROM (SELECT * FROM table1) AS alias_name
This example is equivalent to FROM table1 AS alias_name. More interesting cases, which cannot be reduced to a plain join, arise when the subquery involves grouping or aggregation. A subquery can also be a VALUES list: FROM (VALUES (’anne’, ’smith’), (’bob’, ’jones’), (’joe’, ’blow’)) AS names(first, last)
Again, a table alias is required. Assigning alias names to the columns of the VALUES list is optional, but is good practice. For more information see Section 7.7.
7.2.1.4. Table Functions Table functions are functions that produce a set of rows, made up of either base data types (scalar types) or composite data types (table rows). They are used like a table, view, or subquery in the FROM clause of a query. Columns returned by table functions can be included in SELECT, JOIN, or WHERE clauses in the same manner as a table, view, or subquery column. If a table function returns a base data type, the single result column name matches the function name. If the function returns a composite type, the result columns get the same names as the individual attributes of the type. A table function can be aliased in the FROM clause, but it also can be left unaliased. If a function is used in the FROM clause with no alias, the function name is used as the resulting table name. Some examples: CREATE TABLE foo (fooid int, foosubid int, fooname text); CREATE FUNCTION getfoo(int) RETURNS SETOF foo AS $$ SELECT * FROM foo WHERE fooid = $1; $$ LANGUAGE SQL; SELECT * FROM getfoo(1) AS t1; SELECT * FROM foo WHERE foosubid IN (
90
Chapter 7. Queries SELECT foosubid FROM getfoo(foo.fooid) z WHERE z.fooid = foo.fooid ); CREATE VIEW vw_getfoo AS SELECT * FROM getfoo(1); SELECT * FROM vw_getfoo;
In some cases it is useful to define table functions that can return different column sets depending on how they are invoked. To support this, the table function can be declared as returning the pseudotype record. When such a function is used in a query, the expected row structure must be specified in the query itself, so that the system can know how to parse and plan the query. Consider this example: SELECT * FROM dblink(’dbname=mydb’, ’SELECT proname, prosrc FROM pg_proc’) AS t1(proname name, prosrc text) WHERE proname LIKE ’bytea%’;
The dblink function (part of the dblink module) executes a remote query. It is declared to return record since it might be used for any kind of query. The actual column set must be specified in the calling query so that the parser knows, for example, what * should expand to.
7.2.1.5. LATERAL Subqueries Subqueries appearing in FROM can be preceded by the key word LATERAL. This allows them to reference columns provided by preceding FROM items. (Without LATERAL, each subquery is evaluated independently and so cannot cross-reference any other FROM item.) Table functions appearing in FROM can also be preceded by the key word LATERAL, but for functions the key word is optional; the function’s arguments can contain references to columns provided by preceding FROM items in any case. A LATERAL item can appear at top level in the FROM list, or within a JOIN tree. In the latter case it can also refer to any items that are on the left-hand side of a JOIN that it is on the right-hand side of. When a FROM item contains LATERAL cross-references, evaluation proceeds as follows: for each row of the FROM item providing the cross-referenced column(s), or set of rows of multiple FROM items providing the columns, the LATERAL item is evaluated using that row or row set’s values of the columns. The resulting row(s) are joined as usual with the rows they were computed from. This is repeated for each row or set of rows from the column source table(s). A trivial example of LATERAL is SELECT * FROM foo, LATERAL (SELECT * FROM bar WHERE bar.id = foo.bar_id) ss;
This is not especially useful since it has exactly the same result as the more conventional SELECT * FROM foo, bar WHERE bar.id = foo.bar_id; LATERAL is primarily useful when the cross-referenced column is necessary for computing the row(s)
to be joined. A common application is providing an argument value for a set-returning function. For example, supposing that vertices(polygon) returns the set of vertices of a polygon, we could identify close-together vertices of polygons stored in a table with:
91
Chapter 7. Queries SELECT p1.id, p2.id, v1, v2 FROM polygons p1, polygons p2, LATERAL vertices(p1.poly) v1, LATERAL vertices(p2.poly) v2 WHERE (v1 v2) < 10 AND p1.id != p2.id;
This query could also be written SELECT p1.id, p2.id, v1, v2 FROM polygons p1 CROSS JOIN LATERAL vertices(p1.poly) v1, polygons p2 CROSS JOIN LATERAL vertices(p2.poly) v2 WHERE (v1 v2) < 10 AND p1.id != p2.id;
or in several other equivalent formulations. (As already mentioned, the LATERAL key word is unnecessary in this example, but we use it for clarity.) It is often particularly handy to LEFT JOIN to a LATERAL subquery, so that source rows will appear in the result even if the LATERAL subquery produces no rows for them. For example, if get_product_names() returns the names of products made by a manufacturer, but some manufacturers in our table currently produce no products, we could find out which ones those are like this: SELECT m.name FROM manufacturers m LEFT JOIN LATERAL get_product_names(m.id) pname ON true WHERE pname IS NULL;
7.2.2. The WHERE Clause The syntax of the WHERE Clause is WHERE search_condition
where search_condition is any value expression (see Section 4.2) that returns a value of type boolean. After the processing of the FROM clause is done, each row of the derived virtual table is checked against the search condition. If the result of the condition is true, the row is kept in the output table, otherwise (i.e., if the result is false or null) it is discarded. The search condition typically references at least one column of the table generated in the FROM clause; this is not required, but otherwise the WHERE clause will be fairly useless. Note: The join condition of an inner join can be written either in the WHERE clause or in the JOIN clause. For example, these table expressions are equivalent: FROM a, b WHERE a.id = b.id AND b.val > 5
and: FROM a INNER JOIN b ON (a.id = b.id) WHERE b.val > 5
or perhaps even: FROM a NATURAL JOIN b WHERE b.val > 5
92
Chapter 7. Queries Which one of these you use is mainly a matter of style. The JOIN syntax in the FROM clause is probably not as portable to other SQL database management systems, even though it is in the SQL standard. For outer joins there is no choice: they must be done in the FROM clause. The ON or USING clause of an outer join is not equivalent to a WHERE condition, because it results in the addition of rows (for unmatched input rows) as well as the removal of rows in the final result.
Here are some examples of WHERE clauses: SELECT ... FROM fdt WHERE c1 > 5 SELECT ... FROM fdt WHERE c1 IN (1, 2, 3) SELECT ... FROM fdt WHERE c1 IN (SELECT c1 FROM t2) SELECT ... FROM fdt WHERE c1 IN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10) SELECT ... FROM fdt WHERE c1 BETWEEN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10) AND 100 SELECT ... FROM fdt WHERE EXISTS (SELECT c1 FROM t2 WHERE c2 > fdt.c1) fdt is the table derived in the FROM clause. Rows that do not meet the search condition of the WHERE clause are eliminated from fdt. Notice the use of scalar subqueries as value expressions. Just like any other query, the subqueries can employ complex table expressions. Notice also how fdt is referenced in the subqueries. Qualifying c1 as fdt.c1 is only necessary if c1 is also the name of a column in the derived input table of the subquery. But qualifying the column name adds clarity even when it is not needed. This example shows how the column naming scope of an outer query extends into its inner queries.
7.2.3. The GROUP BY and HAVING Clauses After passing the WHERE filter, the derived input table might be subject to grouping, using the GROUP BY clause, and elimination of group rows using the HAVING clause. SELECT select_list FROM ... [WHERE ...] GROUP BY grouping_column_reference [, grouping_column_reference]...
The GROUP BY Clause is used to group together those rows in a table that have the same values in all the columns listed. The order in which the columns are listed does not matter. The effect is to combine each set of rows having common values into one group row that represents all rows in the group. This is done to eliminate redundancy in the output and/or compute aggregates that apply to these groups. For instance: => SELECT * FROM test1;
x | y ---+--a | 3 c | 2 b | 5 a | 1 (4 rows)
93
Chapter 7. Queries => SELECT x FROM test1 GROUP BY x;
x --a b c (3 rows)
In the second query, we could not have written SELECT * FROM test1 GROUP BY x, because there is no single value for the column y that could be associated with each group. The groupedby columns can be referenced in the select list since they have a single value in each group. In general, if a table is grouped, columns that are not listed in GROUP BY cannot be referenced except in aggregate expressions. An example with aggregate expressions is: => SELECT x, sum(y) FROM test1 GROUP BY x;
x | sum ---+----a | 4 b | 5 c | 2 (3 rows)
Here sum is an aggregate function that computes a single value over the entire group. More information about the available aggregate functions can be found in Section 9.20. Tip: Grouping without aggregate expressions effectively calculates the set of distinct values in a column. This can also be achieved using the DISTINCT clause (see Section 7.3.3).
Here is another example: it calculates the total sales for each product (rather than the total sales of all products): SELECT product_id, p.name, (sum(s.units) * p.price) AS sales FROM products p LEFT JOIN sales s USING (product_id) GROUP BY product_id, p.name, p.price;
In this example, the columns product_id, p.name, and p.price must be in the GROUP BY clause since they are referenced in the query select list (but see below). The column s.units does not have to be in the GROUP BY list since it is only used in an aggregate expression (sum(...)), which represents the sales of a product. For each product, the query returns a summary row about all sales of the product. If the products table is set up so that, say, product_id is the primary key, then it would be enough to group by product_id in the above example, since name and price would be functionally dependent on the product ID, and so there would be no ambiguity about which name and price value to return for each product ID group. In strict SQL, GROUP BY can only group by columns of the source table but PostgreSQL extends this to also allow GROUP BY to group by columns in the select list. Grouping by value expressions instead of simple column names is also allowed. If a table has been grouped using GROUP BY, but only certain groups are of interest, the HAVING clause can be used, much like a WHERE clause, to eliminate groups from the result. The syntax is: SELECT select_list FROM ... [WHERE ...] GROUP BY ... HAVING boolean_expression
94
Chapter 7. Queries Expressions in the HAVING clause can refer both to grouped expressions and to ungrouped expressions (which necessarily involve an aggregate function). Example: => SELECT x, sum(y) FROM test1 GROUP BY x HAVING sum(y) > 3;
x | sum ---+----a | 4 b | 5 (2 rows) => SELECT x, sum(y) FROM test1 GROUP BY x HAVING x < ’c’;
x | sum ---+----a | 4 b | 5 (2 rows)
Again, a more realistic example: SELECT product_id, p.name, (sum(s.units) * (p.price - p.cost)) AS profit FROM products p LEFT JOIN sales s USING (product_id) WHERE s.date > CURRENT_DATE - INTERVAL ’4 weeks’ GROUP BY product_id, p.name, p.price, p.cost HAVING sum(p.price * s.units) > 5000;
In the example above, the WHERE clause is selecting rows by a column that is not grouped (the expression is only true for sales during the last four weeks), while the HAVING clause restricts the output to groups with total gross sales over 5000. Note that the aggregate expressions do not necessarily need to be the same in all parts of the query. If a query contains aggregate function calls, but no GROUP BY clause, grouping still occurs: the result is a single group row (or perhaps no rows at all, if the single row is then eliminated by HAVING). The same is true if it contains a HAVING clause, even without any aggregate function calls or GROUP BY clause.
7.2.4. Window Function Processing If the query contains any window functions (see Section 3.5, Section 9.21 and Section 4.2.8), these functions are evaluated after any grouping, aggregation, and HAVING filtering is performed. That is, if the query uses any aggregates, GROUP BY, or HAVING, then the rows seen by the window functions are the group rows instead of the original table rows from FROM/WHERE. When multiple window functions are used, all the window functions having syntactically equivalent PARTITION BY and ORDER BY clauses in their window definitions are guaranteed to be evaluated in a single pass over the data. Therefore they will see the same sort ordering, even if the ORDER BY does not uniquely determine an ordering. However, no guarantees are made about the evaluation of functions having different PARTITION BY or ORDER BY specifications. (In such cases a sort step is typically required between the passes of window function evaluations, and the sort is not guaranteed to preserve ordering of rows that its ORDER BY sees as equivalent.) Currently, window functions always require presorted data, and so the query output will be ordered according to one or another of the window functions’ PARTITION BY/ORDER BY clauses. It is not
95
Chapter 7. Queries recommended to rely on this, however. Use an explicit top-level ORDER BY clause if you want to be sure the results are sorted in a particular way.
7.3. Select Lists As shown in the previous section, the table expression in the SELECT command constructs an intermediate virtual table by possibly combining tables, views, eliminating rows, grouping, etc. This table is finally passed on to processing by the select list. The select list determines which columns of the intermediate table are actually output.
7.3.1. Select-List Items The simplest kind of select list is * which emits all columns that the table expression produces. Otherwise, a select list is a comma-separated list of value expressions (as defined in Section 4.2). For instance, it could be a list of column names: SELECT a, b, c FROM ...
The columns names a, b, and c are either the actual names of the columns of tables referenced in the FROM clause, or the aliases given to them as explained in Section 7.2.1.2. The name space available in the select list is the same as in the WHERE clause, unless grouping is used, in which case it is the same as in the HAVING clause. If more than one table has a column of the same name, the table name must also be given, as in: SELECT tbl1.a, tbl2.a, tbl1.b FROM ...
When working with multiple tables, it can also be useful to ask for all the columns of a particular table: SELECT tbl1.*, tbl2.a FROM ...
See Section 8.16.5 for more about the table_name.* notation. If an arbitrary value expression is used in the select list, it conceptually adds a new virtual column to the returned table. The value expression is evaluated once for each result row, with the row’s values substituted for any column references. But the expressions in the select list do not have to reference any columns in the table expression of the FROM clause; they can be constant arithmetic expressions, for instance.
7.3.2. Column Labels The entries in the select list can be assigned names for subsequent processing, such as for use in an ORDER BY clause or for display by the client application. For example: SELECT a AS value, b + c AS sum FROM ...
If no output column name is specified using AS, the system assigns a default column name. For simple column references, this is the name of the referenced column. For function calls, this is the name of the function. For complex expressions, the system will generate a generic name.
96
Chapter 7. Queries The AS keyword is optional, but only if the new column name does not match any PostgreSQL keyword (see Appendix C). To avoid an accidental match to a keyword, you can double-quote the column name. For example, VALUE is a keyword, so this does not work: SELECT a value, b + c AS sum FROM ...
but this does: SELECT a "value", b + c AS sum FROM ...
For protection against possible future keyword additions, it is recommended that you always either write AS or double-quote the output column name. Note: The naming of output columns here is different from that done in the FROM clause (see Section 7.2.1.2). It is possible to rename the same column twice, but the name assigned in the select list is the one that will be passed on.
7.3.3. DISTINCT After the select list has been processed, the result table can optionally be subject to the elimination of duplicate rows. The DISTINCT key word is written directly after SELECT to specify this: SELECT DISTINCT select_list ...
(Instead of DISTINCT the key word ALL can be used to specify the default behavior of retaining all rows.) Obviously, two rows are considered distinct if they differ in at least one column value. Null values are considered equal in this comparison. Alternatively, an arbitrary expression can determine what rows are to be considered distinct: SELECT DISTINCT ON (expression [, expression ...]) select_list ...
Here expression is an arbitrary value expression that is evaluated for all rows. A set of rows for which all the expressions are equal are considered duplicates, and only the first row of the set is kept in the output. Note that the “first row” of a set is unpredictable unless the query is sorted on enough columns to guarantee a unique ordering of the rows arriving at the DISTINCT filter. (DISTINCT ON processing occurs after ORDER BY sorting.) The DISTINCT ON clause is not part of the SQL standard and is sometimes considered bad style because of the potentially indeterminate nature of its results. With judicious use of GROUP BY and subqueries in FROM, this construct can be avoided, but it is often the most convenient alternative.
7.4. Combining Queries The results of two queries can be combined using the set operations union, intersection, and difference. The syntax is query1 UNION [ALL] query2 query1 INTERSECT [ALL] query2
97
Chapter 7. Queries query1 EXCEPT [ALL] query2
query1 and query2 are queries that can use any of the features discussed up to this point. Set
operations can also be nested and chained, for example query1 UNION query2 UNION query3
which is executed as: (query1 UNION query2) UNION query3
UNION effectively appends the result of query2 to the result of query1 (although there is no guaran-
tee that this is the order in which the rows are actually returned). Furthermore, it eliminates duplicate rows from its result, in the same way as DISTINCT, unless UNION ALL is used. INTERSECT returns all rows that are both in the result of query1 and in the result of query2. Duplicate rows are eliminated unless INTERSECT ALL is used. EXCEPT returns all rows that are in the result of query1 but not in the result of query2. (This is sometimes called the difference between two queries.) Again, duplicates are eliminated unless EXCEPT ALL is used.
In order to calculate the union, intersection, or difference of two queries, the two queries must be “union compatible”, which means that they return the same number of columns and the corresponding columns have compatible data types, as described in Section 10.5.
7.5. Sorting Rows After a query has produced an output table (after the select list has been processed) it can optionally be sorted. If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on. A particular output ordering can only be guaranteed if the sort step is explicitly chosen. The ORDER BY clause specifies the sort order: SELECT select_list FROM table_expression ORDER BY sort_expression1 [ASC | DESC] [NULLS { FIRST | LAST }] [, sort_expression2 [ASC | DESC] [NULLS { FIRST | LAST }] ...]
The sort expression(s) can be any expression that would be valid in the query’s select list. An example is: SELECT a, b FROM table1 ORDER BY a + b, c;
When more than one expression is specified, the later values are used to sort rows that are equal according to the earlier values. Each expression can be followed by an optional ASC or DESC keyword to set the sort direction to ascending or descending. ASC order is the default. Ascending order puts smaller values first, where “smaller” is defined in terms of the < operator. Similarly, descending order is determined with the > operator. 1 1. Actually, PostgreSQL uses the default B-tree operator class for the expression’s data type to determine the sort ordering for ASC and DESC. Conventionally, data types will be set up so that the < and > operators correspond to this sort ordering, but a user-defined data type’s designer could choose to do something different.
98
Chapter 7. Queries The NULLS FIRST and NULLS LAST options can be used to determine whether nulls appear before or after non-null values in the sort ordering. By default, null values sort as if larger than any non-null value; that is, NULLS FIRST is the default for DESC order, and NULLS LAST otherwise. Note that the ordering options are considered independently for each sort column. For example ORDER BY x, y DESC means ORDER BY x ASC, y DESC, which is not the same as ORDER BY x DESC, y DESC. A sort_expression can also be the column label or number of an output column, as in: SELECT a + b AS sum, c FROM table1 ORDER BY sum; SELECT a, max(b) FROM table1 GROUP BY a ORDER BY 1;
both of which sort by the first output column. Note that an output column name has to stand alone, that is, it cannot be used in an expression — for example, this is not correct: SELECT a + b AS sum, c FROM table1 ORDER BY sum + c;
-- wrong
This restriction is made to reduce ambiguity. There is still ambiguity if an ORDER BY item is a simple name that could match either an output column name or a column from the table expression. The output column is used in such cases. This would only cause confusion if you use AS to rename an output column to match some other table column’s name. ORDER BY can be applied to the result of a UNION, INTERSECT, or EXCEPT combination, but in this
case it is only permitted to sort by output column names or numbers, not by expressions.
7.6. LIMIT and OFFSET LIMIT and OFFSET allow you to retrieve just a portion of the rows that are generated by the rest of
the query: SELECT select_list FROM table_expression [ ORDER BY ... ] [ LIMIT { number | ALL } ] [ OFFSET number ]
If a limit count is given, no more than that many rows will be returned (but possibly less, if the query itself yields less rows). LIMIT ALL is the same as omitting the LIMIT clause. OFFSET says to skip that many rows before beginning to return rows. OFFSET 0 is the same as omitting the OFFSET clause, and LIMIT NULL is the same as omitting the LIMIT clause. If both OFFSET and LIMIT appear, then OFFSET rows are skipped before starting to count the LIMIT rows
that are returned. When using LIMIT, it is important to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query’s rows. You might be asking for the tenth through twentieth rows, but tenth through twentieth in what ordering? The ordering is unknown, unless you specified ORDER BY. The query optimizer takes LIMIT into account when generating query plans, so you are very likely to get different plans (yielding different row orders) depending on what you give for LIMIT and OFFSET. Thus, using different LIMIT/OFFSET values to select different subsets of a query result will give inconsistent results unless you enforce a predictable result ordering with ORDER BY. This is not
99
Chapter 7. Queries a bug; it is an inherent consequence of the fact that SQL does not promise to deliver the results of a query in any particular order unless ORDER BY is used to constrain the order. The rows skipped by an OFFSET clause still have to be computed inside the server; therefore a large OFFSET might be inefficient.
7.7. VALUES Lists VALUES provides a way to generate a “constant table” that can be used in a query without having to
actually create and populate a table on-disk. The syntax is VALUES ( expression [, ...] ) [, ...]
Each parenthesized list of expressions generates a row in the table. The lists must all have the same number of elements (i.e., the number of columns in the table), and corresponding entries in each list must have compatible data types. The actual data type assigned to each column of the result is determined using the same rules as for UNION (see Section 10.5). As an example: VALUES (1, ’one’), (2, ’two’), (3, ’three’);
will return a table of two columns and three rows. It’s effectively equivalent to: SELECT 1 AS column1, ’one’ AS column2 UNION ALL SELECT 2, ’two’ UNION ALL SELECT 3, ’three’;
By default, PostgreSQL assigns the names column1, column2, etc. to the columns of a VALUES table. The column names are not specified by the SQL standard and different database systems do it differently, so it’s usually better to override the default names with a table alias list, like this: => SELECT * FROM (VALUES (1, ’one’), (2, ’two’), (3, ’three’)) AS t (num,letter); num | letter -----+-------1 | one 2 | two 3 | three (3 rows)
Syntactically, VALUES followed by expression lists is treated as equivalent to: SELECT select_list FROM table_expression
and can appear anywhere a SELECT can. For example, you can use it as part of a UNION, or attach a sort_specification (ORDER BY, LIMIT, and/or OFFSET) to it. VALUES is most commonly used as the data source in an INSERT command, and next most commonly as a subquery. For more information see VALUES.
100
Chapter 7. Queries
7.8. WITH Queries (Common Table Expressions) WITH provides a way to write auxiliary statements for use in a larger query. These statements, which
are often referred to as Common Table Expressions or CTEs, can be thought of as defining temporary tables that exist just for one query. Each auxiliary statement in a WITH clause can be a SELECT, INSERT, UPDATE, or DELETE; and the WITH clause itself is attached to a primary statement that can also be a SELECT, INSERT, UPDATE, or DELETE.
7.8.1. SELECT in WITH The basic value of SELECT in WITH is to break down complicated queries into simpler parts. An example is: WITH regional_sales AS ( SELECT region, SUM(amount) AS total_sales FROM orders GROUP BY region ), top_regions AS ( SELECT region FROM regional_sales WHERE total_sales > (SELECT SUM(total_sales)/10 FROM regional_sales) ) SELECT region, product, SUM(quantity) AS product_units, SUM(amount) AS product_sales FROM orders WHERE region IN (SELECT region FROM top_regions) GROUP BY region, product;
which displays per-product sales totals in only the top sales regions. The WITH clause defines two auxiliary statements named regional_sales and top_regions, where the output of regional_sales is used in top_regions and the output of top_regions is used in the primary SELECT query. This example could have been written without WITH, but we’d have needed two levels of nested sub-SELECTs. It’s a bit easier to follow this way. The optional RECURSIVE modifier changes WITH from a mere syntactic convenience into a feature that accomplishes things not otherwise possible in standard SQL. Using RECURSIVE, a WITH query can refer to its own output. A very simple example is this query to sum the integers from 1 through 100: WITH RECURSIVE t(n) AS ( VALUES (1) UNION ALL SELECT n+1 FROM t WHERE n < 100 ) SELECT sum(n) FROM t;
The general form of a recursive WITH query is always a non-recursive term, then UNION (or UNION ALL), then a recursive term, where only the recursive term can contain a reference to the query’s own output. Such a query is executed as follows:
101
Chapter 7. Queries Recursive Query Evaluation 1.
Evaluate the non-recursive term. For UNION (but not UNION ALL), discard duplicate rows. Include all remaining rows in the result of the recursive query, and also place them in a temporary working table.
2.
So long as the working table is not empty, repeat these steps: a.
Evaluate the recursive term, substituting the current contents of the working table for the recursive self-reference. For UNION (but not UNION ALL), discard duplicate rows and rows that duplicate any previous result row. Include all remaining rows in the result of the recursive query, and also place them in a temporary intermediate table.
b.
Replace the contents of the working table with the contents of the intermediate table, then empty the intermediate table.
Note: Strictly speaking, this process is iteration not recursion, but RECURSIVE is the terminology chosen by the SQL standards committee.
In the example above, the working table has just a single row in each step, and it takes on the values from 1 through 100 in successive steps. In the 100th step, there is no output because of the WHERE clause, and so the query terminates. Recursive queries are typically used to deal with hierarchical or tree-structured data. A useful example is this query to find all the direct and indirect sub-parts of a product, given only a table that shows immediate inclusions: WITH RECURSIVE included_parts(sub_part, part, quantity) AS ( SELECT sub_part, part, quantity FROM parts WHERE part = ’our_product’ UNION ALL SELECT p.sub_part, p.part, p.quantity FROM included_parts pr, parts p WHERE p.part = pr.sub_part ) SELECT sub_part, SUM(quantity) as total_quantity FROM included_parts GROUP BY sub_part
When working with recursive queries it is important to be sure that the recursive part of the query will eventually return no tuples, or else the query will loop indefinitely. Sometimes, using UNION instead of UNION ALL can accomplish this by discarding rows that duplicate previous output rows. However, often a cycle does not involve output rows that are completely duplicate: it may be necessary to check just one or a few fields to see if the same point has been reached before. The standard method for handling such situations is to compute an array of the already-visited values. For example, consider the following query that searches a table graph using a link field: WITH RECURSIVE search_graph(id, link, data, depth) AS ( SELECT g.id, g.link, g.data, 1 FROM graph g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1 FROM graph g, search_graph sg WHERE g.id = sg.link
102
Chapter 7. Queries ) SELECT * FROM search_graph;
This query will loop if the link relationships contain cycles. Because we require a “depth” output, just changing UNION ALL to UNION would not eliminate the looping. Instead we need to recognize whether we have reached the same row again while following a particular path of links. We add two columns path and cycle to the loop-prone query: WITH RECURSIVE search_graph(id, link, data, depth, path, cycle) AS ( SELECT g.id, g.link, g.data, 1, ARRAY[g.id], false FROM graph g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1, path || g.id, g.id = ANY(path) FROM graph g, search_graph sg WHERE g.id = sg.link AND NOT cycle ) SELECT * FROM search_graph;
Aside from preventing cycles, the array value is often useful in its own right as representing the “path” taken to reach any particular row. In the general case where more than one field needs to be checked to recognize a cycle, use an array of rows. For example, if we needed to compare fields f1 and f2: WITH RECURSIVE search_graph(id, link, data, depth, path, cycle) AS ( SELECT g.id, g.link, g.data, 1, ARRAY[ROW(g.f1, g.f2)], false FROM graph g UNION ALL SELECT g.id, g.link, g.data, sg.depth + 1, path || ROW(g.f1, g.f2), ROW(g.f1, g.f2) = ANY(path) FROM graph g, search_graph sg WHERE g.id = sg.link AND NOT cycle ) SELECT * FROM search_graph;
Tip: Omit the ROW() syntax in the common case where only one field needs to be checked to recognize a cycle. This allows a simple array rather than a composite-type array to be used, gaining efficiency.
Tip: The recursive query evaluation algorithm produces its output in breadth-first search order. You can display the results in depth-first search order by making the outer query ORDER BY a “path” column constructed in this way.
A helpful trick for testing queries when you are not certain if they might loop is to place a LIMIT in the parent query. For example, this query would loop forever without the LIMIT:
103
Chapter 7. Queries WITH RECURSIVE t(n) AS ( SELECT 1 UNION ALL SELECT n+1 FROM t ) SELECT n FROM t LIMIT 100;
This works because PostgreSQL’s implementation evaluates only as many rows of a WITH query as are actually fetched by the parent query. Using this trick in production is not recommended, because other systems might work differently. Also, it usually won’t work if you make the outer query sort the recursive query’s results or join them to some other table, because in such cases the outer query will usually try to fetch all of the WITH query’s output anyway. A useful property of WITH queries is that they are evaluated only once per execution of the parent query, even if they are referred to more than once by the parent query or sibling WITH queries. Thus, expensive calculations that are needed in multiple places can be placed within a WITH query to avoid redundant work. Another possible application is to prevent unwanted multiple evaluations of functions with side-effects. However, the other side of this coin is that the optimizer is less able to push restrictions from the parent query down into a WITH query than an ordinary sub-query. The WITH query will generally be evaluated as written, without suppression of rows that the parent query might discard afterwards. (But, as mentioned above, evaluation might stop early if the reference(s) to the query demand only a limited number of rows.) The examples above only show WITH being used with SELECT, but it can be attached in the same way to INSERT, UPDATE, or DELETE. In each case it effectively provides temporary table(s) that can be referred to in the main command.
7.8.2. Data-Modifying Statements in WITH You can use data-modifying statements (INSERT, UPDATE, or DELETE) in WITH. This allows you to perform several different operations in the same query. An example is: WITH moved_rows AS ( DELETE FROM products WHERE "date" >= ’2010-10-01’ AND "date" < ’2010-11-01’ RETURNING * ) INSERT INTO products_log SELECT * FROM moved_rows;
This query effectively moves rows from products to products_log. The DELETE in WITH deletes the specified rows from products, returning their contents by means of its RETURNING clause; and then the primary query reads that output and inserts it into products_log. A fine point of the above example is that the WITH clause is attached to the INSERT, not the subSELECT within the INSERT. This is necessary because data-modifying statements are only allowed in WITH clauses that are attached to the top-level statement. However, normal WITH visibility rules apply, so it is possible to refer to the WITH statement’s output from the sub-SELECT. Data-modifying statements in WITH usually have RETURNING clauses (see Section 6.4), as shown in the example above. It is the output of the RETURNING clause, not the target table of the datamodifying statement, that forms the temporary table that can be referred to by the rest of the query. If a data-modifying statement in WITH lacks a RETURNING clause, then it forms no temporary table
104
Chapter 7. Queries and cannot be referred to in the rest of the query. Such a statement will be executed nonetheless. A not-particularly-useful example is: WITH t AS ( DELETE FROM foo ) DELETE FROM bar;
This example would remove all rows from tables foo and bar. The number of affected rows reported to the client would only include rows removed from bar. Recursive self-references in data-modifying statements are not allowed. In some cases it is possible to work around this limitation by referring to the output of a recursive WITH, for example: WITH RECURSIVE included_parts(sub_part, part) AS ( SELECT sub_part, part FROM parts WHERE part = ’our_product’ UNION ALL SELECT p.sub_part, p.part FROM included_parts pr, parts p WHERE p.part = pr.sub_part ) DELETE FROM parts WHERE part IN (SELECT part FROM included_parts);
This query would remove all direct and indirect subparts of a product. Data-modifying statements in WITH are executed exactly once, and always to completion, independently of whether the primary query reads all (or indeed any) of their output. Notice that this is different from the rule for SELECT in WITH: as stated in the previous section, execution of a SELECT is carried only as far as the primary query demands its output. The sub-statements in WITH are executed concurrently with each other and with the main query. Therefore, when using data-modifying statements in WITH, the order in which the specified updates actually happen is unpredictable. All the statements are executed with the same snapshot (see Chapter 13), so they cannot “see” one another’s effects on the target tables. This alleviates the effects of the unpredictability of the actual order of row updates, and means that RETURNING data is the only way to communicate changes between different WITH sub-statements and the main query. An example of this is that in WITH t AS ( UPDATE products SET price = price * 1.05 RETURNING * ) SELECT * FROM products;
the outer SELECT would return the original prices before the action of the UPDATE, while in WITH t AS ( UPDATE products SET price = price * 1.05 RETURNING * ) SELECT * FROM t;
the outer SELECT would return the updated data. Trying to update the same row twice in a single statement is not supported. Only one of the modifications takes place, but it is not easy (and sometimes not possible) to reliably predict which one. This also applies to deleting a row that was already updated in the same statement: only the update
105
Chapter 7. Queries is performed. Therefore you should generally avoid trying to modify a single row twice in a single statement. In particular avoid writing WITH sub-statements that could affect the same rows changed by the main statement or a sibling sub-statement. The effects of such a statement will not be predictable. At present, any table used as the target of a data-modifying statement in WITH must not have a conditional rule, nor an ALSO rule, nor an INSTEAD rule that expands to multiple statements.
106
Chapter 8. Data Types PostgreSQL has a rich set of native data types available to users. Users can add new types to PostgreSQL using the CREATE TYPE command. Table 8-1 shows all the built-in general-purpose data types. Most of the alternative names listed in the “Aliases” column are the names used internally by PostgreSQL for historical reasons. In addition, some internally used or deprecated types are available, but are not listed here. Table 8-1. Data Types Name
Aliases
Description
bigint
int8
signed eight-byte integer
bigserial
serial8
autoincrementing eight-byte integer fixed-length bit string
bit [ (n) ] bit varying [ (n) ]
varbit
variable-length bit string
boolean
bool
logical Boolean (true/false)
box
rectangular box on a plane
bytea
binary data (“byte array”)
character [ (n) ]
char [ (n) ]
fixed-length character string
character varying [ (n) ]
varchar [ (n) ]
variable-length character string
cidr
IPv4 or IPv6 network address
circle
circle on a plane
date
calendar date (year, month, day)
double precision
float8
IPv4 or IPv6 host address
inet integer
double precision floating-point number (8 bytes)
int, int4
signed four-byte integer
interval [ fields ] [ (p ) ]
time span
json
JSON data
line
infinite line on a plane
lseg
line segment on a plane
macaddr
MAC (Media Access Control) address
money
currency amount
numeric [ (p, s) ]
decimal [ (p, s) ]
exact numeric of selectable precision
path
geometric path on a plane
point
geometric point on a plane
polygon
closed geometric path on a plane
107
Chapter 8. Data Types Name
Aliases
Description
real
float4
single precision floating-point number (4 bytes)
smallint
int2
signed two-byte integer
smallserial
serial2
autoincrementing two-byte integer
serial
serial4
autoincrementing four-byte integer
text
variable-length character string
time [ (p) ] [ without time zone ]
time of day (no time zone)
time [ (p) ] with time zone
time of day, including time zone
timetz
date and time (no time zone)
timestamp [ (p) ] [ without time zone ] timestamp [ (p) ] with time zone
date and time, including time zone
timestamptz
tsquery
text search query
tsvector
text search document
txid_snapshot
user-level transaction ID snapshot
uuid
universally unique identifier
xml
XML data Compatibility: The following types (or spellings thereof) are specified by SQL: bigint, bit, bit varying, boolean, char, character varying, character, varchar, date, double precision, integer, interval, numeric, decimal, real, smallint, time (with or without time zone), timestamp (with or without time zone), xml.
Each data type has an external representation determined by its input and output functions. Many of the built-in types have obvious external formats. However, several types are either unique to PostgreSQL, such as geometric paths, or have several possible formats, such as the date and time types. Some of the input and output functions are not invertible, i.e., the result of an output function might lose accuracy when compared to the original input.
8.1. Numeric Types Numeric types consist of two-, four-, and eight-byte integers, four- and eight-byte floating-point numbers, and selectable-precision decimals. Table 8-2 lists the available types. Table 8-2. Numeric Types Name
Storage Size
Description
Range
smallint
2 bytes
small-range integer
-32768 to +32767
108
Chapter 8. Data Types Name
Storage Size
Description
Range
integer
4 bytes
typical choice for integer
-2147483648 to +2147483647
bigint
8 bytes
large-range integer
9223372036854775808 to +9223372036854775807
decimal
variable
user-specified precision, exact
up to 131072 digits before the decimal point; up to 16383 digits after the decimal point
numeric
variable
user-specified precision, exact
up to 131072 digits before the decimal point; up to 16383 digits after the decimal point
real
4 bytes
variable-precision, inexact
6 decimal digits precision
double precision
8 bytes
variable-precision, inexact
15 decimal digits precision
smallserial
2 bytes
small autoincrementing 1 to 32767 integer
serial
4 bytes
autoincrementing integer
bigserial
8 bytes
large autoincrementing 1 to integer 9223372036854775807
1 to 2147483647
The syntax of constants for the numeric types is described in Section 4.1.2. The numeric types have a full set of corresponding arithmetic operators and functions. Refer to Chapter 9 for more information. The following sections describe the types in detail.
8.1.1. Integer Types The types smallint, integer, and bigint store whole numbers, that is, numbers without fractional components, of various ranges. Attempts to store values outside of the allowed range will result in an error. The type integer is the common choice, as it offers the best balance between range, storage size, and performance. The smallint type is generally only used if disk space is at a premium. The bigint type is designed to be used when the range of the integer type is insufficient. SQL only specifies the integer types integer (or int), smallint, and bigint. The type names int2, int4, and int8 are extensions, which are also used by some other SQL database systems.
109
Chapter 8. Data Types
8.1.2. Arbitrary Precision Numbers The type numeric can store numbers with a very large number of digits and perform calculations exactly. It is especially recommended for storing monetary amounts and other quantities where exactness is required. However, arithmetic on numeric values is very slow compared to the integer types, or to the floating-point types described in the next section. We use the following terms below: The scale of a numeric is the count of decimal digits in the fractional part, to the right of the decimal point. The precision of a numeric is the total count of significant digits in the whole number, that is, the number of digits to both sides of the decimal point. So the number 23.5141 has a precision of 6 and a scale of 4. Integers can be considered to have a scale of zero. Both the maximum precision and the maximum scale of a numeric column can be configured. To declare a column of type numeric use the syntax: NUMERIC(precision, scale)
The precision must be positive, the scale zero or positive. Alternatively: NUMERIC(precision)
selects a scale of 0. Specifying: NUMERIC
without any precision or scale creates a column in which numeric values of any precision and scale can be stored, up to the implementation limit on precision. A column of this kind will not coerce input values to any particular scale, whereas numeric columns with a declared scale will coerce input values to that scale. (The SQL standard requires a default scale of 0, i.e., coercion to integer precision. We find this a bit useless. If you’re concerned about portability, always specify the precision and scale explicitly.) Note: The maximum allowed precision when explicitly specified in the type declaration is 1000; NUMERIC without a specified precision is subject to the limits described in Table 8-2.
If the scale of a value to be stored is greater than the declared scale of the column, the system will round the value to the specified number of fractional digits. Then, if the number of digits to the left of the decimal point exceeds the declared precision minus the declared scale, an error is raised. Numeric values are physically stored without any extra leading or trailing zeroes. Thus, the declared precision and scale of a column are maximums, not fixed allocations. (In this sense the numeric type is more akin to varchar(n) than to char(n).) The actual storage requirement is two bytes for each group of four decimal digits, plus three to eight bytes overhead. In addition to ordinary numeric values, the numeric type allows the special value NaN, meaning “not-a-number”. Any operation on NaN yields another NaN. When writing this value as a constant in an SQL command, you must put quotes around it, for example UPDATE table SET x = ’NaN’. On input, the string NaN is recognized in a case-insensitive manner. Note: In most implementations of the “not-a-number” concept, NaN is not considered equal to any other numeric value (including NaN). In order to allow numeric values to be sorted and used in tree-based indexes, PostgreSQL treats NaN values as equal, and greater than all non-NaN values.
110
Chapter 8. Data Types The types decimal and numeric are equivalent. Both types are part of the SQL standard.
8.1.3. Floating-Point Types The data types real and double precision are inexact, variable-precision numeric types. In practice, these types are usually implementations of IEEE Standard 754 for Binary Floating-Point Arithmetic (single and double precision, respectively), to the extent that the underlying processor, operating system, and compiler support it. Inexact means that some values cannot be converted exactly to the internal format and are stored as approximations, so that storing and retrieving a value might show slight discrepancies. Managing these errors and how they propagate through calculations is the subject of an entire branch of mathematics and computer science and will not be discussed here, except for the following points: •
If you require exact storage and calculations (such as for monetary amounts), use the numeric type instead.
•
If you want to do complicated calculations with these types for anything important, especially if you rely on certain behavior in boundary cases (infinity, underflow), you should evaluate the implementation carefully.
•
Comparing two floating-point values for equality might not always work as expected.
On most platforms, the real type has a range of at least 1E-37 to 1E+37 with a precision of at least 6 decimal digits. The double precision type typically has a range of around 1E-307 to 1E+308 with a precision of at least 15 digits. Values that are too large or too small will cause an error. Rounding might take place if the precision of an input number is too high. Numbers too close to zero that are not representable as distinct from zero will cause an underflow error. Note: The extra_float_digits setting controls the number of extra significant digits included when a floating point value is converted to text for output. With the default value of 0, the output is the same on every platform supported by PostgreSQL. Increasing it will produce output that more accurately represents the stored value, but may be unportable.
In addition to ordinary numeric values, the floating-point types have several special values: Infinity -Infinity NaN
These represent the IEEE 754 special values “infinity”, “negative infinity”, and “not-a-number”, respectively. (On a machine whose floating-point arithmetic does not follow IEEE 754, these values will probably not work as expected.) When writing these values as constants in an SQL command, you must put quotes around them, for example UPDATE table SET x = ’Infinity’. On input, these strings are recognized in a case-insensitive manner. Note: IEEE754 specifies that NaN should not compare equal to any other floating-point value (including NaN). In order to allow floating-point values to be sorted and used in tree-based indexes, PostgreSQL treats NaN values as equal, and greater than all non-NaN values.
111
Chapter 8. Data Types PostgreSQL also supports the SQL-standard notations float and float(p) for specifying inexact numeric types. Here, p specifies the minimum acceptable precision in binary digits. PostgreSQL accepts float(1) to float(24) as selecting the real type, while float(25) to float(53) select double precision. Values of p outside the allowed range draw an error. float with no precision specified is taken to mean double precision. Note: Prior to PostgreSQL 7.4, the precision in float(p) was taken to mean so many decimal digits. This has been corrected to match the SQL standard, which specifies that the precision is measured in binary digits. The assumption that real and double precision have exactly 24 and 53 bits in the mantissa respectively is correct for IEEE-standard floating point implementations. On non-IEEE platforms it might be off a little, but for simplicity the same ranges of p are used on all platforms.
8.1.4. Serial Types The data types smallserial, serial and bigserial are not true types, but merely a notational convenience for creating unique identifier columns (similar to the AUTO_INCREMENT property supported by some other databases). In the current implementation, specifying: CREATE TABLE tablename ( colname SERIAL );
is equivalent to specifying: CREATE SEQUENCE tablename_colname_seq; CREATE TABLE tablename ( colname integer NOT NULL DEFAULT nextval(’tablename_colname_seq’) ); ALTER SEQUENCE tablename_colname_seq OWNED BY tablename.colname;
Thus, we have created an integer column and arranged for its default values to be assigned from a sequence generator. A NOT NULL constraint is applied to ensure that a null value cannot be inserted. (In most cases you would also want to attach a UNIQUE or PRIMARY KEY constraint to prevent duplicate values from being inserted by accident, but this is not automatic.) Lastly, the sequence is marked as “owned by” the column, so that it will be dropped if the column or table is dropped. Note: Because smallserial, serial and bigserial are implemented using sequences, there may be "holes" or gaps in the sequence of values which appears in the column, even if no rows are ever deleted. A value allocated from the sequence is still "used up" even if a row containing that value is never successfully inserted into the table column. This may happen, for example, if the inserting transaction rolls back. See nextval() in Section 9.16 for details.
Note: Prior to PostgreSQL 7.3, serial implied UNIQUE. This is no longer automatic. If you wish a serial column to have a unique constraint or be a primary key, it must now be specified, just like any other data type.
112
Chapter 8. Data Types To insert the next value of the sequence into the serial column, specify that the serial column should be assigned its default value. This can be done either by excluding the column from the list of columns in the INSERT statement, or through the use of the DEFAULT key word. The type names serial and serial4 are equivalent: both create integer columns. The type names bigserial and serial8 work the same way, except that they create a bigint column. bigserial should be used if you anticipate the use of more than 231 identifiers over the lifetime of the table. The type names smallserial and serial2 also work the same way, except that they create a smallint column. The sequence created for a serial column is automatically dropped when the owning column is dropped. You can drop the sequence without dropping the column, but this will force removal of the column default expression.
8.2. Monetary Types The money type stores a currency amount with a fixed fractional precision; see Table 8-3. The fractional precision is determined by the database’s lc_monetary setting. The range shown in the table assumes there are two fractional digits. Input is accepted in a variety of formats, including integer and floating-point literals, as well as typical currency formatting, such as ’$1,000.00’. Output is generally in the latter form but depends on the locale. Table 8-3. Monetary Types Name
Storage Size
Description
Range
money
8 bytes
currency amount
92233720368547758.08 to +92233720368547758.07
Since the output of this data type is locale-sensitive, it might not work to load money data into a database that has a different setting of lc_monetary. To avoid problems, before restoring a dump into a new database make sure lc_monetary has the same or equivalent value as in the database that was dumped. Values of the numeric, int, and bigint data types can be cast to money. Conversion from the real and double precision data types can be done by casting to numeric first, for example: SELECT ’12.34’::float8::numeric::money;
However, this is not recommended. Floating point numbers should not be used to handle money due to the potential for rounding errors. A money value can be cast to numeric without loss of precision. Conversion to other types could potentially lose precision, and must also be done in two stages: SELECT ’52093.89’::money::numeric::float8;
Division of a money value by an integer value is performed with truncation of the fractional part towards zero. To get a rounded result, divide by a floating-point value, or cast the money value
113
Chapter 8. Data Types to numeric before dividing and back to money afterwards. (The latter is preferable to avoid risking precision loss.) When a money value is divided by another money value, the result is double precision (i.e., a pure number, not money); the currency units cancel each other out in the division.
8.3. Character Types Table 8-4. Character Types Name
Description
character varying(n), varchar(n)
variable-length with limit
character(n), char(n)
fixed-length, blank padded
text
variable unlimited length
Table 8-4 shows the general-purpose character types available in PostgreSQL. SQL defines two primary character types: character varying(n) and character(n), where n is a positive integer. Both of these types can store strings up to n characters (not bytes) in length. An attempt to store a longer string into a column of these types will result in an error, unless the excess characters are all spaces, in which case the string will be truncated to the maximum length. (This somewhat bizarre exception is required by the SQL standard.) If the string to be stored is shorter than the declared length, values of type character will be space-padded; values of type character varying will simply store the shorter string. If one explicitly casts a value to character varying(n) or character(n), then an over-length value will be truncated to n characters without raising an error. (This too is required by the SQL standard.) The notations varchar(n) and char(n) are aliases for character varying(n) and character(n), respectively. character without length specifier is equivalent to character(1). If character varying is used without length specifier, the type accepts strings of any size. The latter is a PostgreSQL extension. In addition, PostgreSQL provides the text type, which stores strings of any length. Although the type text is not in the SQL standard, several other SQL database management systems have it as well. Values of type character are physically padded with spaces to the specified width n, and are stored and displayed that way. However, the padding spaces are treated as semantically insignificant. Trailing spaces are disregarded when comparing two values of type character, and they will be removed when converting a character value to one of the other string types. Note that trailing spaces are semantically significant in character varying and text values, and when using pattern matching, e.g. LIKE, regular expressions. The storage requirement for a short string (up to 126 bytes) is 1 byte plus the actual string, which includes the space padding in the case of character. Longer strings have 4 bytes of overhead instead of 1. Long strings are compressed by the system automatically, so the physical requirement on disk might be less. Very long values are also stored in background tables so that they do not interfere with rapid access to shorter column values. In any case, the longest possible character string that can be stored is about 1 GB. (The maximum value that will be allowed for n in the data type declaration is less than that. It wouldn’t be useful to change this because with multibyte character encodings the number of characters and bytes can be quite different. If you desire to store long strings with no specific upper limit, use text or character varying without a length specifier, rather than making up an arbitrary length limit.)
114
Chapter 8. Data Types Tip: There is no performance difference among these three types, apart from increased storage space when using the blank-padded type, and a few extra CPU cycles to check the length when storing into a length-constrained column. While character(n) has performance advantages in some other database systems, there is no such advantage in PostgreSQL; in fact character(n) is usually the slowest of the three because of its additional storage costs. In most situations text or character varying should be used instead.
Refer to Section 4.1.2.1 for information about the syntax of string literals, and to Chapter 9 for information about available operators and functions. The database character set determines the character set used to store textual values; for more information on character set support, refer to Section 22.3. Example 8-1. Using the Character Types CREATE TABLE test1 (a character(4)); INSERT INTO test1 VALUES (’ok’); SELECT a, char_length(a) FROM test1; -- Ê a | char_length ------+------------ok | 2
CREATE INSERT INSERT INSERT
TABLE test2 (b varchar(5)); INTO test2 VALUES (’ok’); INTO test2 VALUES (’good ’); INTO test2 VALUES (’too long’);
ERROR:
value too long for type character varying(5)
INSERT INTO test2 VALUES (’too long’::varchar(5)); -- explicit truncation SELECT b, char_length(b) FROM test2; b | char_length -------+------------ok | 2 good | 5 too l | 5
Ê
The char_length function is discussed in Section 9.4.
There are two other fixed-length character types in PostgreSQL, shown in Table 8-5. The name type exists only for the storage of identifiers in the internal system catalogs and is not intended for use by the general user. Its length is currently defined as 64 bytes (63 usable characters plus terminator) but should be referenced using the constant NAMEDATALEN in C source code. The length is set at compile time (and is therefore adjustable for special uses); the default maximum length might change in a future release. The type "char" (note the quotes) is different from char(1) in that it only uses one byte of storage. It is internally used in the system catalogs as a simplistic enumeration type. Table 8-5. Special Character Types Name
Storage Size
Description
"char"
1 byte
single-byte internal type
name
64 bytes
internal type for object names
115
Chapter 8. Data Types
8.4. Binary Data Types The bytea data type allows storage of binary strings; see Table 8-6. Table 8-6. Binary Data Types Name
Storage Size
Description
bytea
1 or 4 bytes plus the actual binary string
variable-length binary string
A binary string is a sequence of octets (or bytes). Binary strings are distinguished from character strings in two ways. First, binary strings specifically allow storing octets of value zero and other “nonprintable” octets (usually, octets outside the range 32 to 126). Character strings disallow zero octets, and also disallow any other octet values and sequences of octet values that are invalid according to the database’s selected character set encoding. Second, operations on binary strings process the actual bytes, whereas the processing of character strings depends on locale settings. In short, binary strings are appropriate for storing data that the programmer thinks of as “raw bytes”, whereas character strings are appropriate for storing text. The bytea type supports two external formats for input and output: PostgreSQL’s historical “escape” format, and “hex” format. Both of these are always accepted on input. The output format depends on the configuration parameter bytea_output; the default is hex. (Note that the hex format was introduced in PostgreSQL 9.0; earlier versions and some tools don’t understand it.) The SQL standard defines a different binary string type, called BLOB or BINARY LARGE OBJECT. The input format is different from bytea, but the provided functions and operators are mostly the same.
8.4.1. bytea Hex Format The “hex” format encodes binary data as 2 hexadecimal digits per byte, most significant nibble first. The entire string is preceded by the sequence \x (to distinguish it from the escape format). In some contexts, the initial backslash may need to be escaped by doubling it, in the same cases in which backslashes have to be doubled in escape format; details appear below. The hexadecimal digits can be either upper or lower case, and whitespace is permitted between digit pairs (but not within a digit pair nor in the starting \x sequence). The hex format is compatible with a wide range of external applications and protocols, and it tends to be faster to convert than the escape format, so its use is preferred. Example: SELECT E’\\xDEADBEEF’;
8.4.2. bytea Escape Format The “escape” format is the traditional PostgreSQL format for the bytea type. It takes the approach of representing a binary string as a sequence of ASCII characters, while converting those bytes that cannot be represented as an ASCII character into special escape sequences. If, from the point of view of the application, representing bytes as characters makes sense, then this representation can be convenient. But in practice it is usually confusing because it fuzzes up the distinction between binary
116
Chapter 8. Data Types strings and character strings, and also the particular escape mechanism that was chosen is somewhat unwieldy. So this format should probably be avoided for most new applications. When entering bytea values in escape format, octets of certain values must be escaped, while all octet values can be escaped. In general, to escape an octet, convert it into its three-digit octal value and precede it by a backslash (or two backslashes, if writing the value as a literal using escape string syntax). Backslash itself (octet value 92) can alternatively be represented by double backslashes. Table 8-7 shows the characters that must be escaped, and gives the alternative escape sequences where applicable. Table 8-7. bytea Literal Escaped Octets Decimal Octet Value
Description
Escaped Input Example Representation
0
zero octet
E’\\000’
SELECT \000 E’\\000’::bytea;
39
single quote
”” or E’\\047’
SELECT E’\”::bytea;
92
backslash
E’\\\\’ or E’\\134’
SELECT \\ E’\\\\’::bytea;
E’\\xxx’ (octal
SELECT \001 E’\\001’::bytea;
0 to 31 and 127 to “non-printable” 255 octets
value)
Output Representation
’
The requirement to escape non-printable octets varies depending on locale settings. In some instances you can get away with leaving them unescaped. Note that the result in each of the examples in Table 8-7 was exactly one octet in length, even though the output representation is sometimes more than one character. The reason multiple backslashes are required, as shown in Table 8-7, is that an input string written as a string literal must pass through two parse phases in the PostgreSQL server. The first backslash of each pair is interpreted as an escape character by the string-literal parser (assuming escape string syntax is used) and is therefore consumed, leaving the second backslash of the pair. (Dollar-quoted strings can be used to avoid this level of escaping.) The remaining backslash is then recognized by the bytea input function as starting either a three digit octal value or escaping another backslash. For example, a string literal passed to the server as E’\\001’ becomes \001 after passing through the escape string parser. The \001 is then sent to the bytea input function, where it is converted to a single octet with a decimal value of 1. Note that the single-quote character is not treated specially by bytea, so it follows the normal rules for string literals. (See also Section 4.1.2.1.) Bytea octets are sometimes escaped when output. In general, each “non-printable” octet is converted into its equivalent three-digit octal value and preceded by one backslash. Most “printable” octets are represented by their standard representation in the client character set. The octet with decimal value 92 (backslash) is doubled in the output. Details are in Table 8-8.
Table 8-8. bytea Output Escaped Octets
117
Chapter 8. Data Types Decimal Octet Value
Description
Escaped Example Output Representation
92
backslash
\\
Output Result
SELECT \\ E’\\134’::bytea;
0 to 31 and 127 to “non-printable” 255 octets
\xxx (octal value) SELECT \001 E’\\001’::bytea;
32 to 126
client character set representation
“printable” octets
SELECT ~ E’\\176’::bytea;
Depending on the front end to PostgreSQL you use, you might have additional work to do in terms of escaping and unescaping bytea strings. For example, you might also have to escape line feeds and carriage returns if your interface automatically translates these.
8.5. Date/Time Types PostgreSQL supports the full set of SQL date and time types, shown in Table 8-9. The operations available on these data types are described in Section 9.9. Dates are counted according to the Gregorian calendar, even in years before that calendar was introduced (see Section B.4 for more information). Table 8-9. Date/Time Types Name
Storage Size Description
Low Value
High Value
Resolution
timestamp [ 8 bytes (p ) ] [ without time zone ]
both date and time (no time zone)
4713 BC
294276 AD
1 microsecond / 14 digits
timestamp [ 8 bytes (p) ] with time zone
both date and 4713 BC time, with time zone
294276 AD
1 microsecond / 14 digits
date (no time of day)
5874897 AD
1 day
24:00:00
1 microsecond / 14 digits
date
4 bytes
4713 BC
time [ (p) 8 bytes ] [ without time zone ]
time of day (no 00:00:00 date)
time [ (p) 12 bytes ] with time zone
times of day 00:00:00+1459 24:00:00-1459 1 microsecond only, with time / 14 digits zone
118
Chapter 8. Data Types Name
Storage Size Description
Low Value
High Value
Resolution
interval [ fields ] [ (p ) ]
16 bytes
-178000000 years
178000000 years
1 microsecond / 14 digits
time interval
Note: The SQL standard requires that writing just timestamp be equivalent to timestamp without time zone, and PostgreSQL honors that behavior. (Releases prior to 7.3 treated it as timestamp with time zone.) timestamptz is accepted as an abbreviation for timestamp with time zone; this is a PostgreSQL extension.
time, timestamp, and interval accept an optional precision value p which specifies the number of fractional digits retained in the seconds field. By default, there is no explicit bound on precision. The allowed range of p is from 0 to 6 for the timestamp and interval types. Note: When timestamp values are stored as eight-byte integers (currently the default), microsecond precision is available over the full range of values. When timestamp values are stored as double precision floating-point numbers instead (a deprecated compile-time option), the effective limit of precision might be less than 6. timestamp values are stored as seconds before or after midnight 2000-01-01. When timestamp values are implemented using floating-point numbers, microsecond precision is achieved for dates within a few years of 2000-01-01, but the precision degrades for dates further away. Note that using floating-point datetimes allows a larger range of timestamp values to be represented than shown above: from 4713 BC up to 5874897 AD. The same compile-time option also determines whether time and interval values are stored as floating-point numbers or eight-byte integers. In the floating-point case, large interval values degrade in precision as the size of the interval increases.
For the time types, the allowed range of p is from 0 to 6 when eight-byte integer storage is used, or from 0 to 10 when floating-point storage is used. The interval type has an additional option, which is to restrict the set of stored fields by writing one of these phrases: YEAR MONTH DAY HOUR MINUTE SECOND YEAR TO MONTH DAY TO HOUR DAY TO MINUTE DAY TO SECOND HOUR TO MINUTE HOUR TO SECOND MINUTE TO SECOND
Note that if both fields and p are specified, the fields must include SECOND, since the precision applies only to the seconds. The type time with time zone is defined by the SQL standard, but the definition exhibits properties which lead to questionable usefulness. In most cases, a combination of date, time,
119
Chapter 8. Data Types timestamp without time zone,
and timestamp with time zone should provide a complete range of date/time functionality required by any application. The types abstime and reltime are lower precision types which are used internally. You are discouraged from using these types in applications; these internal types might disappear in a future release.
8.5.1. Date/Time Input Date and time input is accepted in almost any reasonable format, including ISO 8601, SQL-compatible, traditional POSTGRES, and others. For some formats, ordering of day, month, and year in date input is ambiguous and there is support for specifying the expected ordering of these fields. Set the DateStyle parameter to MDY to select month-day-year interpretation, DMY to select day-month-year interpretation, or YMD to select year-month-day interpretation. PostgreSQL is more flexible in handling date/time input than the SQL standard requires. See Appendix B for the exact parsing rules of date/time input and for the recognized text fields including months, days of the week, and time zones. Remember that any date or time literal input needs to be enclosed in single quotes, like text strings. Refer to Section 4.1.2.7 for more information. SQL requires the following syntax type [ (p) ] ’value’
where p is an optional precision specification giving the number of fractional digits in the seconds field. Precision can be specified for time, timestamp, and interval types. The allowed values are mentioned above. If no precision is specified in a constant specification, it defaults to the precision of the literal value.
8.5.1.1. Dates Table 8-10 shows some possible inputs for the date type. Table 8-10. Date Input Example
Description
1999-01-08
ISO 8601; January 8 in any mode (recommended format)
January 8, 1999
unambiguous in any datestyle input mode
1/8/1999
January 8 in MDY mode; August 1 in DMY mode
1/18/1999
January 18 in MDY mode; rejected in other modes
01/02/03
January 2, 2003 in MDY mode; February 1, 2003 in DMY mode; February 3, 2001 in YMD mode
1999-Jan-08
January 8 in any mode
Jan-08-1999
January 8 in any mode
08-Jan-1999
January 8 in any mode
99-Jan-08
January 8 in YMD mode, else error
08-Jan-99
January 8, except error in YMD mode
Jan-08-99
January 8, except error in YMD mode
19990108
ISO 8601; January 8, 1999 in any mode
120
Chapter 8. Data Types Example
Description
990108
ISO 8601; January 8, 1999 in any mode
1999.008
year and day of year
J2451187
Julian date
January 8, 99 BC
year 99 BC
8.5.1.2. Times The time-of-day types are time [ (p) ] without time zone and time [ (p) ] with time zone. time alone is equivalent to time without time zone. Valid input for these types consists of a time of day followed by an optional time zone. (See Table 8-11 and Table 8-12.) If a time zone is specified in the input for time without time zone, it is silently ignored. You can also specify a date but it will be ignored, except when you use a time zone name that involves a daylight-savings rule, such as America/New_York. In this case specifying the date is required in order to determine whether standard or daylight-savings time applies. The appropriate time zone offset is recorded in the time with time zone value. Table 8-11. Time Input Example
Description
04:05:06.789
ISO 8601
04:05:06
ISO 8601
04:05
ISO 8601
040506
ISO 8601
04:05 AM
same as 04:05; AM does not affect value
04:05 PM
same as 16:05; input hour must be ’sad’; name | current_mood -------+-------------Moe | happy Curly | ok (2 rows) SELECT * FROM person WHERE current_mood > ’sad’ ORDER BY current_mood; name | current_mood -------+-------------Curly | ok Moe | happy (2 rows) SELECT name FROM person WHERE current_mood = (SELECT MIN(current_mood) FROM person); name ------Larry (1 row)
8.7.3. Type Safety Each enumerated data type is separate and cannot be compared with other enumerated types. See this example: CREATE TYPE happiness AS ENUM (’happy’, ’very happy’, ’ecstatic’); CREATE TABLE holidays ( num_weeks integer, happiness happiness ); INSERT INTO holidays(num_weeks,happiness) VALUES (4, ’happy’); INSERT INTO holidays(num_weeks,happiness) VALUES (6, ’very happy’); INSERT INTO holidays(num_weeks,happiness) VALUES (8, ’ecstatic’); INSERT INTO holidays(num_weeks,happiness) VALUES (2, ’sad’); ERROR: invalid input value for enum happiness: "sad" SELECT person.name, holidays.num_weeks FROM person, holidays WHERE person.current_mood = holidays.happiness; ERROR: operator does not exist: mood = happiness
130
Chapter 8. Data Types If you really need to do something like that, you can either write a custom operator or add explicit casts to your query: SELECT person.name, holidays.num_weeks FROM person, holidays WHERE person.current_mood::text = holidays.happiness::text; name | num_weeks ------+----------Moe | 4 (1 row)
8.7.4. Implementation Details An enum value occupies four bytes on disk. The length of an enum value’s textual label is limited by the NAMEDATALEN setting compiled into PostgreSQL; in standard builds this means at most 63 bytes. Enum labels are case sensitive, so ’happy’ is not the same as ’HAPPY’. White space in the labels is significant too. The translations from internal enum values to textual labels are kept in the system catalog pg_enum. Querying this catalog directly can be useful.
8.8. Geometric Types Geometric data types represent two-dimensional spatial objects. Table 8-20 shows the geometric types available in PostgreSQL. The most fundamental type, the point, forms the basis for all of the other types. Table 8-20. Geometric Types Name
Storage Size
Representation
Description
point
16 bytes
Point on a plane
(x,y)
line
32 bytes
Infinite line (not fully implemented)
((x1,y1),(x2,y2))
lseg
32 bytes
Finite line segment
((x1,y1),(x2,y2))
box
32 bytes
Rectangular box
((x1,y1),(x2,y2))
path
16+16n bytes
Closed path (similar to ((x1,y1),...) polygon)
path
16+16n bytes
Open path
[(x1,y1),...]
polygon
40+16n bytes
Polygon (similar to closed path)
((x1,y1),...)
circle
24 bytes
Circle
(center point and radius)
A rich set of functions and operators is available to perform various geometric operations such as scaling, translation, rotation, and determining intersections. They are explained in Section 9.11.
131
Chapter 8. Data Types
8.8.1. Points Points are the fundamental two-dimensional building block for geometric types. Values of type point are specified using either of the following syntaxes: ( x , y ) x , y
where x and y are the respective coordinates, as floating-point numbers. Points are output using the first syntax.
8.8.2. Line Segments Line segments (lseg) are represented by pairs of points. Values of type lseg are specified using any of the following syntaxes: [ ( x1 , y1 ) ( ( x1 , y1 ) ( x1 , y1 ) x1 , y1
where (x1,y1) and (x2,y2) are the end points of the line segment. Line segments are output using the first syntax.
8.8.3. Boxes Boxes are represented by pairs of points that are opposite corners of the box. Values of type box are specified using any of the following syntaxes: ( ( x1 , y1 ) , ( x2 , y2 ) ) ( x1 , y1 ) , ( x2 , y2 ) x1 , y1 , x2 , y2
where (x1,y1) and (x2,y2) are any two opposite corners of the box. Boxes are output using the second syntax. Any two opposite corners can be supplied on input, but the values will be reordered as needed to store the upper right and lower left corners, in that order.
8.8.4. Paths Paths are represented by lists of connected points. Paths can be open, where the first and last points in the list are considered not connected, or closed, where the first and last points are considered connected. Values of type path are specified using any of the following syntaxes: [ ( x1 ( ( x1 ( x1 ( x1
, , , , x1 ,
y1 ) , ... , ( xn , yn y1 ) , ... , ( xn , yn y1 ) , ... , ( xn , yn y1 , ... , xn , yn y1 , ... , xn , yn
) ] ) ) ) )
132
Chapter 8. Data Types where the points are the end points of the line segments comprising the path. Square brackets ([]) indicate an open path, while parentheses (()) indicate a closed path. When the outermost parentheses are omitted, as in the third through fifth syntaxes, a closed path is assumed. Paths are output using the first or second syntax, as appropriate.
8.8.5. Polygons Polygons are represented by lists of points (the vertexes of the polygon). Polygons are very similar to closed paths, but are stored differently and have their own set of support routines. Values of type polygon are specified using any of the following syntaxes: ( ( x1 , y1 ) , ... , ( xn , yn ) ) ( x1 , y1 ) , ... , ( xn , yn ) ( x1 , y1 , ... , xn , yn ) x1 , y1 , ... , xn , yn
where the points are the end points of the line segments comprising the boundary of the polygon. Polygons are output using the first syntax.
8.8.6. Circles Circles are represented by a center point and radius. Values of type circle are specified using any of the following syntaxes: < ( x , y ) ( ( x , y ) ( x , y ) x , y
, , , ,
r > r ) r r
where (x ,y ) is the center point and r is the radius of the circle. Circles are output using the first syntax.
8.9. Network Address Types PostgreSQL offers data types to store IPv4, IPv6, and MAC addresses, as shown in Table 8-21. It is better to use these types instead of plain text types to store network addresses, because these types offer input error checking and specialized operators and functions (see Section 9.12). Table 8-21. Network Address Types Name
Storage Size
Description
cidr
7 or 19 bytes
IPv4 and IPv6 networks
inet
7 or 19 bytes
IPv4 and IPv6 hosts and networks
macaddr
6 bytes
MAC addresses
When sorting inet or cidr data types, IPv4 addresses will always sort before IPv6 addresses, in-
133
Chapter 8. Data Types cluding IPv4 addresses encapsulated or mapped to IPv6 addresses, such as ::10.2.3.4 or ::ffff:10.4.3.2.
8.9.1. inet The inet type holds an IPv4 or IPv6 host address, and optionally its subnet, all in one field. The subnet is represented by the number of network address bits present in the host address (the “netmask”). If the netmask is 32 and the address is IPv4, then the value does not indicate a subnet, only a single host. In IPv6, the address length is 128 bits, so 128 bits specify a unique host address. Note that if you want to accept only networks, you should use the cidr type rather than inet. The input format for this type is address/y where address is an IPv4 or IPv6 address and y is the number of bits in the netmask. If the /y portion is missing, the netmask is 32 for IPv4 and 128 for IPv6, so the value represents just a single host. On display, the /y portion is suppressed if the netmask specifies a single host.
8.9.2. cidr The cidr type holds an IPv4 or IPv6 network specification. Input and output formats follow Classless Internet Domain Routing conventions. The format for specifying networks is address/y where address is the network represented as an IPv4 or IPv6 address, and y is the number of bits in the netmask. If y is omitted, it is calculated using assumptions from the older classful network numbering system, except it will be at least large enough to include all of the octets written in the input. It is an error to specify a network address that has bits set to the right of the specified netmask. Table 8-22 shows some examples. Table 8-22. cidr Type Input Examples cidr Input
8.9.3. inet vs. cidr The essential difference between inet and cidr data types is that inet accepts values with nonzero bits to the right of the netmask, whereas cidr does not. Tip: If you do not like the output format for inet or cidr values, try the functions host, text, and abbrev.
8.9.4. macaddr The macaddr type stores MAC addresses, known for example from Ethernet card hardware addresses (although MAC addresses are used for other purposes as well). Input is accepted in the following formats: ’08:00:2b:01:02:03’ ’08-00-2b-01-02-03’ ’08002b:010203’ ’08002b-010203’ ’0800.2b01.0203’ ’08002b010203’
These examples would all specify the same address. Upper and lower case is accepted for the digits a through f. Output is always in the first of the forms shown. IEEE Std 802-2001 specifies the second shown form (with hyphens) as the canonical form for MAC addresses, and specifies the first form (with colons) as the bit-reversed notation, so that 08-00-2b-0102-03 = 01:00:4D:08:04:0C. This convention is widely ignored nowadays, and it is relevant only for obsolete network protocols (such as Token Ring). PostgreSQL makes no provisions for bit reversal, and all accepted formats use the canonical LSB order. The remaining four input formats are not part of any standard.
8.10. Bit String Types Bit strings are strings of 1’s and 0’s. They can be used to store or visualize bit masks. There are two SQL bit types: bit(n) and bit varying(n), where n is a positive integer. bit type data must match the length n exactly; it is an error to attempt to store shorter or longer bit strings. bit varying data is of variable length up to the maximum length n; longer strings will be rejected. Writing bit without a length is equivalent to bit(1), while bit varying without a length
specification means unlimited length. Note: If one explicitly casts a bit-string value to bit(n), it will be truncated or zero-padded on the right to be exactly n bits, without raising an error. Similarly, if one explicitly casts a bit-string value to bit varying(n), it will be truncated on the right if it is more than n bits.
Refer to Section 4.1.2.5 for information about the syntax of bit string constants. Bit-logical operators and string manipulation functions are available; see Section 9.6.
135
Chapter 8. Data Types Example 8-3. Using the Bit String Types CREATE TABLE test (a BIT(3), b BIT VARYING(5)); INSERT INTO test VALUES (B’101’, B’00’); INSERT INTO test VALUES (B’10’, B’101’); ERROR:
bit string length 2 does not match type bit(3)
INSERT INTO test VALUES (B’10’::bit(3), B’101’); SELECT * FROM test; a | b -----+----101 | 00 100 | 101
A bit string value requires 1 byte for each group of 8 bits, plus 5 or 8 bytes overhead depending on the length of the string (but long values may be compressed or moved out-of-line, as explained in Section 8.3 for character strings).
8.11. Text Search Types PostgreSQL provides two data types that are designed to support full text search, which is the activity of searching through a collection of natural-language documents to locate those that best match a query. The tsvector type represents a document in a form optimized for text search; the tsquery type similarly represents a text query. Chapter 12 provides a detailed explanation of this facility, and Section 9.13 summarizes the related functions and operators.
8.11.1. tsvector A tsvector value is a sorted list of distinct lexemes, which are words that have been normalized to merge different variants of the same word (see Chapter 12 for details). Sorting and duplicateelimination are done automatically during input, as shown in this example: SELECT ’a fat cat sat on a mat and ate a fat rat’::tsvector; tsvector ---------------------------------------------------’a’ ’and’ ’ate’ ’cat’ ’fat’ ’mat’ ’on’ ’rat’ ’sat’
To represent lexemes containing whitespace or punctuation, surround them with quotes: SELECT $$the lexeme ’ ’ contains spaces$$::tsvector; tsvector ------------------------------------------’ ’ ’contains’ ’lexeme’ ’spaces’ ’the’
(We use dollar-quoted string literals in this example and the next one to avoid the confusion of having to double quote marks within the literals.) Embedded quotes and backslashes must be doubled: SELECT $$the lexeme ’Joe”s’ contains a quote$$::tsvector; tsvector -----------------------------------------------’Joe”s’ ’a’ ’contains’ ’lexeme’ ’quote’ ’the’
Optionally, integer positions can be attached to lexemes: SELECT ’a:1 fat:2 cat:3 sat:4 on:5 a:6 mat:7 and:8 ate:9 a:10 fat:11 rat:12’::tsvector;
A position normally indicates the source word’s location in the document. Positional information can be used for proximity ranking. Position values can range from 1 to 16383; larger numbers are silently set to 16383. Duplicate positions for the same lexeme are discarded. Lexemes that have positions can further be labeled with a weight, which can be A, B, C, or D. D is the default and hence is not shown on output: SELECT ’a:1A fat:2B,4C cat:5D’::tsvector; tsvector ---------------------------’a’:1A ’cat’:5 ’fat’:2B,4C
Weights are typically used to reflect document structure, for example by marking title words differently from body words. Text search ranking functions can assign different priorities to the different weight markers. It is important to understand that the tsvector type itself does not perform any normalization; it assumes the words it is given are normalized appropriately for the application. For example, select ’The Fat Rats’::tsvector; tsvector -------------------’Fat’ ’Rats’ ’The’
For most English-text-searching applications the above words would be considered non-normalized, but tsvector doesn’t care. Raw document text should usually be passed through to_tsvector to normalize the words appropriately for searching: SELECT to_tsvector(’english’, ’The Fat Rats’); to_tsvector ----------------’fat’:2 ’rat’:3
Again, see Chapter 12 for more detail.
8.11.2. tsquery A tsquery value stores lexemes that are to be searched for, and combines them honoring the Boolean operators & (AND), | (OR), and ! (NOT). Parentheses can be used to enforce grouping of the operators: SELECT ’fat & rat’::tsquery; tsquery --------------’fat’ & ’rat’ SELECT ’fat & (rat | cat)’::tsquery; tsquery --------------------------’fat’ & ( ’rat’ | ’cat’ ) SELECT ’fat & rat & ! cat’::tsquery;
137
Chapter 8. Data Types tsquery -----------------------’fat’ & ’rat’ & !’cat’
In the absence of parentheses, ! (NOT) binds most tightly, and & (AND) binds more tightly than | (OR). Optionally, lexemes in a tsquery can be labeled with one or more weight letters, which restricts them to match only tsvector lexemes with matching weights: SELECT ’fat:ab & cat’::tsquery; tsquery -----------------’fat’:AB & ’cat’
Also, lexemes in a tsquery can be labeled with * to specify prefix matching: SELECT ’super:*’::tsquery; tsquery ----------’super’:*
This query will match any word in a tsvector that begins with “super”. Note that prefixes are first processed by text search configurations, which means this comparison returns true: SELECT to_tsvector( ’postgraduate’ ) @@ to_tsquery( ’postgres:*’ ); ?column? ---------t (1 row)
because postgres gets stemmed to postgr: SELECT to_tsquery(’postgres:*’); to_tsquery -----------’postgr’:* (1 row)
which then matches postgraduate. Quoting rules for lexemes are the same as described previously for lexemes in tsvector; and, as with tsvector, any required normalization of words must be done before converting to the tsquery type. The to_tsquery function is convenient for performing such normalization: SELECT to_tsquery(’Fat:ab & Cats’); to_tsquery -----------------’fat’:AB & ’cat’
138
Chapter 8. Data Types
8.12. UUID Type The data type uuid stores Universally Unique Identifiers (UUID) as defined by RFC 4122, ISO/IEC 9834-8:2005, and related standards. (Some systems refer to this data type as a globally unique identifier, or GUID, instead.) This identifier is a 128-bit quantity that is generated by an algorithm chosen to make it very unlikely that the same identifier will be generated by anyone else in the known universe using the same algorithm. Therefore, for distributed systems, these identifiers provide a better uniqueness guarantee than sequence generators, which are only unique within a single database. A UUID is written as a sequence of lower-case hexadecimal digits, in several groups separated by hyphens, specifically a group of 8 digits followed by three groups of 4 digits followed by a group of 12 digits, for a total of 32 digits representing the 128 bits. An example of a UUID in this standard form is: a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11
PostgreSQL also accepts the following alternative forms for input: use of upper-case digits, the standard format surrounded by braces, omitting some or all hyphens, adding a hyphen after any group of four digits. Examples are: A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11 {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11} a0eebc999c0b4ef8bb6d6bb9bd380a11 a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11 {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}
Output is always in the standard form. PostgreSQL provides storage and comparison functions for UUIDs, but the core database does not include any function for generating UUIDs, because no single algorithm is well suited for every application. The uuid-ossp module provides functions that implement several standard algorithms. Alternatively, UUIDs could be generated by client applications or other libraries invoked through a server-side function.
8.13. XML Type The xml data type can be used to store XML data. Its advantage over storing XML data in a text field is that it checks the input values for well-formedness, and there are support functions to perform type-safe operations on it; see Section 9.14. Use of this data type requires the installation to have been built with configure --with-libxml. The xml type can store well-formed “documents”, as defined by the XML standard, as well as “content” fragments, which are defined by the production XMLDecl? content in the XML standard. Roughly, this means that content fragments can have more than one top-level element or character node. The expression xmlvalue IS DOCUMENT can be used to evaluate whether a particular xml value is a full document or only a content fragment.
8.13.1. Creating XML Values To produce a value of type xml from character data, use the function xmlparse: XMLPARSE ( { DOCUMENT | CONTENT } value)
Examples:
139
Chapter 8. Data Types
XMLPARSE (DOCUMENT ’Manual... 3; -- Overlaps SELECT numrange(11.1, 22.2) && numrange(20.0, 30.0); -- Extract the upper bound SELECT upper(int8range(15, 25)); -- Compute the intersection SELECT int4range(10, 20) * int4range(15, 25); -- Is the range empty?
157
Chapter 8. Data Types SELECT isempty(numrange(1, 5));
See Table 9-45 and Table 9-46 for complete lists of operators and functions on range types.
8.17.3. Inclusive and Exclusive Bounds Every non-empty range has two bounds, the lower bound and the upper bound. All points between these values are included in the range. An inclusive bound means that the boundary point itself is included in the range as well, while an exclusive bound means that the boundary point is not included in the range. In the text form of a range, an inclusive lower bound is represented by “[” while an exclusive lower bound is represented by “(”. Likewise, an inclusive upper bound is represented by “]”, while an exclusive upper bound is represented by “)”. (See Section 8.17.5 for more details.) The functions lower_inc and upper_inc test the inclusivity of the lower and upper bounds of a range value, respectively.
8.17.4. Infinite (Unbounded) Ranges The lower bound of a range can be omitted, meaning that all points less than the upper bound are included in the range. Likewise, if the upper bound of the range is omitted, then all points greater than the lower bound are included in the range. If both lower and upper bounds are omitted, all values of the element type are considered to be in the range. This is equivalent to considering that the lower bound is “minus infinity”, or the upper bound is “plus infinity”, respectively. But note that these infinite values are never values of the range’s element type, and can never be part of the range. (So there is no such thing as an inclusive infinite bound — if you try to write one, it will automatically be converted to an exclusive bound.) Also, some element types have a notion of “infinity”, but that is just another value so far as the range type mechanisms are concerned. For example, in timestamp ranges, [today,] means the same thing as [today,). But [today,infinity] means something different from [today,infinity) — the latter excludes the special timestamp value infinity. The functions lower_inf and upper_inf test for infinite lower and upper bounds of a range, respectively.
8.17.5. Range Input/Output The input for a range value must follow one of the following patterns: (lower-bound ,upper-bound ) (lower-bound ,upper-bound ] [lower-bound ,upper-bound ) [lower-bound ,upper-bound ] empty
The parentheses or brackets indicate whether the lower and upper bounds are exclusive or inclusive, as described previously. Notice that the final pattern is empty, which represents an empty range (a range that contains no points).
158
Chapter 8. Data Types The lower-bound may be either a string that is valid input for the subtype, or empty to indicate no lower bound. Likewise, upper-bound may be either a string that is valid input for the subtype, or empty to indicate no upper bound. Each bound value can be quoted using " (double quote) characters. This is necessary if the bound value contains parentheses, brackets, commas, double quotes, or backslashes, since these characters would otherwise be taken as part of the range syntax. To put a double quote or backslash in a quoted bound value, precede it with a backslash. (Also, a pair of double quotes within a double-quoted bound value is taken to represent a double quote character, analogously to the rules for single quotes in SQL literal strings.) Alternatively, you can avoid quoting and use backslash-escaping to protect all data characters that would otherwise be taken as range syntax. Also, to write a bound value that is an empty string, write "", since writing nothing means an infinite bound. Whitespace is allowed before and after the range value, but any whitespace between the parentheses or brackets is taken as part of the lower or upper bound value. (Depending on the element type, it might or might not be significant.) Note: These rules are very similar to those for writing field values in composite-type literals. See Section 8.16.6 for additional commentary.
Examples: -- includes 3, does not include 7, and does include all points in between SELECT ’[3,7)’::int4range; -- does not include either 3 or 7, but includes all points in between SELECT ’(3,7)’::int4range; -- includes only the single point 4 SELECT ’[4,4]’::int4range; -- includes no points (and will be normalized to ’empty’) SELECT ’[4,4)’::int4range;
8.17.6. Constructing Ranges Each range type has a constructor function with the same name as the range type. Using the constructor function is frequently more convenient than writing a range literal constant, since it avoids the need for extra quoting of the bound values. The constructor function accepts two or three arguments. The two-argument form constructs a range in standard form (lower bound inclusive, upper bound exclusive), while the three-argument form constructs a range with bounds of the form specified by the third argument. The third argument must be one of the strings “()”, “(]”, “[)”, or “[]”. For example: -- The full form is: lower bound, upper bound, and text argument indicating -- inclusivity/exclusivity of bounds. SELECT numrange(1.0, 14.0, ’(]’); -- If the third argument is omitted, ’[)’ is assumed. SELECT numrange(1.0, 14.0);
159
Chapter 8. Data Types -- Although ’(]’ is specified here, on display the value will be converted to -- canonical form, since int8range is a discrete range type (see below). SELECT int8range(1, 14, ’(]’); -- Using NULL for either bound causes the range to be unbounded on that side. SELECT numrange(NULL, 2.2);
8.17.7. Discrete Range Types A discrete range is one whose element type has a well-defined “step”, such as integer or date. In these types two elements can be said to be adjacent, when there are no valid values between them. This contrasts with continuous ranges, where it’s always (or almost always) possible to identify other element values between two given values. For example, a range over the numeric type is continuous, as is a range over timestamp. (Even though timestamp has limited precision, and so could theoretically be treated as discrete, it’s better to consider it continuous since the step size is normally not of interest.) Another way to think about a discrete range type is that there is a clear idea of a “next” or “previous” value for each element value. Knowing that, it is possible to convert between inclusive and exclusive representations of a range’s bounds, by choosing the next or previous element value instead of the one originally given. For example, in an integer range type [4,8] and (3,9) denote the same set of values; but this would not be so for a range over numeric. A discrete range type should have a canonicalization function that is aware of the desired step size for the element type. The canonicalization function is charged with converting equivalent values of the range type to have identical representations, in particular consistently inclusive or exclusive bounds. If a canonicalization function is not specified, then ranges with different formatting will always be treated as unequal, even though they might represent the same set of values in reality. The built-in range types int4range, int8range, and daterange all use a canonical form that includes the lower bound and excludes the upper bound; that is, [). User-defined range types can use other conventions, however.
8.17.8. Defining New Range Types Users can define their own range types. The most common reason to do this is to use ranges over subtypes not provided among the built-in range types. For example, to define a new range type of subtype float8: CREATE TYPE floatrange AS RANGE ( subtype = float8, subtype_diff = float8mi ); SELECT ’[1.234, 5.678]’::floatrange;
Because float8 has no meaningful “step”, we do not define a canonicalization function in this example. If the subtype is considered to have discrete rather than continuous values, the CREATE TYPE command should specify a canonical function. The canonicalization function takes an input range value,
160
Chapter 8. Data Types and must return an equivalent range value that may have different bounds and formatting. The canonical output for two ranges that represent the same set of values, for example the integer ranges [1, 7] and [1, 8), must be identical. It doesn’t matter which representation you choose to be the canonical one, so long as two equivalent values with different formattings are always mapped to the same value with the same formatting. In addition to adjusting the inclusive/exclusive bounds format, a canonicalization function might round off boundary values, in case the desired step size is larger than what the subtype is capable of storing. For instance, a range type over timestamp could be defined to have a step size of an hour, in which case the canonicalization function would need to round off bounds that weren’t a multiple of an hour, or perhaps throw an error instead. Defining your own range type also allows you to specify a different subtype B-tree operator class or collation to use, so as to change the sort ordering that determines which values fall into a given range. In addition, any range type that is meant to be used with GiST or SP-GiST indexes should define a subtype difference, or subtype_diff, function. (the index will still work without subtype_diff, but it is likely to be considerably less efficient than if a difference function is provided.) The subtype difference function takes two input values of the subtype, and returns their difference (i.e., X minus Y ) represented as a float8 value. In our example above, the function that underlies the regular float8 minus operator can be used; but for any other subtype, some type conversion would be necessary. Some creative thought about how to represent differences as numbers might be needed, too. To the greatest extent possible, the subtype_diff function should agree with the sort ordering implied by the selected operator class and collation; that is, its result should be positive whenever its first argument is greater than its second according to the sort ordering. See CREATE TYPE for more information about creating range types.
8.17.9. Indexing GiST and SP-GiST indexes can be created for table columns of range types. For instance, to create a GiST index: CREATE INDEX reservation_idx ON reservation USING gist (during);
A GiST or SP-GiST index can accelerate queries involving these range operators: =, &&, , , -|-, & (see Table 9-45 for more information). In addition, B-tree and hash indexes can be created for table columns of range types. For these index types, basically the only useful range operation is equality. There is a B-tree sort ordering defined for range values, with corresponding < and > operators, but the ordering is rather arbitrary and not usually useful in the real world. Range types’ B-tree and hash support is primarily meant to allow sorting and hashing internally in queries, rather than creation of actual indexes.
8.17.10. Constraints on Ranges While UNIQUE is a natural constraint for scalar values, it is usually unsuitable for range types. Instead, an exclusion constraint is often more appropriate (see CREATE TABLE ... CONSTRAINT ... EXCLUDE). Exclusion constraints allow the specification of constraints such as “non-overlapping” on a range type. For example: CREATE TABLE reservation ( during tsrange, EXCLUDE USING gist (during WITH &&) );
161
Chapter 8. Data Types That constraint will prevent any overlapping values from existing in the table at the same time: INSERT INTO reservation VALUES (’[2010-01-01 11:30, 2010-01-01 15:00)’); INSERT 0 1 INSERT INTO reservation VALUES (’[2010-01-01 14:45, 2010-01-01 15:45)’); ERROR: conflicting key value violates exclusion constraint "reservation_during_excl" DETAIL: Key (during)=(["2010-01-01 14:45:00","2010-01-01 15:45:00")) conflicts with existing key (during)=(["2010-01-01 11:30:00","2010-01-01 15:00:00")).
You can use the btree_gist extension to define exclusion constraints on plain scalar data types, which can then be combined with range exclusions for maximum flexibility. For example, after btree_gist is installed, the following constraint will reject overlapping ranges only if the meeting room numbers are equal: CREATE EXTENSION btree_gist; CREATE TABLE room_reservation ( room text, during tsrange, EXCLUDE USING gist (room WITH =, during WITH &&) ); INSERT INTO room_reservation VALUES (’123A’, ’[2010-01-01 14:00, 2010-01-01 15:00)’); INSERT 0 1
8.18. Object Identifier Types Object identifiers (OIDs) are used internally by PostgreSQL as primary keys for various system tables. OIDs are not added to user-created tables, unless WITH OIDS is specified when the table is created, or the default_with_oids configuration variable is enabled. Type oid represents an object identifier. There are also several alias types for oid: regproc, regprocedure, regoper, regoperator, regclass, regtype, regconfig, and regdictionary. Table 8-23 shows an overview. The oid type is currently implemented as an unsigned four-byte integer. Therefore, it is not large enough to provide database-wide uniqueness in large databases, or even in large individual tables. So, using a user-created table’s OID column as a primary key is discouraged. OIDs are best used only for references to system tables.
162
Chapter 8. Data Types The oid type itself has few operations beyond comparison. It can be cast to integer, however, and then manipulated using the standard integer operators. (Beware of possible signed-versus-unsigned confusion if you do this.) The OID alias types have no operations of their own except for specialized input and output routines. These routines are able to accept and display symbolic names for system objects, rather than the raw numeric value that type oid would use. The alias types allow simplified lookup of OID values for objects. For example, to examine the pg_attribute rows related to a table mytable, one could write: SELECT * FROM pg_attribute WHERE attrelid = ’mytable’::regclass;
rather than: SELECT * FROM pg_attribute WHERE attrelid = (SELECT oid FROM pg_class WHERE relname = ’mytable’);
While that doesn’t look all that bad by itself, it’s still oversimplified. A far more complicated subselect would be needed to select the right OID if there are multiple tables named mytable in different schemas. The regclass input converter handles the table lookup according to the schema path setting, and so it does the “right thing” automatically. Similarly, casting a table’s OID to regclass is handy for symbolic display of a numeric OID. Table 8-23. Object Identifier Types Name
References
Description
Value Example
oid
any
numeric object identifier
564182
regproc
pg_proc
function name
sum
regprocedure
pg_proc
function with argument sum(int4) types
regoper
pg_operator
operator name
regoperator
pg_operator
operator with argument *(integer,integer) or -(NONE,integer) types
regclass
pg_class
relation name
pg_type
regtype
pg_type
data type name
integer
regconfig
pg_ts_config
text search configuration
english
regdictionary
pg_ts_dict
text search dictionary
simple
+
All of the OID alias types accept schema-qualified names, and will display schema-qualified names on output if the object would not be found in the current search path without being qualified. The regproc and regoper alias types will only accept input names that are unique (not overloaded), so they are of limited use; for most uses regprocedure or regoperator are more appropriate. For regoperator, unary operators are identified by writing NONE for the unused operand. An additional property of the OID alias types is the creation of dependencies. If a constant of one of these types appears in a stored expression (such as a column default expression or view), it creates a dependency on the referenced object. For example, if a column has a default expression nextval(’my_seq’::regclass), PostgreSQL understands that the default expression depends on the sequence my_seq; the system will not let the sequence be dropped without first removing the default expression.
163
Chapter 8. Data Types Another identifier type used by the system is xid, or transaction (abbreviated xact) identifier. This is the data type of the system columns xmin and xmax. Transaction identifiers are 32-bit quantities. A third identifier type used by the system is cid, or command identifier. This is the data type of the system columns cmin and cmax. Command identifiers are also 32-bit quantities. A final identifier type used by the system is tid, or tuple identifier (row identifier). This is the data type of the system column ctid. A tuple ID is a pair (block number, tuple index within block) that identifies the physical location of the row within its table. (The system columns are further explained in Section 5.4.)
8.19. Pseudo-Types The PostgreSQL type system contains a number of special-purpose entries that are collectively called pseudo-types. A pseudo-type cannot be used as a column data type, but it can be used to declare a function’s argument or result type. Each of the available pseudo-types is useful in situations where a function’s behavior does not correspond to simply taking or returning a value of a specific SQL data type. Table 8-24 lists the existing pseudo-types. Table 8-24. Pseudo-Types Name
Description
any
Indicates that a function accepts any input data type.
anyelement
Indicates that a function accepts any data type (see Section 35.2.5).
anyarray
Indicates that a function accepts any array data type (see Section 35.2.5).
anynonarray
Indicates that a function accepts any non-array data type (see Section 35.2.5).
anyenum
Indicates that a function accepts any enum data type (see Section 35.2.5 and Section 8.7).
anyrange
Indicates that a function accepts any range data type (see Section 35.2.5 and Section 8.17).
cstring
Indicates that a function accepts or returns a null-terminated C string.
internal
Indicates that a function accepts or returns a server-internal data type.
language_handler
A procedural language call handler is declared to return language_handler.
fdw_handler
A foreign-data wrapper handler is declared to return fdw_handler.
record
Identifies a function taking or returning an unspecified row type.
trigger
A trigger function is declared to return trigger.
event_trigger
An event trigger function is declared to return event_trigger.
164
Chapter 8. Data Types Name
Description
void
Indicates that a function returns no value.
opaque
An obsolete type name that formerly served all the above purposes.
Functions coded in C (whether built-in or dynamically loaded) can be declared to accept or return any of these pseudo data types. It is up to the function author to ensure that the function will behave safely when a pseudo-type is used as an argument type. Functions coded in procedural languages can use pseudo-types only as allowed by their implementation languages. At present most procedural languages forbid use of a pseudo-type as an argument type, and allow only void and record as a result type (plus trigger or event_trigger when the function is used as a trigger or event trigger). Some also support polymorphic functions using the types anyelement, anyarray, anynonarray, anyenum, and anyrange. The internal pseudo-type is used to declare functions that are meant only to be called internally by the database system, and not by direct invocation in an SQL query. If a function has at least one internal-type argument then it cannot be called from SQL. To preserve the type safety of this restriction it is important to follow this coding rule: do not create any function that is declared to return internal unless it has at least one internal argument.
165
Chapter 9. Functions and Operators PostgreSQL provides a large number of functions and operators for the built-in data types. Users can also define their own functions and operators, as described in Part V. The psql commands \df and \do can be used to list all available functions and operators, respectively. If you are concerned about portability then note that most of the functions and operators described in this chapter, with the exception of the most trivial arithmetic and comparison operators and some explicitly marked functions, are not specified by the SQL standard. Some of this extended functionality is present in other SQL database management systems, and in many cases this functionality is compatible and consistent between the various implementations. This chapter is also not exhaustive; additional functions appear in relevant sections of the manual.
9.1. Logical Operators The usual logical operators are available: AND OR NOT
SQL uses a three-valued logic system with true, false, and null, which represents “unknown”. Observe the following truth tables: a
b
a AND b
a OR b
TRUE
TRUE
TRUE
TRUE
TRUE
FALSE
FALSE
TRUE
TRUE
NULL
NULL
TRUE
FALSE
FALSE
FALSE
FALSE
FALSE
NULL
FALSE
NULL
NULL
NULL
NULL
NULL
a
NOT a
TRUE
FALSE
FALSE
TRUE
NULL
NULL
The operators AND and OR are commutative, that is, you can switch the left and right operand without affecting the result. But see Section 4.2.14 for more information about the order of evaluation of subexpressions.
9.2. Comparison Operators The usual comparison operators are available, shown in Table 9-1.
166
Chapter 9. Functions and Operators Table 9-1. Comparison Operators Operator
Description
greater than
=
greater than or equal to
=
equal
or !=
not equal
Note: The != operator is converted to in the parser stage. It is not possible to implement != and operators that do different things.
Comparison operators are available for all relevant data types. All comparison operators are binary operators that return values of type boolean; expressions like 1 < 2 < 3 are not valid (because there is no < operator to compare a Boolean value with 3). In addition to the comparison operators, the special BETWEEN construct is available: a BETWEEN x AND y
is equivalent to a >= x AND a y
BETWEEN SYMMETRIC is the same as BETWEEN except there is no requirement that the argument to the left of AND be less than or equal to the argument on the right. If it is not, those two arguments are
automatically swapped, so that a nonempty range is always implied. Ordinary comparison operators yield null (signifying “unknown”), not true or false, when either input is null. For example, 7 = NULL yields null, as does 7 NULL. When this behavior is not suitable, use the IS [ NOT ] DISTINCT FROM constructs: a IS DISTINCT FROM b a IS NOT DISTINCT FROM b
For non-null inputs, IS DISTINCT FROM is the same as the operator. However, if both inputs are null it returns false, and if only one input is null it returns true. Similarly, IS NOT DISTINCT FROM is identical to = for non-null inputs, but it returns true when both inputs are null, and false when only one input is null. Thus, these constructs effectively act as though null were a normal data value, rather than “unknown”. To check whether a value is or is not null, use the constructs: expression IS NULL
167
Chapter 9. Functions and Operators expression IS NOT NULL
or the equivalent, but nonstandard, constructs: expression ISNULL expression NOTNULL
Do not write expression = NULL because NULL is not “equal to” NULL. (The null value represents an unknown value, and it is not known whether two unknown values are equal.) Tip: Some applications might expect that expression = NULL returns true if expression evaluates to the null value. It is highly recommended that these applications be modified to comply with the SQL standard. However, if that cannot be done the transform_null_equals configuration variable is available. If it is enabled, PostgreSQL will convert x = NULL clauses to x IS NULL.
If the expression is row-valued, then IS NULL is true when the row expression itself is null or when all the row’s fields are null, while IS NOT NULL is true when the row expression itself is non-null and all the row’s fields are non-null. Because of this behavior, IS NULL and IS NOT NULL do not always return inverse results for row-valued expressions; in particular, a row-valued expression that contains both null and non-null fields will return false for both tests. In some cases, it may be preferable to write row IS DISTINCT FROM NULL or row IS NOT DISTINCT FROM NULL, which will simply check whether the overall row value is null without any additional tests on the row fields. Boolean values can also be tested using the constructs expression expression expression expression expression expression
IS IS IS IS IS IS
TRUE NOT TRUE FALSE NOT FALSE UNKNOWN NOT UNKNOWN
These will always return true or false, never a null value, even when the operand is null. A null input is treated as the logical value “unknown”. Notice that IS UNKNOWN and IS NOT UNKNOWN are effectively the same as IS NULL and IS NOT NULL, respectively, except that the input expression must be of Boolean type.
9.3. Mathematical Functions and Operators Mathematical operators are provided for many PostgreSQL types. For types without standard mathematical conventions (e.g., date/time types) we describe the actual behavior in subsequent sections. Table 9-2 shows the available mathematical operators. Table 9-2. Mathematical Operators Operator
Description
Example
Result
+
addition
2 + 3
5
-
subtraction
2 - 3
-1
*
multiplication
2 * 3
6
168
Chapter 9. Functions and Operators Operator
Description
Example
Result
/
division (integer division truncates the result)
4 / 2
2
%
modulo (remainder)
5 % 4
1
^
exponentiation 2.0 ^ 3.0 (associates left to right)
8
|/
square root
|/ 25.0
5
||/
cube root
||/ 27.0
3
!
factorial
5 !
120
!!
factorial (prefix operator)
!! 5
120
@
absolute value
@ -5.0
5
&
bitwise AND
91 & 15
11
|
bitwise OR
32 | 3
35
#
bitwise XOR
17 # 5
20
~
bitwise NOT
~1
-2
> 2
2
The bitwise operators work only on integral data types, whereas the others are available for all numeric data types. The bitwise operators are also available for the bit string types bit and bit varying, as shown in Table 9-11. Table 9-3 shows the available mathematical functions. In the table, dp indicates double precision. Many of these functions are provided in multiple forms with different argument types. Except where noted, any given form of a function returns the same data type as its argument. The functions working with double precision data are mostly implemented on top of the host system’s C library; accuracy and behavior in boundary cases can therefore vary depending on the host system. Table 9-3. Mathematical Functions Function
Return Type
Description
Example
Result
abs(x )
(same as input)
absolute value
abs(-17.4)
17.4
cbrt(dp)
dp
cube root
cbrt(27.0)
3
ceil(dp or
(same as input)
nearest integer ceil(-42.8) greater than or equal to argument
(same as input)
nearest integer ceiling(-95.3) -95 greater than or equal to argument (same as ceil)
degrees(dp)
dp
radians to degrees degrees(0.5)
28.6478897565412
div(y numeric,
numeric
integer quotient of div(9,4) y/ x
2
numeric)
ceiling(dp or numeric)
x numeric)
-42
169
Chapter 9. Functions and Operators Function
Return Type
Description
Example
Result
(same as input)
exponential
exp(1.0)
2.71828182845905
(same as input)
nearest integer less than or equal to argument
floor(-42.8)
-43
(same as input)
natural logarithm
ln(2.0)
0.693147180559945
(same as input)
base 10 logarithm log(100.0)
2
log(b numeric, x numeric
logarithm to base b
log(2.0, 64.0)
6.0000000000
numeric)
mod(9,4)
1
exp(dp or numeric)
floor(dp or numeric)
ln(dp or numeric)
log(dp or numeric)
mod(y, x)
(same as argument remainder of y/x types)
pi()
dp
“π” constant
pi()
3.14159265358979
power(a dp, b
dp
a raised to the power of b
power(9.0, 3.0)
729
a raised to the power of b
power(9.0, 3.0)
729
dp)
power(a numeric, numeric b numeric)
radians(dp)
dp
degrees to radians radians(45.0)
0.785398163397448
round(dp or
(same as input)
round to nearest integer
round(42.4)
42
round to s decimal places
round(42.4382, 42.44 2)
(same as input)
sign of the argument (-1, 0, +1)
sign(-8.4)
-1
(same as input)
square root
sqrt(2.0)
1.4142135623731
(same as input)
truncate toward zero
trunc(42.8)
42
truncate to s decimal places
trunc(42.4382, 42.43 2)
numeric)
round(v numeric, numeric s int)
sign(dp or numeric)
sqrt(dp or numeric)
trunc(dp or numeric)
trunc(v numeric, numeric s int)
int width_bucket(op numeric, b1 numeric, b2 numeric, count int)
return the bucket width_bucket(5.35, 3 to which operand 0.024, 10.06, would be assigned 5) in an equidepth histogram with count buckets, in the range b1 to b2
170
Chapter 9. Functions and Operators Function
Return Type
Description
width_bucket(op
int
return the bucket width_bucket(5.35, 3 to which operand 0.024, 10.06, would be assigned 5) in an equidepth histogram with count buckets, in the range b1 to b2
dp, b1 dp, b2 dp, count int)
Example
Result
Table 9-4 shows functions for generating random numbers. Table 9-4. Random Functions Function
Return Type
Description
random()
dp
random value in the range 0.0 2
00100
189
Chapter 9. Functions and Operators The following SQL-standard functions work on bit strings as well as character strings: length, bit_length, octet_length, position, substring, overlay. The following functions work on bit strings as well as binary strings: get_bit, set_bit. When working with a bit string, these functions number the first (leftmost) bit of the string as bit 0. In addition, it is possible to cast integral values to and from type bit. Some examples: 44::bit(10) 44::bit(3) cast(-44 as bit(12)) ’1110’::bit(4)::integer
0000101100 100 111111010100 14
Note that casting to just “bit” means casting to bit(1), and so will deliver only the least significant bit of the integer. Note: Prior to PostgreSQL 8.0, casting an integer to bit(n) would copy the leftmost n bits of the integer, whereas now it copies the rightmost n bits. Also, casting an integer to a bit string width wider than the integer itself will sign-extend on the left.
9.7. Pattern Matching There are three separate approaches to pattern matching provided by PostgreSQL: the traditional SQL LIKE operator, the more recent SIMILAR TO operator (added in SQL:1999), and POSIX-style regular expressions. Aside from the basic “does this string match this pattern?” operators, functions are available to extract or replace matching substrings and to split a string at matching locations. Tip: If you have pattern matching needs that go beyond this, consider writing a user-defined function in Perl or Tcl.
Caution While most regular-expression searches can be executed very quickly, regular expressions can be contrived that take arbitrary amounts of time and memory to process. Be wary of accepting regular-expression search patterns from hostile sources. If you must do so, it is advisable to impose a statement timeout. Searches using SIMILAR TO patterns have the same security hazards, since SIMILAR TO provides many of the same capabilities as POSIX-style regular expressions. LIKE searches, being much simpler than the other two options, are safer to use
with possibly-hostile pattern sources.
9.7.1. LIKE string LIKE pattern [ESCAPE escape-character ] string NOT LIKE pattern [ESCAPE escape-character ]
190
Chapter 9. Functions and Operators The LIKE expression returns true if the string matches the supplied pattern. (As expected, the NOT LIKE expression returns false if LIKE returns true, and vice versa. An equivalent expression is NOT (string LIKE pattern).) If pattern does not contain percent signs or underscores, then the pattern only represents the string itself; in that case LIKE acts like the equals operator. An underscore (_) in pattern stands for (matches) any single character; a percent sign (%) matches any sequence of zero or more characters. Some examples: ’abc’ ’abc’ ’abc’ ’abc’
LIKE LIKE LIKE LIKE
’abc’ ’a%’ ’_b_’ ’c’
true true true false
LIKE pattern matching always covers the entire string. Therefore, if it’s desired to match a sequence
anywhere within a string, the pattern must start and end with a percent sign. To match a literal underscore or percent sign without matching other characters, the respective character in pattern must be preceded by the escape character. The default escape character is the backslash but a different one can be selected by using the ESCAPE clause. To match the escape character itself, write two escape characters. Note: If you have standard_conforming_strings turned off, any backslashes you write in literal string constants will need to be doubled. See Section 4.1.2.1 for more information.
It’s also possible to select no escape character by writing ESCAPE ”. This effectively disables the escape mechanism, which makes it impossible to turn off the special meaning of underscore and percent signs in the pattern. The key word ILIKE can be used instead of LIKE to make the match case-insensitive according to the active locale. This is not in the SQL standard but is a PostgreSQL extension. The operator ~~ is equivalent to LIKE, and ~~* corresponds to ILIKE. There are also !~~ and !~~* operators that represent NOT LIKE and NOT ILIKE, respectively. All of these operators are PostgreSQL-specific.
9.7.2. SIMILAR TO Regular Expressions string SIMILAR TO pattern [ESCAPE escape-character ] string NOT SIMILAR TO pattern [ESCAPE escape-character ]
The SIMILAR TO operator returns true or false depending on whether its pattern matches the given string. It is similar to LIKE, except that it interprets the pattern using the SQL standard’s definition of a regular expression. SQL regular expressions are a curious cross between LIKE notation and common regular expression notation. Like LIKE, the SIMILAR TO operator succeeds only if its pattern matches the entire string; this is unlike common regular expression behavior where the pattern can match any part of the string. Also like LIKE, SIMILAR TO uses _ and % as wildcard characters denoting any single character and any string, respectively (these are comparable to . and .* in POSIX regular expressions).
191
Chapter 9. Functions and Operators In addition to these facilities borrowed from LIKE, SIMILAR TO supports these pattern-matching metacharacters borrowed from POSIX regular expressions: • |
denotes alternation (either of two alternatives).
• *
denotes repetition of the previous item zero or more times.
• +
denotes repetition of the previous item one or more times.
• ?
denotes repetition of the previous item zero or one time.
• {m}
denotes repetition of the previous item exactly m times.
• {m,}
denotes repetition of the previous item m or more times.
• {m,n}
denotes repetition of the previous item at least m and not more than n times.
•
Parentheses () can be used to group items into a single logical item.
•
A bracket expression [...] specifies a character class, just as in POSIX regular expressions.
Notice that the period (.) is not a metacharacter for SIMILAR TO. As with LIKE, a backslash disables the special meaning of any of these metacharacters; or a different escape character can be specified with ESCAPE. Some examples: ’abc’ ’abc’ ’abc’ ’abc’
SIMILAR SIMILAR SIMILAR SIMILAR
TO TO TO TO
’abc’ ’a’ ’%(b|d)%’ ’(b|c)%’
true false true false
The substring function with three parameters, substring(string from pattern for escape-character ), provides extraction of a substring that matches an SQL regular expression pattern. As with SIMILAR TO, the specified pattern must match the entire data string, or else the function fails and returns null. To indicate the part of the pattern that should be returned on success, the pattern must contain two occurrences of the escape character followed by a double quote ("). The text matching the portion of the pattern between these markers is returned. Some examples, with #" delimiting the return string: substring(’foobar’ from ’%#"o_b#"%’ for ’#’) substring(’foobar’ from ’#"o_b#"%’ for ’#’)
oob NULL
9.7.3. POSIX Regular Expressions Table 9-12 lists the available operators for pattern matching using POSIX regular expressions. Table 9-12. Regular Expression Match Operators Operator
Description
Example
~
Matches regular expression, case sensitive
’thomas’ ~ ’.*thomas.*’
192
Chapter 9. Functions and Operators Operator
Description
Example
~*
Matches regular expression, case insensitive
’thomas’ ~* ’.*Thomas.*’
!~
Does not match regular expression, case sensitive
’thomas’ !~ ’.*Thomas.*’
!~*
Does not match regular expression, case insensitive
’thomas’ !~* ’.*vadim.*’
POSIX regular expressions provide a more powerful means for pattern matching than the LIKE and SIMILAR TO operators. Many Unix tools such as egrep, sed, or awk use a pattern matching language that is similar to the one described here. A regular expression is a character sequence that is an abbreviated definition of a set of strings (a regular set). A string is said to match a regular expression if it is a member of the regular set described by the regular expression. As with LIKE, pattern characters match string characters exactly unless they are special characters in the regular expression language — but regular expressions use different special characters than LIKE does. Unlike LIKE patterns, a regular expression is allowed to match anywhere within a string, unless the regular expression is explicitly anchored to the beginning or end of the string. Some examples: ’abc’ ’abc’ ’abc’ ’abc’
~ ~ ~ ~
’abc’ ’^a’ ’(b|d)’ ’^(b|c)’
true true true false
The POSIX pattern language is described in much greater detail below. The substring function with two parameters, substring(string from pattern), provides extraction of a substring that matches a POSIX regular expression pattern. It returns null if there is no match, otherwise the portion of the text that matched the pattern. But if the pattern contains any parentheses, the portion of the text that matched the first parenthesized subexpression (the one whose left parenthesis comes first) is returned. You can put parentheses around the whole expression if you want to use parentheses within it without triggering this exception. If you need parentheses in the pattern before the subexpression you want to extract, see the non-capturing parentheses described below. Some examples: substring(’foobar’ from ’o.b’) substring(’foobar’ from ’o(.)b’)
oob o
The regexp_replace function provides substitution of new text for substrings that match POSIX regular expression patterns. It has the syntax regexp_replace(source, pattern, replacement [, flags ]). The source string is returned unchanged if there is no match to the pattern. If there is a match, the source string is returned with the replacement string substituted for the matching substring. The replacement string can contain \n, where n is 1 through 9, to indicate that the source substring matching the n’th parenthesized subexpression of the pattern should be inserted, and it can contain \& to indicate that the substring matching the entire pattern should be inserted. Write \\ if you need to put a literal backslash in the replacement text. The flags parameter is an optional text string containing zero or more single-letter flags that change the function’s behavior. Flag i specifies
193
Chapter 9. Functions and Operators case-insensitive matching, while flag g specifies replacement of each matching substring rather than only the first one. Other supported flags are described in Table 9-20. Some examples: regexp_replace(’foobarbaz’, ’b..’, ’X’) fooXbaz regexp_replace(’foobarbaz’, ’b..’, ’X’, ’g’) fooXX regexp_replace(’foobarbaz’, ’b(..)’, E’X\\1Y’, ’g’) fooXarYXazY
The regexp_matches function returns a text array of all of the captured substrings resulting from matching a POSIX regular expression pattern. It has the syntax regexp_matches(string , pattern [, flags ]). The function can return no rows, one row, or multiple rows (see the g flag below). If the pattern does not match, the function returns no rows. If the pattern contains no parenthesized subexpressions, then each row returned is a single-element text array containing the substring matching the whole pattern. If the pattern contains parenthesized subexpressions, the function returns a text array whose n’th element is the substring matching the n’th parenthesized subexpression of the pattern (not counting “non-capturing” parentheses; see below for details). The flags parameter is an optional text string containing zero or more single-letter flags that change the function’s behavior. Flag g causes the function to find each match in the string, not only the first one, and return a row for each such match. Other supported flags are described in Table 9-20. Some examples: SELECT regexp_matches(’foobarbequebaz’, ’(bar)(beque)’); regexp_matches ---------------{bar,beque} (1 row) SELECT regexp_matches(’foobarbequebazilbarfbonk’, ’(b[^b]+)(b[^b]+)’, ’g’); regexp_matches ---------------{bar,beque} {bazil,barf} (2 rows) SELECT regexp_matches(’foobarbequebaz’, ’barbeque’); regexp_matches ---------------{barbeque} (1 row)
It is possible to force regexp_matches() to always return one row by using a sub-select; this is particularly useful in a SELECT target list when you want all rows returned, even non-matching ones: SELECT col1, (SELECT regexp_matches(col2, ’(bar)(beque)’)) FROM tab;
The regexp_split_to_table function splits a string using a POSIX regular expression pattern as a delimiter. It has the syntax regexp_split_to_table(string , pattern [, flags ]). If there is no match to the pattern, the function returns the string . If there is at least one match, for each
194
Chapter 9. Functions and Operators match it returns the text from the end of the last match (or the beginning of the string) to the beginning of the match. When there are no more matches, it returns the text from the end of the last match to the end of the string. The flags parameter is an optional text string containing zero or more single-letter flags that change the function’s behavior. regexp_split_to_table supports the flags described in Table 9-20. The regexp_split_to_array function behaves the same as regexp_split_to_table, except that regexp_split_to_array returns its result as an array of text. It has the syntax regexp_split_to_array(string , pattern [, flags ]). The parameters are the same as for regexp_split_to_table. Some examples:
SELECT foo FROM regexp_split_to_table(’the quick brown fox jumps over the lazy dog’, E’\\ foo ------the quick brown fox jumps over the lazy dog (9 rows) SELECT regexp_split_to_array(’the quick brown fox jumps over the lazy dog’, E’\\s+’); regexp_split_to_array ----------------------------------------------{the,quick,brown,fox,jumps,over,the,lazy,dog} (1 row) SELECT foo FROM regexp_split_to_table(’the quick brown fox’, E’\\s*’) AS foo; foo ----t h e q u i c k b r o w n f o x (16 rows)
As the last example demonstrates, the regexp split functions ignore zero-length matches that occur at the start or end of the string or immediately after a previous match. This is contrary to the strict
195
Chapter 9. Functions and Operators definition of regexp matching that is implemented by regexp_matches, but is usually the most convenient behavior in practice. Other software systems such as Perl use similar definitions.
9.7.3.1. Regular Expression Details PostgreSQL’s regular expressions are implemented using a software package written by Henry Spencer. Much of the description of regular expressions below is copied verbatim from his manual. Regular expressions (REs), as defined in POSIX 1003.2, come in two forms: extended REs or EREs (roughly those of egrep), and basic REs or BREs (roughly those of ed). PostgreSQL supports both forms, and also implements some extensions that are not in the POSIX standard, but have become widely used due to their availability in programming languages such as Perl and Tcl. REs using these non-POSIX extensions are called advanced REs or AREs in this documentation. AREs are almost an exact superset of EREs, but BREs have several notational incompatibilities (as well as being much more limited). We first describe the ARE and ERE forms, noting features that apply only to AREs, and then describe how BREs differ. Note: PostgreSQL always initially presumes that a regular expression follows the ARE rules. However, the more limited ERE or BRE rules can be chosen by prepending an embedded option to the RE pattern, as described in Section 9.7.3.4. This can be useful for compatibility with applications that expect exactly the POSIX 1003.2 rules.
A regular expression is defined as one or more branches, separated by |. It matches anything that matches one of the branches. A branch is zero or more quantified atoms or constraints, concatenated. It matches a match for the first, followed by a match for the second, etc; an empty branch matches the empty string. A quantified atom is an atom possibly followed by a single quantifier. Without a quantifier, it matches a match for the atom. With a quantifier, it can match some number of matches of the atom. An atom can be any of the possibilities shown in Table 9-13. The possible quantifiers and their meanings are shown in Table 9-14. A constraint matches an empty string, but matches only when specific conditions are met. A constraint can be used where an atom could be used, except it cannot be followed by a quantifier. The simple constraints are shown in Table 9-15; some more constraints are described later. Table 9-13. Regular Expression Atoms Atom
Description
(re)
(where re is any regular expression) matches a match for re, with the match noted for possible reporting
(?:re)
as above, but the match is not noted for reporting (a “non-capturing” set of parentheses) (AREs only)
.
matches any single character
[chars]
a bracket expression, matching any one of the chars (see Section 9.7.3.2 for more detail)
196
Chapter 9. Functions and Operators Atom
Description
\k
(where k is a non-alphanumeric character) matches that character taken as an ordinary character, e.g., \\ matches a backslash character
\c
where c is alphanumeric (possibly followed by other characters) is an escape, see Section 9.7.3.3 (AREs only; in EREs and BREs, this matches c)
{
when followed by a character other than a digit, matches the left-brace character {; when followed by a digit, it is the beginning of a bound (see below)
x
where x is a single character with no other significance, matches that character
An RE cannot end with a backslash (\). Note: If you have standard_conforming_strings turned off, any backslashes you write in literal string constants will need to be doubled. See Section 4.1.2.1 for more information.
a sequence of m through n (inclusive) matches of the atom; m cannot exceed n
*?
non-greedy version of *
+?
non-greedy version of +
??
non-greedy version of ?
{m}?
non-greedy version of {m}
{m,}?
non-greedy version of {m,}
{m,n}?
non-greedy version of {m,n}
The forms using {...} are known as bounds. The numbers m and n within a bound are unsigned decimal integers with permissible values from 0 to 255 inclusive. Non-greedy quantifiers (available in AREs only) match the same possibilities as their corresponding normal (greedy) counterparts, but prefer the smallest number rather than the largest number of matches. See Section 9.7.3.5 for more detail. Note: A quantifier cannot immediately follow another quantifier, e.g., ** is invalid. A quantifier cannot begin an expression or subexpression or follow ^ or |.
positive lookahead matches at any point where a substring matching re begins (AREs only)
(?!re)
negative lookahead matches at any point where no substring matching re begins (AREs only)
Lookahead constraints cannot contain back references (see Section 9.7.3.3), and all parentheses within them are considered non-capturing.
9.7.3.2. Bracket Expressions A bracket expression is a list of characters enclosed in []. It normally matches any single character from the list (but see below). If the list begins with ^, it matches any single character not from the rest of the list. If two characters in the list are separated by -, this is shorthand for the full range of characters between those two (inclusive) in the collating sequence, e.g., [0-9] in ASCII matches any decimal digit. It is illegal for two ranges to share an endpoint, e.g., a-c-e. Ranges are very collating-sequence-dependent, so portable programs should avoid relying on them. To include a literal ] in the list, make it the first character (after ^, if that is used). To include a literal -, make it the first or last character, or the second endpoint of a range. To use a literal - as the first endpoint of a range, enclose it in [. and .] to make it a collating element (see below). With the exception of these characters, some combinations using [ (see next paragraphs), and escapes (AREs only), all other special characters lose their special significance within a bracket expression. In particular, \ is not special when following ERE or BRE rules, though it is special (as introducing an escape) in AREs. Within a bracket expression, a collating element (a character, a multiple-character sequence that collates as if it were a single character, or a collating-sequence name for either) enclosed in [. and .] stands for the sequence of characters of that collating element. The sequence is treated as a single element of the bracket expression’s list. This allows a bracket expression containing a multiple-character collating element to match more than one character, e.g., if the collating sequence includes a ch collating element, then the RE [[.ch.]]*c matches the first five characters of chchcc. Note: PostgreSQL currently does not support multi-character collating elements. This information describes possible future behavior.
Within a bracket expression, a collating element enclosed in [= and =] is an equivalence class, standing for the sequences of characters of all collating elements equivalent to that one, including itself. (If there are no other equivalent collating elements, the treatment is as if the enclosing delimiters were [. and .].) For example, if o and ^ are the members of an equivalence class, then [[=o=]], [[=^=]], and [o^] are all synonymous. An equivalence class cannot be an endpoint of a range. Within a bracket expression, the name of a character class enclosed in [: and :] stands for the list of all characters belonging to that class. Standard character class names are: alnum, alpha, blank, cntrl, digit, graph, lower, print, punct, space, upper, xdigit. These stand for the character
198
Chapter 9. Functions and Operators classes defined in ctype. A locale can provide others. A character class cannot be used as an endpoint of a range. There are two special cases of bracket expressions: the bracket expressions [[::]] are constraints, matching empty strings at the beginning and end of a word respectively. A word is defined as a sequence of word characters that is neither preceded nor followed by word characters. A word character is an alnum character (as defined by ctype) or an underscore. This is an extension, compatible with but not specified by POSIX 1003.2, and should be used with caution in software intended to be portable to other systems. The constraint escapes described below are usually preferable; they are no more standard, but are easier to type.
9.7.3.3. Regular Expression Escapes Escapes are special sequences beginning with \ followed by an alphanumeric character. Escapes come in several varieties: character entry, class shorthands, constraint escapes, and back references. A \ followed by an alphanumeric character but not constituting a valid escape is illegal in AREs. In EREs, there are no escapes: outside a bracket expression, a \ followed by an alphanumeric character merely stands for that character as an ordinary character, and inside a bracket expression, \ is an ordinary character. (The latter is the one actual incompatibility between EREs and AREs.) Character-entry escapes exist to make it easier to specify non-printing and other inconvenient characters in REs. They are shown in Table 9-16. Class-shorthand escapes provide shorthands for certain commonly-used character classes. They are shown in Table 9-17. A constraint escape is a constraint, matching the empty string if specific conditions are met, written as an escape. They are shown in Table 9-18. A back reference (\n) matches the same string matched by the previous parenthesized subexpression specified by the number n (see Table 9-19). For example, ([bc])\1 matches bb or cc but not bc or cb. The subexpression must entirely precede the back reference in the RE. Subexpressions are numbered in the order of their leading parentheses. Non-capturing parentheses do not define subexpressions. Note: Keep in mind that an escape’s leading \ will need to be doubled when entering the pattern as an SQL string constant. For example: ’123’ ~ E’^\\d{3}’ true
synonym for backslash (\) to help reduce the need for backslash doubling
\cX
(where X is any character) the character whose low-order 5 bits are the same as those of X , and whose other bits are all zero
199
Chapter 9. Functions and Operators Escape \e
Description the character whose collating-sequence name is ESC, or failing that, the character with octal value 033
\f
form feed, as in C
\n
newline, as in C
\r
carriage return, as in C
\t
horizontal tab, as in C
\uwxyz
(where wxyz is exactly four hexadecimal digits) the character whose hexadecimal value is 0xwxyz
\Ustuvwxyz
(where stuvwxyz is exactly eight hexadecimal digits) the character whose hexadecimal value is 0xstuvwxyz
\v
vertical tab, as in C
\xhhh
(where hhh is any sequence of hexadecimal digits) the character whose hexadecimal value is 0xhhh (a single character no matter how many hexadecimal digits are used)
\0
the character whose value is 0 (the null byte)
\xy
(where xy is exactly two octal digits, and is not a back reference) the character whose octal value is 0xy
\xyz
(where xyz is exactly three octal digits, and is not a back reference) the character whose octal value is 0xyz
Hexadecimal digits are 0-9, a-f, and A-F. Octal digits are 0-7. Numeric character-entry escapes specifying values outside the ASCII range (0-127) have meanings dependent on the database encoding. When the encoding is UTF-8, escape values are equivalent to Unicode code points, for example \u1234 means the character U+1234. For other multibyte encodings, character-entry escapes usually just specify the concatenation of the byte values for the character. If the escape value does not correspond to any legal character in the database encoding, no error will be raised, but it will never match any data. The character-entry escapes are always taken as ordinary characters. For example, \135 is ] in ASCII, but \135 does not terminate a bracket expression. Table 9-17. Regular Expression Class-shorthand Escapes Escape
Description
\d
[[:digit:]]
\s
[[:space:]]
\w
[[:alnum:]_] (note underscore is included)
\D
[^[:digit:]]
\S
[^[:space:]]
\W
[^[:alnum:]_] (note underscore is included)
200
Chapter 9. Functions and Operators Within bracket expressions, \d, \s, and \w lose their outer brackets, and \D, \S, and \W are illegal. (So, for example, [a-c\d] is equivalent to [a-c[:digit:]]. Also, [a-c\D], which is equivalent to [a-c^[:digit:]], is illegal.) Table 9-18. Regular Expression Constraint Escapes Escape
Description
\A
matches only at the beginning of the string (see Section 9.7.3.5 for how this differs from ^)
\m
matches only at the beginning of a word
\M
matches only at the end of a word
\y
matches only at the beginning or end of a word
\Y
matches only at a point that is not the beginning or end of a word
\Z
matches only at the end of the string (see Section 9.7.3.5 for how this differs from $)
A word is defined as in the specification of [[::]] above. Constraint escapes are illegal within bracket expressions. Table 9-19. Regular Expression Back References Escape
Description
\m
(where m is a nonzero digit) a back reference to the m’th subexpression
\mnn
(where m is a nonzero digit, and nn is some more digits, and the decimal value mnn is not greater than the number of closing capturing parentheses seen so far) a back reference to the mnn’th subexpression
Note: There is an inherent ambiguity between octal character-entry escapes and back references, which is resolved by the following heuristics, as hinted at above. A leading zero always indicates an octal escape. A single non-zero digit, not followed by another digit, is always taken as a back reference. A multi-digit sequence not starting with a zero is taken as a back reference if it comes after a suitable subexpression (i.e., the number is in the legal range for a back reference), and otherwise is taken as octal.
9.7.3.4. Regular Expression Metasyntax In addition to the main syntax described above, there are some special forms and miscellaneous syntactic facilities available. An RE can begin with one of two special director prefixes. If an RE begins with ***:, the rest of the RE is taken as an ARE. (This normally has no effect in PostgreSQL, since REs are assumed to be AREs; but it does have an effect if ERE or BRE mode had been specified by the flags parameter to a regex function.) If an RE begins with ***=, the rest of the RE is taken to be a literal string, with all characters considered ordinary characters.
201
Chapter 9. Functions and Operators An ARE can begin with embedded options: a sequence (?xyz) (where xyz is one or more alphabetic characters) specifies options affecting the rest of the RE. These options override any previously determined options — in particular, they can override the case-sensitivity behavior implied by a regex operator, or the flags parameter to a regex function. The available option letters are shown in Table 9-20. Note that these same option letters are used in the flags parameters of regex functions. Table 9-20. ARE Embedded-option Letters Option
Description
b
rest of RE is a BRE
c
case-sensitive matching (overrides operator type)
e
rest of RE is an ERE
i
case-insensitive matching (see Section 9.7.3.5) (overrides operator type)
m
historical synonym for n
n
newline-sensitive matching (see Section 9.7.3.5)
p
partial newline-sensitive matching (see Section 9.7.3.5)
q
rest of RE is a literal (“quoted”) string, all ordinary characters
s
non-newline-sensitive matching (default)
t
tight syntax (default; see below)
w
inverse partial newline-sensitive (“weird”) matching (see Section 9.7.3.5)
x
expanded syntax (see below)
Embedded options take effect at the ) terminating the sequence. They can appear only at the start of an ARE (after the ***: director if any). In addition to the usual (tight) RE syntax, in which all characters are significant, there is an expanded syntax, available by specifying the embedded x option. In the expanded syntax, white-space characters in the RE are ignored, as are all characters between a # and the following newline (or the end of the RE). This permits paragraphing and commenting a complex RE. There are three exceptions to that basic rule: •
a white-space character or # preceded by \ is retained
•
white space or # within a bracket expression is retained
•
white space and comments cannot appear within multi-character symbols, such as (?:
For this purpose, white-space characters are blank, tab, newline, and any character that belongs to the space character class. Finally, in an ARE, outside bracket expressions, the sequence (?#ttt) (where ttt is any text not containing a )) is a comment, completely ignored. Again, this is not allowed between the characters of multi-character symbols, like (?:. Such comments are more a historical artifact than a useful facility, and their use is deprecated; use the expanded syntax instead. None of these metasyntax extensions is available if an initial ***= director has specified that the user’s input be treated as a literal string rather than as an RE.
202
Chapter 9. Functions and Operators
9.7.3.5. Regular Expression Matching Rules In the event that an RE could match more than one substring of a given string, the RE matches the one starting earliest in the string. If the RE could match more than one substring starting at that point, either the longest possible match or the shortest possible match will be taken, depending on whether the RE is greedy or non-greedy. Whether an RE is greedy or not is determined by the following rules: •
Most atoms, and all constraints, have no greediness attribute (because they cannot match variable amounts of text anyway).
•
Adding parentheses around an RE does not change its greediness.
•
A quantified atom with a fixed-repetition quantifier ({m} or {m}?) has the same greediness (possibly none) as the atom itself.
•
A quantified atom with other normal quantifiers (including {m,n} with m equal to n) is greedy (prefers longest match).
•
A quantified atom with a non-greedy quantifier (including {m,n}? with m equal to n) is non-greedy (prefers shortest match).
•
A branch — that is, an RE that has no top-level | operator — has the same greediness as the first quantified atom in it that has a greediness attribute.
•
An RE consisting of two or more branches connected by the | operator is always greedy.
The above rules associate greediness attributes not only with individual quantified atoms, but with branches and entire REs that contain quantified atoms. What that means is that the matching is done in such a way that the branch, or whole RE, matches the longest or shortest possible substring as a whole. Once the length of the entire match is determined, the part of it that matches any particular subexpression is determined on the basis of the greediness attribute of that subexpression, with subexpressions starting earlier in the RE taking priority over ones starting later. An example of what this means: SELECT SUBSTRING(’XY1234Z’, ’Y*([0-9]{1,3})’); Result: 123 SELECT SUBSTRING(’XY1234Z’, ’Y*?([0-9]{1,3})’); Result: 1
In the first case, the RE as a whole is greedy because Y* is greedy. It can match beginning at the Y, and it matches the longest possible string starting there, i.e., Y123. The output is the parenthesized part of that, or 123. In the second case, the RE as a whole is non-greedy because Y*? is non-greedy. It can match beginning at the Y, and it matches the shortest possible string starting there, i.e., Y1. The subexpression [0-9]{1,3} is greedy but it cannot change the decision as to the overall match length; so it is forced to match just 1. In short, when an RE contains both greedy and non-greedy subexpressions, the total match length is either as long as possible or as short as possible, according to the attribute assigned to the whole RE. The attributes assigned to the subexpressions only affect how much of that match they are allowed to “eat” relative to each other. The quantifiers {1,1} and {1,1}? can be used to force greediness or non-greediness, respectively, on a subexpression or a whole RE. This is useful when you need the whole RE to have a greediness attribute different from what’s deduced from its elements. As an example, suppose that we are trying
203
Chapter 9. Functions and Operators to separate a string containing some digits into the digits and the parts before and after them. We might try to do that like this: SELECT regexp_matches(’abc01234xyz’, ’(.*)(\d+)(.*)’); Result: {abc0123,4,xyz}
That didn’t work: the first .* is greedy so it “eats” as much as it can, leaving the \d+ to match at the last possible place, the last digit. We might try to fix that by making it non-greedy: SELECT regexp_matches(’abc01234xyz’, ’(.*?)(\d+)(.*)’); Result: {abc,0,""}
That didn’t work either, because now the RE as a whole is non-greedy and so it ends the overall match as soon as possible. We can get what we want by forcing the RE as a whole to be greedy: SELECT regexp_matches(’abc01234xyz’, ’(?:(.*?)(\d+)(.*)){1,1}’); Result: {abc,01234,xyz}
Controlling the RE’s overall greediness separately from its components’ greediness allows great flexibility in handling variable-length patterns. When deciding what is a longer or shorter match, match lengths are measured in characters, not collating elements. An empty string is considered longer than no match at all. For example: bb* matches the three middle characters of abbbc; (week|wee)(night|knights) matches all ten characters of weeknights; when (.*).* is matched against abc the parenthesized subexpression matches all three characters; and when (a*)* is matched against bc both the whole RE and the parenthesized subexpression match an empty string. If case-independent matching is specified, the effect is much as if all case distinctions had vanished from the alphabet. When an alphabetic that exists in multiple cases appears as an ordinary character outside a bracket expression, it is effectively transformed into a bracket expression containing both cases, e.g., x becomes [xX]. When it appears inside a bracket expression, all case counterparts of it are added to the bracket expression, e.g., [x] becomes [xX] and [^x] becomes [^xX]. If newline-sensitive matching is specified, . and bracket expressions using ^ will never match the newline character (so that matches will never cross newlines unless the RE explicitly arranges it) and ^ and $ will match the empty string after and before a newline respectively, in addition to matching at beginning and end of string respectively. But the ARE escapes \A and \Z continue to match beginning or end of string only. If partial newline-sensitive matching is specified, this affects . and bracket expressions as with newline-sensitive matching, but not ^ and $. If inverse partial newline-sensitive matching is specified, this affects ^ and $ as with newline-sensitive matching, but not . and bracket expressions. This isn’t very useful but is provided for symmetry.
9.7.3.6. Limits and Compatibility No particular limit is imposed on the length of REs in this implementation. However, programs intended to be highly portable should not employ REs longer than 256 bytes, as a POSIX-compliant implementation can refuse to accept such REs. The only feature of AREs that is actually incompatible with POSIX EREs is that \ does not lose its special significance inside bracket expressions. All other ARE features use syntax which is illegal or has undefined or unspecified effects in POSIX EREs; the *** syntax of directors likewise is outside the POSIX syntax for both BREs and EREs.
204
Chapter 9. Functions and Operators Many of the ARE extensions are borrowed from Perl, but some have been changed to clean them up, and a few Perl extensions are not present. Incompatibilities of note include \b, \B, the lack of special treatment for a trailing newline, the addition of complemented bracket expressions to the things affected by newline-sensitive matching, the restrictions on parentheses and back references in lookahead constraints, and the longest/shortest-match (rather than first-match) matching semantics. Two significant incompatibilities exist between AREs and the ERE syntax recognized by pre-7.4 releases of PostgreSQL: •
In AREs, \ followed by an alphanumeric character is either an escape or an error, while in previous releases, it was just another way of writing the alphanumeric. This should not be much of a problem because there was no reason to write such a sequence in earlier releases.
•
In AREs, \ remains a special character within [], so a literal \ within a bracket expression must be written \\.
9.7.3.7. Basic Regular Expressions BREs differ from EREs in several respects. In BREs, |, +, and ? are ordinary characters and there is no equivalent for their functionality. The delimiters for bounds are \{ and \}, with { and } by themselves ordinary characters. The parentheses for nested subexpressions are \( and \), with ( and ) by themselves ordinary characters. ^ is an ordinary character except at the beginning of the RE or the beginning of a parenthesized subexpression, $ is an ordinary character except at the end of the RE or the end of a parenthesized subexpression, and * is an ordinary character if it appears at the beginning of the RE or the beginning of a parenthesized subexpression (after a possible leading ^). Finally, single-digit back references are available, and \< and \> are synonyms for [[::]] respectively; no other escapes are available in BREs.
9.8. Data Type Formatting Functions The PostgreSQL formatting functions provide a powerful set of tools for converting various data types (date/time, integer, floating point, numeric) to formatted strings and for converting from formatted strings to specific data types. Table 9-21 lists them. These functions all follow a common calling convention: the first argument is the value to be formatted and the second argument is a template that defines the output or input format. A single-argument to_timestamp function is also available; it accepts a double precision argument and converts from Unix epoch (seconds since 1970-01-01 00:00:00+00) to timestamp with time zone. (Integer Unix epochs are implicitly cast to double precision.) Table 9-21. Formatting Functions Function to_char(timestamp, text)
Return Type
Description
Example
text
convert time stamp to string
to_char(current_timestamp, ’HH12:MI:SS’)
205
Chapter 9. Functions and Operators Function
Return Type
Description
Example
to_char(interval,
text
convert interval to string
to_char(interval ’15h 2m 12s’, ’HH24:MI:SS’)
text
convert integer to string to_char(125,
text)
to_char(int, text)
’999’) to_char(double
text
convert real/double precision to string
to_char(125.8::real, ’999D9’)
text
convert numeric to string
to_char(-125.8, ’999D99S’)
convert string to date
to_date(’05 Dec 2000’, ’DD Mon YYYY’)
numeric
convert string to numeric
to_number(’12,454.8-’, ’99G999D9S’)
timestamp with time zone
convert string to time stamp
to_timestamp(’05 Dec 2000’, ’DD Mon YYYY’)
timestamp with time zone
convert Unix epoch to time stamp
to_timestamp(1284352323)
precision, text) to_char(numeric, text)
to_date(text, text) date
to_number(text, text)
to_timestamp(text, text)
to_timestamp(double precision)
In a to_char output template string, there are certain patterns that are recognized and replaced with appropriately-formatted data based on the given value. Any text that is not a template pattern is simply copied verbatim. Similarly, in an input template string (for the other functions), template patterns identify the values to be supplied by the input data string. Table 9-22 shows the template patterns available for formatting date and time values. Table 9-22. Template Patterns for Date/Time Formatting Pattern
Description
HH
hour of day (01-12)
HH12
hour of day (01-12)
HH24
hour of day (00-23)
MI
minute (00-59)
SS
second (00-59)
MS
millisecond (000-999)
US
microsecond (000000-999999)
SSSS
seconds past midnight (0-86399)
AM, am, PM or pm
meridiem indicator (without periods)
A.M., a.m., P.M. or p.m.
meridiem indicator (with periods)
Y,YYY
year (4 or more digits) with comma
YYYY
year (4 or more digits)
YYY
last 3 digits of year
YY
last 2 digits of year
Y
last digit of year
IYYY
ISO 8601 week-numbering year (4 or more digits)
IYY
last 3 digits of ISO 8601 week-numbering year
206
Chapter 9. Functions and Operators Pattern
Description
IY
last 2 digits of ISO 8601 week-numbering year
I
last digit of ISO 8601 week-numbering year
BC, bc, AD or ad
era indicator (without periods)
B.C., b.c., A.D. or a.d.
era indicator (with periods)
MONTH
full upper case month name (blank-padded to 9 chars)
Month
full capitalized month name (blank-padded to 9 chars)
month
full lower case month name (blank-padded to 9 chars)
MON
abbreviated upper case month name (3 chars in English, localized lengths vary)
Mon
abbreviated capitalized month name (3 chars in English, localized lengths vary)
mon
abbreviated lower case month name (3 chars in English, localized lengths vary)
MM
month number (01-12)
DAY
full upper case day name (blank-padded to 9 chars)
Day
full capitalized day name (blank-padded to 9 chars)
day
full lower case day name (blank-padded to 9 chars)
DY
abbreviated upper case day name (3 chars in English, localized lengths vary)
Dy
abbreviated capitalized day name (3 chars in English, localized lengths vary)
dy
abbreviated lower case day name (3 chars in English, localized lengths vary)
DDD
day of year (001-366)
IDDD
day of ISO 8601 week-numbering year (001-371; day 1 of the year is Monday of the first ISO week)
DD
day of month (01-31)
D
day of the week, Sunday (1) to Saturday (7)
ID
ISO 8601 day of the week, Monday (1) to Sunday (7)
W
week of month (1-5) (the first week starts on the first day of the month)
WW
week number of year (1-53) (the first week starts on the first day of the year)
IW
week number of ISO 8601 week-numbering year (01-53; the first Thursday of the year is in week 1)
207
Chapter 9. Functions and Operators Pattern
Description
CC
century (2 digits) (the twenty-first century starts on 2001-01-01)
J
Julian Day (integer days since November 24, 4714 BC at midnight UTC)
Q
quarter (ignored by to_date and to_timestamp)
RM
month in upper case Roman numerals (I-XII; I=January)
rm
month in lower case Roman numerals (i-xii; i=January)
TZ
upper case time-zone abbreviation (only supported in to_char)
tz
lower case time-zone abbreviation (only supported in to_char)
Modifiers can be applied to any template pattern to alter its behavior. For example, FMMonth is the Month pattern with the FM modifier. Table 9-23 shows the modifier patterns for date/time formatting. Table 9-23. Template Pattern Modifiers for Date/Time Formatting Modifier
Description
Example
FM prefix
fill mode (suppress leading zeroes and padding blanks)
FMMonth
TH suffix
upper case ordinal number suffix
DDTH, e.g., 12TH
th suffix
lower case ordinal number suffix
DDth, e.g., 12th
FX prefix
fixed format global option (see usage notes)
FX Month DD Day
TM prefix
translation mode (print TMMonth localized day and month names based on lc_time)
SP suffix
spell mode (not implemented)
DDSP
Usage notes for date/time formatting:
suppresses leading zeroes and trailing blanks that would otherwise be added to make the output of a pattern be fixed-width. In PostgreSQL, FM modifies only the next specification, while in Oracle FM affects all subsequent specifications, and repeated FM modifiers toggle fill mode on and off.
• FM
• TM
does not include trailing blanks.
and to_date skip multiple blank spaces in the input string unless the FX option is used. For example, to_timestamp(’2000 JUN’, ’YYYY MON’) works, but to_timestamp(’2000 JUN’, ’FXYYYY MON’) returns an error because to_timestamp expects one space only. FX must be specified as the first item in the template.
• to_timestamp
•
Ordinary text is allowed in to_char templates and will be output literally. You can put a substring in double quotes to force it to be interpreted as literal text even if it contains pattern key words. For
208
Chapter 9. Functions and Operators example, in ’"Hello Year "YYYY’, the YYYY will be replaced by the year data, but the single Y in Year will not be. In to_date, to_number, and to_timestamp, double-quoted strings skip the number of input characters contained in the string, e.g. "XX" skips two input characters. •
If you want to have a double quote in the output you must precede it with a backslash, for example ’\"YYYY Month\"’.
•
If the year format specification is less than four digits, e.g. YYY, and the supplied year is less than four digits, the year will be adjusted to be nearest to the year 2020, e.g. 95 becomes 1995.
•
The YYYY conversion from string to timestamp or date has a restriction when processing years with more than 4 digits. You must use some non-digit character or template after YYYY, otherwise the year is always interpreted as 4 digits. For example (with the year 20000): to_date(’200001131’, ’YYYYMMDD’) will be interpreted as a 4-digit year; instead use a non-digit separator after the year, like to_date(’20000-1131’, ’YYYY-MMDD’) or to_date(’20000Nov31’, ’YYYYMonDD’).
•
In conversions from string to timestamp or date, the CC (century) field is ignored if there is a YYY, YYYY or Y,YYY field. If CC is used with YY or Y then the year is computed as the year in the specified century. If the century is specified but the year is not, the first year of the century is assumed.
•
An ISO 8601 week-numbering date (as distinct from a Gregorian date) can be specified to to_timestamp and to_date in one of two ways: •
Year, week number, and weekday: for example to_date(’2006-42-4’, ’IYYY-IW-ID’) returns the date 2006-10-19. If you omit the weekday it is assumed to be 1 (Monday).
•
Year and day of year: for example to_date(’2006-291’, ’IYYY-IDDD’) also returns 2006-10-19.
Attempting to enter a date using a mixture of ISO 8601 week-numbering fields and Gregorian date fields is nonsensical, and will cause an error. In the context of an ISO 8601 week-numbering year, the concept of a “month” or “day of month” has no meaning. In the context of a Gregorian year, the ISO week has no meaning.
Caution While to_date will reject a mixture of Gregorian and ISO week-numbering date fields, to_char will not, since output format specifications like YYYY-MM-DD (IYYY-IDDD) can be useful. But avoid writing something like IYYY-MM-DD; that would yield surprising results near the start of the year. (See Section 9.9.1 for more information.)
•
In a conversion from string to timestamp, millisecond (MS) or microsecond (US) values are used as the seconds digits after the decimal point. For example to_timestamp(’12:3’, ’SS:MS’) is not 3 milliseconds, but 300, because the conversion counts it as 12 + 0.3 seconds. This means for the format SS:MS, the input values 12:3, 12:30, and 12:300 specify the same number of milliseconds. To get three milliseconds, one must use 12:003, which the conversion counts as 12 + 0.003 = 12.003 seconds. Here
is a more complex example: to_timestamp(’15:12:02.020.001230’, ’HH24:MI:SS.MS.US’) is 15 hours, 12 minutes, and 2 seconds + 20 milliseconds + 1230 microseconds = 2.021230 seconds.
209
Chapter 9. Functions and Operators • to_char(..., ’ID’)’s day of the week numbering matches the extract(isodow from ...)
function, but to_char(..., ’D’)’s does not match extract(dow from ...)’s day numbering. formats HH and HH12 as shown on a 12-hour clock, i.e. zero hours and 36 hours output as 12, while HH24 outputs the full hour value, which can exceed 23 for intervals.
• to_char(interval)
Table 9-24 shows the template patterns available for formatting numeric values. Table 9-24. Template Patterns for Numeric Formatting Pattern
Description
9
value with the specified number of digits
0
value with leading zeros
. (period)
decimal point
, (comma)
group (thousand) separator
PR
negative value in angle brackets
S
sign anchored to number (uses locale)
L
currency symbol (uses locale)
D
decimal point (uses locale)
G
group separator (uses locale)
MI
minus sign in specified position (if number < 0)
PL
plus sign in specified position (if number > 0)
SG
plus/minus sign in specified position
RN
Roman numeral (input between 1 and 3999)
TH or th
ordinal number suffix
V
shift specified number of digits (see notes)
EEEE
exponent for scientific notation
Usage notes for numeric formatting:
•
A sign formatted using SG, PL, or MI is not anchored to the number; for example, to_char(-12, ’MI9999’) produces ’- 12’ but to_char(-12, ’S9999’) produces ’ -12’. The Oracle implementation does not allow the use of MI before 9, but rather requires that 9 precede MI. results in a value with the same number of digits as there are 9s. If a digit is not available it outputs a space.
• 9
• TH
does not convert values less than zero and does not convert fractional numbers.
• PL, SG,
and TH are PostgreSQL extensions.
effectively multiplies the input values by 10^n, where n is the number of digits following V. to_char does not support the use of V combined with a decimal point (e.g., 99.9V99 is not allowed).
• V
(scientific notation) cannot be used in combination with any of the other formatting patterns or modifiers other than digit and decimal point patterns, and must be at the end of the format string (e.g., 9.99EEEE is a valid pattern).
• EEEE
210
Chapter 9. Functions and Operators Certain modifiers can be applied to any template pattern to alter its behavior. For example, FM9999 is the 9999 pattern with the FM modifier. Table 9-25 shows the modifier patterns for numeric formatting. Table 9-25. Template Pattern Modifiers for Numeric Formatting Modifier
Description
Example
FM prefix
fill mode (suppress leading zeroes and padding blanks)
FM9999
TH suffix
upper case ordinal number suffix
999TH
th suffix
lower case ordinal number suffix
999th
Table 9-26 shows some examples of the use of the to_char function. Table 9-26. to_char Examples Expression
9.9. Date/Time Functions and Operators Table 9-28 shows the available functions for date/time value processing, with details appearing in the following subsections. Table 9-27 illustrates the behaviors of the basic arithmetic operators (+, *, etc.). For formatting functions, refer to Section 9.8. You should be familiar with the background information on date/time data types from Section 8.5. All the functions and operators described below that take time or timestamp inputs actually come in two variants: one that takes time with time zone or timestamp with time zone, and one that takes time without time zone or timestamp without time zone. For brevity, these variants are not shown separately. Also, the + and * operators come in commutative pairs (for example both date + integer and integer + date); we show only one of each such pair. Table 9-27. Date/Time Operators Operator
extract(hour from timestamp ’2001-02-16 20:38:40’)
20
double precision
Get subfield; see Section 9.9.1
extract(month from interval ’2 years 3 months’)
3
boolean
Test for finite date isfinite(date (not +/-infinity) ’2001-02-16’)
date_trunc(text, timestamp)
extract(field
from timestamp)
extract(field from interval)
isfinite(date)
Result
double precision
timestamp)
date_part(text,
Example
true
isfinite(timestampboolean )
Test for finite time isfinite(timestamp true stamp (not ’2001-02-16 +/-infinity) 21:28:30’)
isfinite(interval)boolean
Test for finite interval
isfinite(interval true ’4 hours’)
Adjust interval so 30-day time periods are represented as months
justify_days(interval 1 mon 5 days ’35 days’)
Adjust interval so 24-hour time periods are represented as days
justify_hours(interval 1 day ’27 hours’) 03:00:00
interval justify_days(interval)
interval justify_hours(interval)
214
Chapter 9. Functions and Operators Function
Return Type
Description
Example
interval
Adjust interval using
justify_interval(interval 29 days ’1 mon -1 23:00:00 hour’)
justify_interval(interval)
justify_days
Result
and justify_hours,
with additional sign adjustments localtime
time
Current time of day; see Section 9.9.4
localtimestamp timestamp
Current date and time (start of current transaction); see Section 9.9.4
timestamp with time zone
Current date and time (start of current transaction); see Section 9.9.4
timestamp statement_timestamp() with time zone
Current date and time (start of current statement); see Section 9.9.4
now()
timeofday()
text
Current date and time (like clock_timestamp, but as a text
string); see Section 9.9.4 timestamp transaction_timestamp() with time zone
Current date and time (start of current transaction); see Section 9.9.4
In addition to these functions, the SQL OVERLAPS operator is supported: (start1, end1) OVERLAPS (start2, end2) (start1, length1) OVERLAPS (start2, length2)
This expression yields true when two time periods (defined by their endpoints) overlap, false when they do not overlap. The endpoints can be specified as pairs of dates, times, or time stamps; or as a date, time, or time stamp followed by an interval. When a pair of values is provided, either the start or the end can be written first; OVERLAPS automatically takes the earlier value of the pair as the start. Each time period is considered to represent the half-open interval start
Contains?
circle ’((0,0),2)’ @> point ’(1,1)’
= inet ’192.168.1/24’
~
bitwise NOT
~ inet ’192.168.1.6’
&
bitwise AND
inet ’192.168.1.6’ & inet ’0.0.0.255’
|
bitwise OR
inet ’192.168.1.6’ | inet ’0.0.0.255’
+
addition
inet ’192.168.1.6’ + 25
-
subtraction
inet ’192.168.1.43’ 36
-
subtraction
inet ’192.168.1.43’ inet ’192.168.1.19’
Table 9-35 shows the functions available for use with the cidr and inet types. The abbrev, host, and text functions are primarily intended to offer alternative display formats. Table 9-35. cidr and inet Functions Function
Return Type
Description
Example
text
abbreviated display format as text
abbrev(inet 10.1.0.0/16 ’10.1.0.0/16’)
text
abbreviated display format as text
abbrev(cidr 10.1/16 ’10.1.0.0/16’)
inet
broadcast address for network
broadcast(’192.168.1.5/24’) 192.168.1.255/24
family(inet)
int
extract family of address; 4 for IPv4, 6 for IPv6
family(’::1’)
host(inet)
text
extract IP address as text
host(’192.168.1.5/24’) 192.168.1.5
hostmask(inet)
inet
construct host mask for network
hostmask(’192.168.23.20/30’) 0.0.0.3
masklen(inet)
int
extract netmask length
masklen(’192.168.1.5/24’) 24
netmask(inet)
inet
construct netmask netmask(’192.168.1.5/24’) 255.255.255.0 for network
abbrev(inet)
abbrev(cidr)
broadcast(inet)
Result
6
229
Chapter 9. Functions and Operators Function network(inet)
Return Type
Description
Example
cidr
extract network part of address
network(’192.168.1.5/24’) 192.168.1.0/24
inet
set netmask length set_masklen(’192.168.1.5/24’, 192.168.1.5/16 for inet value 16)
set_masklen(inet,
Result
int)
set_masklen(cidr, cidr int)
text(inet)
text
set netmask length set_masklen(’192.168.1.0/24’::cidr, 192.168.0.0/16 for cidr value 16) extract IP address and netmask length as text
text(inet 192.168.1.5/32 ’192.168.1.5’)
Any cidr value can be cast to inet implicitly or explicitly; therefore, the functions shown above as operating on inet also work on cidr values. (Where there are separate functions for inet and cidr, it is because the behavior should be different for the two cases.) Also, it is permitted to cast an inet value to cidr. When this is done, any bits to the right of the netmask are silently zeroed to create a valid cidr value. In addition, you can cast a text value to inet or cidr using normal casting syntax: for example, inet(expression) or colname::cidr. Table 9-36 shows the functions available for use with the macaddr type. The function trunc(macaddr) returns a MAC address with the last 3 bytes set to zero. This can be used to associate the remaining prefix with a manufacturer. Table 9-36. macaddr Functions Function trunc(macaddr)
The macaddr type also supports the standard relational operators (>,
tsquery contains
’cat’::tsquery @> f ’cat & rat’::tsquery
another ? tsquery is contained
Content of other types will be formatted into valid XML character data. This means in particular that the characters , and & will be converted to entities. Binary data (data type bytea) will be represented in base64 or hex encoding, depending on the setting of the configuration parameter xmlbinary. The particular behavior for individual data types is expected to evolve in order to align the SQL and PostgreSQL data types with the XML Schema specification, at which point a more precise description will appear.
The xmlforest expression produces an XML forest (sequence) of elements using the given names and content. Examples: SELECT xmlforest(’abc’ AS foo, 123 AS bar); xmlforest -----------------------------abc123
SELECT xmlforest(table_name, column_name) FROM information_schema.columns WHERE table_schema = ’pg_catalog’;
As seen in the second example, the element name can be omitted if the content value is a column reference, in which case the column name is used by default. Otherwise, a name must be specified. Element names that are not valid XML names are escaped as shown for xmlelement above. Similarly, content data is escaped to make valid XML content, unless it is already of type xml. Note that XML forests are not valid XML documents if they consist of more than one element, so it might be useful to wrap xmlforest expressions in xmlelement.
9.14.1.5. xmlpi xmlpi(name target [, content])
The xmlpi expression creates an XML processing instruction. The content, if present, must not contain the character sequence ?>. Example:
9.14.1.6. xmlroot xmlroot(xml, version text | no value [, standalone yes|no|no value])
The xmlroot expression alters the properties of the root node of an XML value. If a version is specified, it replaces the value in the root node’s version declaration; if a standalone setting is specified, it replaces the value in the root node’s standalone declaration. SELECT xmlroot(xmlparse(document ’abc’), version ’1.0’, standalone yes); xmlroot --------------------------------------- abc
9.14.1.7. xmlagg xmlagg(xml)
The function xmlagg is, unlike the other functions described here, an aggregate function. It concatenates the input values to the aggregate function call, much like xmlconcat does, except that concatenation occurs across rows rather than across expressions in a single row. See Section 9.20 for additional information about aggregate functions. Example: CREATE INSERT INSERT SELECT
TABLE test (y int, x xml); INTO test VALUES (1, ’abc’); INTO test VALUES (2, ’’); xmlagg(x) FROM test; xmlagg ---------------------abc
To determine the order of the concatenation, an ORDER BY clause may be added to the aggregate call as described in Section 4.2.7. For example: SELECT xmlagg(x ORDER BY y DESC) FROM test; xmlagg ---------------------abc
238
Chapter 9. Functions and Operators
The following non-standard approach used to be recommended in previous versions, and may still be useful in specific cases: SELECT xmlagg(x) FROM (SELECT * FROM test ORDER BY y DESC) AS tab; xmlagg ---------------------abc
9.14.2. XML Predicates The expressions described in this section check properties of xml values.
9.14.2.1. IS DOCUMENT xml IS DOCUMENT
The expression IS DOCUMENT returns true if the argument XML value is a proper XML document, false if it is not (that is, it is a content fragment), or null if the argument is null. See Section 8.13 about the difference between documents and content fragments.
9.14.2.2. XMLEXISTS XMLEXISTS(text PASSING [BY REF] xml [BY REF])
The function xmlexists returns true if the XPath expression in the first argument returns any nodes, and false otherwise. (If either argument is null, the result is null.) Example:
The BY REF clauses have no effect in PostgreSQL, but are allowed for SQL conformance and compatibility with other implementations. Per SQL standard, the first BY REF is required, the second is optional. Also note that the SQL standard specifies the xmlexists construct to take an XQuery expression as first argument, but PostgreSQL currently only supports XPath, which is a subset of XQuery.
Chapter 9. Functions and Operators xml_is_well_formed_content(text)
These functions check whether a text string is well-formed XML, returning a Boolean result. xml_is_well_formed_document checks for a well-formed document, while xml_is_well_formed_content checks for well-formed content. xml_is_well_formed does the former if the xmloption configuration parameter is set to DOCUMENT, or the latter if it is set to CONTENT. This means that xml_is_well_formed is useful for seeing whether a simple cast to type xml will succeed, whereas the other two functions are useful for seeing whether the corresponding variants of XMLPARSE will succeed. Examples: SET xmloption TO DOCUMENT; SELECT xml_is_well_formed(’’); xml_is_well_formed -------------------f (1 row) SELECT xml_is_well_formed(’’); xml_is_well_formed -------------------t (1 row) SET xmloption TO CONTENT; SELECT xml_is_well_formed(’abc’); xml_is_well_formed -------------------t (1 row)
SELECT xml_is_well_formed_document(’bar
int
Get JSON array element as text
’[1,2,3]’::json->>2
->>
text
Get JSON object field as text
’{"a":1,"b":2}’::json->>’b’
#>
array of text
Get JSON object at specified path
’{"a":[1,2,3],"b":[4,5,6]}’::jso
#>>
array of text
Get JSON object at specified path as text
’{"a":[1,2,3],"b":[4,5,6]}’::jso
Table 9-41 shows the functions that are available for creating and manipulating JSON (see Section 8.14) data. Table 9-41. JSON Support Functions Function
Return Type
Description
Example
Example Result
245
Chapter 9. Functions and Operators Function
Return Type
Description
Example
Example Result
json array_to_json(anyarray [, pretty_bool])
Returns the array array_to_json(’{{1,5},{99,100}}’::int[]) [[1,5],[99,100]] as JSON. A PostgreSQL multidimensional array becomes a JSON array of arrays. Line feeds will be added between dimension 1 elements if pretty_bool is true.
json row_to_json(record [, pretty_bool])
Returns the row row_to_json(row(1,’foo’)) {"f1":1,"f2":"foo"} as JSON. Line feeds will be added between level 1 elements if pretty_bool is true.
json to_json(anyelement)
Returns the value to_json(’Fred as JSON. If the said data type is not "Hi."’::text) built in, and there is a cast from the type to json, the cast function will be used to perform the conversion. Otherwise, for any value other than a number, a Boolean, or a null value, the text representation will be used, escaped and quoted so that it is legal JSON.
int json_array_length(json)
Returns the number of elements in the outermost JSON array.
SETOF key json_each(json)text, value json
Expands the select * from key | value outermost JSON json_each(’{"a":"foo", -----+------object into a set of "b":"bar"}’) a | "foo" key/value pairs. b | "bar"
"Fred said \"Hi.\""
json_array_length(’[1,2,3,{"f1":1,"f2":[5,6 5
246
Chapter 9. Functions and Operators Function
Return Type
Description
Example
Example Result
SETOF key json_each_text(from_json text, value json) text
Expands the select * from key | value outermost JSON json_each_text(’{"a":"foo", -----+------object into a set of "b":"bar"}’) a | foo key/value pairs. b | bar The returned value will be of type text.
Returns set of json_object_keys(’{"f1":"abc","f2":{"f3":"a json_object_keys keys in the JSON "f4":"b"}}’) -----------------object. Only the f1 f2 “outer” object will be displayed.
Expands the select * from a | b object in json_populate_record(null::x, ---+--from_json to a ’{"a":1,"b":2}’) 1 | 2 row whose columns match the record type defined by base. Conversion will be best effort; columns in base with no corresponding key in from_json will be left null. If a column is specified more than once, the last value is used.
Expands the select * from a | b outermost set of json_populate_recordset(null::x, ---+--objects in ’[{"a":1,"b":2},{"a":3,"b":4}]’) 1 | 2 3 | 4 from_json to a set whose columns match the record type defined by base. Conversion will be best effort; columns in base with no corresponding key in from_json will be left null. If a column is specified more than once, the last value is used.
SETOF json json_array_elements(json)
Expands a JSON array to a set of JSON elements.
json_array_elements(’[1,true, value [2,false]]’) ----------1 true [2,false]
Note: The json functions and operators can impose stricter validity requirements than the type’s input functions. In particular, they check much more closely that any use of Unicode surrogate pairs to designate characters outside the Unicode Basic Multilingual Plane is correct.
Note: Many of these functions and operators will convert Unicode escapes in the JSON text to the appropriate UTF8 character when the database encoding is UTF8. In other encodings the escape sequence must be for an ASCII character, and any other code point in a Unicode escape sequence will result in an error. In general, it is best to avoid mixing Unicode escapes in JSON with a non-UTF8 database encoding, if possible.
Note: The hstore extension has a cast from hstore to json, so that converted hstore values are represented as JSON objects, not as string values.
See also Section 9.20 about the aggregate function json_agg which aggregates record values as JSON efficiently.
9.16. Sequence Manipulation Functions This section describes functions for operating on sequence objects, also called sequence generators or
248
Chapter 9. Functions and Operators just sequences. Sequence objects are special single-row tables created with CREATE SEQUENCE. Sequence objects are commonly used to generate unique identifiers for rows of a table. The sequence functions, listed in Table 9-42, provide simple, multiuser-safe methods for obtaining successive sequence values from sequence objects. Table 9-42. Sequence Functions Function
Return Type
Description
currval(regclass)
bigint
Return value most recently obtained with nextval for specified sequence
lastval()
bigint
Return value most recently obtained with nextval for any sequence
nextval(regclass)
bigint
Advance sequence and return new value
setval(regclass, bigint)
bigint
Set sequence’s current value
setval(regclass, bigint,
bigint
Set sequence’s current value and is_called flag
boolean)
The sequence to be operated on by a sequence function is specified by a regclass argument, which is simply the OID of the sequence in the pg_class system catalog. You do not have to look up the OID by hand, however, since the regclass data type’s input converter will do the work for you. Just write the sequence name enclosed in single quotes so that it looks like a literal constant. For compatibility with the handling of ordinary SQL names, the string will be converted to lower case unless it contains double quotes around the sequence name. Thus: nextval(’foo’) nextval(’FOO’) nextval(’"Foo"’)
operates on sequence foo operates on sequence foo operates on sequence Foo
The sequence name can be schema-qualified if necessary: nextval(’myschema.foo’) nextval(’"myschema".foo’) nextval(’foo’)
operates on myschema.foo same as above searches search path for foo
See Section 8.18 for more information about regclass. Note: Before PostgreSQL 8.1, the arguments of the sequence functions were of type text, not regclass, and the above-described conversion from a text string to an OID value would happen at run time during each call. For backward compatibility, this facility still exists, but internally it is now handled as an implicit coercion from text to regclass before the function is invoked. When you write the argument of a sequence function as an unadorned literal string, it becomes a constant of type regclass. Since this is really just an OID, it will track the originally identified sequence despite later renaming, schema reassignment, etc. This “early binding” behavior is usually desirable for sequence references in column defaults and views. But sometimes you might want “late binding” where the sequence reference is resolved at run time. To get late-binding behavior, force the constant to be stored as a text constant instead of regclass: nextval(’foo’::text)
foo is looked up at runtime
Note that late binding was the only behavior supported in PostgreSQL releases before 8.1, so you might need to do this to preserve the semantics of old applications.
249
Chapter 9. Functions and Operators Of course, the argument of a sequence function can be an expression as well as a constant. If it is a text expression then the implicit coercion will result in a run-time lookup.
The available sequence functions are:
nextval
Advance the sequence object to its next value and return that value. This is done atomically: even if multiple sessions execute nextval concurrently, each will safely receive a distinct sequence value. If a sequence object has been created with default parameters, successive nextval calls will return successive values beginning with 1. Other behaviors can be obtained by using special parameters in the CREATE SEQUENCE command; see its command reference page for more information. Important: To avoid blocking concurrent transactions that obtain numbers from the same sequence, a nextval operation is never rolled back; that is, once a value has been fetched it is considered used, even if the transaction that did the nextval later aborts. This means that aborted transactions might leave unused “holes” in the sequence of assigned values.
currval
Return the value most recently obtained by nextval for this sequence in the current session. (An error is reported if nextval has never been called for this sequence in this session.) Because this is returning a session-local value, it gives a predictable answer whether or not other sessions have executed nextval since the current session did. lastval
Return the value most recently returned by nextval in the current session. This function is identical to currval, except that instead of taking the sequence name as an argument it fetches the value of the last sequence used by nextval in the current session. It is an error to call lastval if nextval has not yet been called in the current session. setval
Reset the sequence object’s counter value. The two-parameter form sets the sequence’s last_value field to the specified value and sets its is_called field to true, meaning that the next nextval will advance the sequence before returning a value. The value reported by currval is also set to the specified value. In the three-parameter form, is_called can be set to either true or false. true has the same effect as the two-parameter form. If it is set to false, the next nextval will return exactly the specified value, and sequence advancement commences with the following nextval. Furthermore, the value reported by currval is not changed in this case (this is a change from pre-8.3 behavior). For example, SELECT setval(’foo’, 42); Next nextval will return 43 SELECT setval(’foo’, 42, true); Same as above SELECT setval(’foo’, 42, false); Next nextval will return 42 The result returned by setval is just the value of its second argument. Important: Because sequences are non-transactional, changes made by setval are not undone if the transaction rolls back.
250
Chapter 9. Functions and Operators
9.17. Conditional Expressions This section describes the SQL-compliant conditional expressions available in PostgreSQL. Tip: If your needs go beyond the capabilities of these conditional expressions, you might want to consider writing a stored procedure in a more expressive programming language.
9.17.1. CASE The SQL CASE expression is a generic conditional expression, similar to if/else statements in other programming languages: CASE WHEN condition THEN result [WHEN ...] [ELSE result] END CASE clauses can be used wherever an expression is valid. Each condition is an expression that returns a boolean result. If the condition’s result is true, the value of the CASE expression is the result that follows the condition, and the remainder of the CASE expression is not processed. If the condition’s result is not true, any subsequent WHEN clauses are examined in the same manner. If no WHEN condition yields true, the value of the CASE expression is the result of the ELSE clause. If the ELSE clause is omitted and no condition is true, the result is null.
An example: SELECT * FROM test; a --1 2 3
SELECT a, CASE WHEN a=1 THEN ’one’ WHEN a=2 THEN ’two’ ELSE ’other’ END FROM test; a | case ---+------1 | one 2 | two 3 | other
251
Chapter 9. Functions and Operators The data types of all the result expressions must be convertible to a single output type. See Section 10.5 for more details. There is a “simple” form of CASE expression that is a variant of the general form above: CASE expression WHEN value THEN result [WHEN ...] [ELSE result] END
The first expression is computed, then compared to each of the value expressions in the WHEN clauses until one is found that is equal to it. If no match is found, the result of the ELSE clause (or a null value) is returned. This is similar to the switch statement in C. The example above can be written using the simple CASE syntax: SELECT a, CASE a WHEN 1 THEN ’one’ WHEN 2 THEN ’two’ ELSE ’other’ END FROM test; a | case ---+------1 | one 2 | two 3 | other
A CASE expression does not evaluate any subexpressions that are not needed to determine the result. For example, this is a possible way of avoiding a division-by-zero failure: SELECT ... WHERE CASE WHEN x 0 THEN y/x > 1.5 ELSE false END;
Note: As described in Section 4.2.14, there are various situations in which subexpressions of an expression are evaluated at different times, so that the principle that “CASE evaluates only necessary subexpressions” is not ironclad. For example a constant 1/0 subexpression will usually result in a division-by-zero failure at planning time, even if it’s within a CASE arm that would never be entered at run time.
9.17.2. COALESCE COALESCE(value [, ...])
The COALESCE function returns the first of its arguments that is not null. Null is returned only if all arguments are null. It is often used to substitute a default value for null values when data is retrieved for display, for example: SELECT COALESCE(description, short_description, ’(none)’) ...
252
Chapter 9. Functions and Operators This returns description if it is not null, otherwise short_description if it is not null, otherwise (none). Like a CASE expression, COALESCE only evaluates the arguments that are needed to determine the result; that is, arguments to the right of the first non-null argument are not evaluated. This SQLstandard function provides capabilities similar to NVL and IFNULL, which are used in some other database systems.
9.17.3. NULLIF NULLIF(value1, value2)
The NULLIF function returns a null value if value1 equals value2; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE example given above: SELECT NULLIF(value, ’(none)’) ...
In this example, if value is (none), null is returned, otherwise the value of value is returned.
9.17.4. GREATEST and LEAST GREATEST(value [, ...]) LEAST(value [, ...])
The GREATEST and LEAST functions select the largest or smallest value from a list of any number of expressions. The expressions must all be convertible to a common data type, which will be the type of the result (see Section 10.5 for details). NULL values in the list are ignored. The result will be NULL only if all the expressions evaluate to NULL. Note that GREATEST and LEAST are not in the SQL standard, but are a common extension. Some other databases make them return NULL if any argument is NULL, rather than only when all are NULL.
9.18. Array Functions and Operators Table 9-43 shows the operators available for array types. Table 9-43. Array Operators Operator
Description
Example
Result
=
equal
ARRAY[1.1,2.1,3.1]::int[] t = ARRAY[1,2,3]
not equal
ARRAY[1,2,3] ARRAY[1,2,4]
t
greater than
ARRAY[1,4,3] > ARRAY[1,2,4]
t
= ARRAY[1,4,3]
t
@>
contains
ARRAY[1,4,3] @> ARRAY[3,1]
t
contains range
int4range(2,4) @> t int4range(2,3)
t
256
Chapter 9. Functions and Operators Operator
Description
Example
Result
@>
contains element
’[2011-01-01,2011-03-01)’::tsrange t @> ’2011-01-10’::timestamp
5000.00; This is not as efficient as a partial index on the amount column would be, since the system has to scan
the entire index. Yet, if there are relatively few unbilled orders, using this partial index just to find the unbilled orders could be a win. Note that this query cannot use this index: SELECT * FROM orders WHERE order_nr = 3501;
The order 3501 might be among the billed or unbilled orders.
Example 11-2 also illustrates that the indexed column and the column used in the predicate do not need to match. PostgreSQL supports partial indexes with arbitrary predicates, so long as only columns of the table being indexed are involved. However, keep in mind that the predicate must match the conditions used in the queries that are supposed to benefit from the index. To be precise, a partial index can be used in a query only if the system can recognize that the WHERE condition of the query mathematically implies the predicate of the index. PostgreSQL does not have a sophisticated theorem prover that can recognize mathematically equivalent expressions that are written in different forms. (Not only is such a general theorem prover extremely difficult to create, it would probably be too slow to be of any real use.) The system can recognize simple inequality implications, for example “x < 1” implies “x < 2”; otherwise the predicate condition must exactly match part of the query’s WHERE condition or the index will not be recognized as usable. Matching takes place at query planning time, not at run time. As a result, parameterized query clauses do not work with a partial index. For example a prepared query with a parameter might specify “x < ?” which will never imply “x < 2” for all possible values of the parameter. A third possible use for partial indexes does not require the index to be used in queries at all. The idea here is to create a unique index over a subset of a table, as in Example 11-3. This enforces uniqueness among the rows that satisfy the index predicate, without constraining those that do not. Example 11-3. Setting up a Partial Unique Index Suppose that we have a table describing test outcomes. We wish to ensure that there is only one “successful” entry for a given subject and target combination, but there might be any number of “unsuccessful” entries. Here is one way to do it: CREATE TABLE tests ( subject text, target text, success boolean, ... ); CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target) WHERE success;
This is a particularly efficient approach when there are few successful tests and many unsuccessful ones.
315
Chapter 11. Indexes
Finally, a partial index can also be used to override the system’s query plan choices. Also, data sets with peculiar distributions might cause the system to use an index when it really should not. In that case the index can be set up so that it is not available for the offending query. Normally, PostgreSQL makes reasonable choices about index usage (e.g., it avoids them when retrieving common values, so the earlier example really only saves index size, it is not required to avoid index usage), and grossly incorrect plan choices are cause for a bug report. Keep in mind that setting up a partial index indicates that you know at least as much as the query planner knows, in particular you know when an index might be profitable. Forming this knowledge requires experience and understanding of how indexes in PostgreSQL work. In most cases, the advantage of a partial index over a regular index will be minimal. More information about partial indexes can be found in The case for partial indexes , Partial indexing in POSTGRES: research project, and Generalized Partial Indexes (cached version) .
11.9. Operator Classes and Operator Families An index definition can specify an operator class for each column of an index. CREATE INDEX name ON table (column opclass [sort options] [, ...]);
The operator class identifies the operators to be used by the index for that column. For example, a Btree index on the type int4 would use the int4_ops class; this operator class includes comparison functions for values of type int4. In practice the default operator class for the column’s data type is usually sufficient. The main reason for having operator classes is that for some data types, there could be more than one meaningful index behavior. For example, we might want to sort a complexnumber data type either by absolute value or by real part. We could do this by defining two operator classes for the data type and then selecting the proper class when making an index. The operator class determines the basic sort ordering (which can then be modified by adding sort options COLLATE, ASC/DESC and/or NULLS FIRST/NULLS LAST). There are also some built-in operator classes besides the default ones: •
The
operator classes text_pattern_ops, varchar_pattern_ops, and bpchar_pattern_ops support B-tree indexes on the types text, varchar, and char respectively. The difference from the default operator classes is that the values are compared strictly character by character rather than according to the locale-specific collation rules. This makes these operator classes suitable for use by queries involving pattern matching expressions (LIKE or POSIX regular expressions) when the database does not use the standard “C” locale. As an example, you might index a varchar column like this: CREATE INDEX test_index ON test_table (col varchar_pattern_ops);
Note that you should also create an index with the default operator class if you want queries involving ordinary = comparisons to use an index. Such queries cannot use the xxx _pattern_ops operator classes. (Ordinary equality comparisons can use these operator classes, however.) It is possible to create multiple indexes on the same column with different operator classes. If you do use the C locale, you do not need the xxx _pattern_ops operator classes, because an index with the default operator class is usable for pattern-matching queries in the C locale.
316
Chapter 11. Indexes The following query shows all defined operator classes: SELECT am.amname AS index_method, opc.opcname AS opclass_name FROM pg_am am, pg_opclass opc WHERE opc.opcmethod = am.oid ORDER BY index_method, opclass_name;
An operator class is actually just a subset of a larger structure called an operator family. In cases where several data types have similar behaviors, it is frequently useful to define cross-data-type operators and allow these to work with indexes. To do this, the operator classes for each of the types must be grouped into the same operator family. The cross-type operators are members of the family, but are not associated with any single class within the family. This query shows all defined operator families and all the operators included in each family: SELECT am.amname AS index_method, opf.opfname AS opfamily_name, amop.amopopr::regoperator AS opfamily_operator FROM pg_am am, pg_opfamily opf, pg_amop amop WHERE opf.opfmethod = am.oid AND amop.amopfamily = opf.oid ORDER BY index_method, opfamily_name, opfamily_operator;
11.10. Indexes and Collations An index can support only one collation per index column. If multiple collations are of interest, multiple indexes may be needed. Consider these statements: CREATE TABLE test1c ( id integer, content varchar COLLATE "x" ); CREATE INDEX test1c_content_index ON test1c (content);
The index automatically uses the collation of the underlying column. So a query of the form SELECT * FROM test1c WHERE content > constant;
could use the index, because the comparison will by default use the collation of the column. However, this index cannot accelerate queries that involve some other collation. So if queries of the form, say, SELECT * FROM test1c WHERE content > constant COLLATE "y";
are also of interest, an additional index could be created that supports the "y" collation, like this: CREATE INDEX test1c_content_y_index ON test1c (content COLLATE "y");
317
Chapter 11. Indexes
11.11. Examining Index Usage Although indexes in PostgreSQL do not need maintenance or tuning, it is still important to check which indexes are actually used by the real-life query workload. Examining index usage for an individual query is done with the EXPLAIN command; its application for this purpose is illustrated in Section 14.1. It is also possible to gather overall statistics about index usage in a running server, as described in Section 27.2. It is difficult to formulate a general procedure for determining which indexes to create. There are a number of typical cases that have been shown in the examples throughout the previous sections. A good deal of experimentation is often necessary. The rest of this section gives some tips for that: •
Always run ANALYZE first. This command collects statistics about the distribution of the values in the table. This information is required to estimate the number of rows returned by a query, which is needed by the planner to assign realistic costs to each possible query plan. In absence of any real statistics, some default values are assumed, which are almost certain to be inaccurate. Examining an application’s index usage without having run ANALYZE is therefore a lost cause. See Section 23.1.3 and Section 23.1.6 for more information.
•
Use real data for experimentation. Using test data for setting up indexes will tell you what indexes you need for the test data, but that is all. It is especially fatal to use very small test data sets. While selecting 1000 out of 100000 rows could be a candidate for an index, selecting 1 out of 100 rows will hardly be, because the 100 rows probably fit within a single disk page, and there is no plan that can beat sequentially fetching 1 disk page. Also be careful when making up test data, which is often unavoidable when the application is not yet in production. Values that are very similar, completely random, or inserted in sorted order will skew the statistics away from the distribution that real data would have.
•
When indexes are not used, it can be useful for testing to force their use. There are run-time parameters that can turn off various plan types (see Section 18.7.1). For instance, turning off sequential scans (enable_seqscan) and nested-loop joins (enable_nestloop), which are the most basic plans, will force the system to use a different plan. If the system still chooses a sequential scan or nested-loop join then there is probably a more fundamental reason why the index is not being used; for example, the query condition does not match the index. (What kind of query can use what kind of index is explained in the previous sections.)
•
If forcing index usage does use the index, then there are two possibilities: Either the system is right and using the index is indeed not appropriate, or the cost estimates of the query plans are not reflecting reality. So you should time your query with and without indexes. The EXPLAIN ANALYZE command can be useful here.
•
If it turns out that the cost estimates are wrong, there are, again, two possibilities. The total cost is computed from the per-row costs of each plan node times the selectivity estimate of the plan node. The costs estimated for the plan nodes can be adjusted via run-time parameters (described in Section 18.7.2). An inaccurate selectivity estimate is due to insufficient statistics. It might be possible to improve this by tuning the statistics-gathering parameters (see ALTER TABLE). If you do not succeed in adjusting the costs to be more appropriate, then you might have to resort to forcing index usage explicitly. You might also want to contact the PostgreSQL developers to examine the issue.
318
Chapter 12. Full Text Search 12.1. Introduction Full Text Searching (or just text search) provides the capability to identify natural-language documents that satisfy a query, and optionally to sort them by relevance to the query. The most common type of search is to find all documents containing given query terms and return them in order of their similarity to the query. Notions of query and similarity are very flexible and depend on the specific application. The simplest search considers query as a set of words and similarity as the frequency of query words in the document. Textual search operators have existed in databases for years. PostgreSQL has ~, ~*, LIKE, and ILIKE operators for textual data types, but they lack many essential properties required by modern information systems: There is no linguistic support, even for English. Regular expressions are not sufficient because they cannot easily handle derived words, e.g., satisfies and satisfy. You might miss documents that contain satisfies, although you probably would like to find them when searching for satisfy. It is possible to use OR to search for multiple derived forms, but this is tedious and error-prone (some words can have several thousand derivatives). • They provide no ordering (ranking) of search results, which makes them ineffective when thousands of matching documents are found. • They tend to be slow because there is no index support, so they must process all documents for every search.
•
Full text indexing allows documents to be preprocessed and an index saved for later rapid searching. Preprocessing includes: Parsing documents into tokens. It is useful to identify various classes of tokens, e.g., numbers, words, complex words, email addresses, so that they can be processed differently. In principle token classes depend on the specific application, but for most purposes it is adequate to use a predefined set of classes. PostgreSQL uses a parser to perform this step. A standard parser is provided, and custom parsers can be created for specific needs. Converting tokens into lexemes. A lexeme is a string, just like a token, but it has been normalized so that different forms of the same word are made alike. For example, normalization almost always includes folding upper-case letters to lower-case, and often involves removal of suffixes (such as s or es in English). This allows searches to find variant forms of the same word, without tediously entering all the possible variants. Also, this step typically eliminates stop words, which are words that are so common that they are useless for searching. (In short, then, tokens are raw fragments of the document text, while lexemes are words that are believed useful for indexing and searching.) PostgreSQL uses dictionaries to perform this step. Various standard dictionaries are provided, and custom ones can be created for specific needs. Storing preprocessed documents optimized for searching. For example, each document can be represented as a sorted array of normalized lexemes. Along with the lexemes it is often desirable to store positional information to use for proximity ranking, so that a document that contains a more “dense” region of query words is assigned a higher rank than one with scattered query words. Dictionaries allow fine-grained control over how tokens are normalized. With appropriate dictionaries, you can:
319
Chapter 12. Full Text Search Define stop words that should not be indexed. Map synonyms to a single word using Ispell. • Map phrases to a single word using a thesaurus. • Map different variations of a word to a canonical form using an Ispell dictionary. • Map different variations of a word to a canonical form using Snowball stemmer rules. •
•
A data type tsvector is provided for storing preprocessed documents, along with a type tsquery for representing processed queries (Section 8.11). There are many functions and operators available for these data types (Section 9.13), the most important of which is the match operator @@, which we introduce in Section 12.1.2. Full text searches can be accelerated using indexes (Section 12.9).
12.1.1. What Is a Document? A document is the unit of searching in a full text search system; for example, a magazine article or email message. The text search engine must be able to parse documents and store associations of lexemes (key words) with their parent document. Later, these associations are used to search for documents that contain query words. For searches within PostgreSQL, a document is normally a textual field within a row of a database table, or possibly a combination (concatenation) of such fields, perhaps stored in several tables or obtained dynamically. In other words, a document can be constructed from different parts for indexing and it might not be stored anywhere as a whole. For example: SELECT title || ’ ’ || FROM messages WHERE mid = 12;
author || ’ ’ ||
abstract || ’ ’ || body AS document
SELECT m.title || ’ ’ || m.author || ’ ’ || m.abstract || ’ ’ || d.body AS document FROM messages m, docs d WHERE mid = did AND mid = 12;
Note: Actually, in these example queries, coalesce should be used to prevent a single NULL attribute from causing a NULL result for the whole document.
Another possibility is to store the documents as simple text files in the file system. In this case, the database can be used to store the full text index and to execute searches, and some unique identifier can be used to retrieve the document from the file system. However, retrieving files from outside the database requires superuser permissions or special function support, so this is usually less convenient than keeping all the data inside PostgreSQL. Also, keeping everything inside the database allows easy access to document metadata to assist in indexing and display. For text search purposes, each document must be reduced to the preprocessed tsvector format. Searching and ranking are performed entirely on the tsvector representation of a document — the original text need only be retrieved when the document has been selected for display to a user. We therefore often speak of the tsvector as being the document, but of course it is only a compact representation of the full document.
320
Chapter 12. Full Text Search
12.1.2. Basic Text Matching Full text searching in PostgreSQL is based on the match operator @@, which returns true if a tsvector (document) matches a tsquery (query). It doesn’t matter which data type is written first: SELECT ’a fat cat sat on a mat and ate a fat rat’::tsvector @@ ’cat & rat’::tsquery; ?column? ---------t SELECT ’fat & cow’::tsquery @@ ’a fat cat sat on a mat and ate a fat rat’::tsvector; ?column? ---------f
As the above example suggests, a tsquery is not just raw text, any more than a tsvector is. A tsquery contains search terms, which must be already-normalized lexemes, and may combine multiple terms using AND, OR, and NOT operators. (For details see Section 8.11.) There are functions to_tsquery and plainto_tsquery that are helpful in converting user-written text into a proper tsquery, for example by normalizing words appearing in the text. Similarly, to_tsvector is used to parse and normalize a document string. So in practice a text search match would look more like this: SELECT to_tsvector(’fat cats ate fat rats’) @@ to_tsquery(’fat & rat’); ?column? ---------t
Observe that this match would not succeed if written as SELECT ’fat cats ate fat rats’::tsvector @@ to_tsquery(’fat & rat’); ?column? ---------f
since here no normalization of the word rats will occur. The elements of a tsvector are lexemes, which are assumed already normalized, so rats does not match rat. The @@ operator also supports text input, allowing explicit conversion of a text string to tsvector or tsquery to be skipped in simple cases. The variants available are: tsvector @@ tsquery tsquery @@ tsvector text @@ tsquery text @@ text
The first two of these we saw already. The form text @@ tsquery is equivalent to to_tsvector(x) @@ y. The form text @@ text is equivalent to to_tsvector(x) @@ plainto_tsquery(y).
321
Chapter 12. Full Text Search
12.1.3. Configurations The above are all simple text search examples. As mentioned before, full text search functionality includes the ability to do many more things: skip indexing certain words (stop words), process synonyms, and use sophisticated parsing, e.g., parse based on more than just white space. This functionality is controlled by text search configurations. PostgreSQL comes with predefined configurations for many languages, and you can easily create your own configurations. (psql’s \dF command shows all available configurations.) During installation an appropriate configuration is selected and default_text_search_config is set accordingly in postgresql.conf. If you are using the same text search configuration for the entire cluster you can use the value in postgresql.conf. To use different configurations throughout the cluster but the same configuration within any one database, use ALTER DATABASE ... SET. Otherwise, you can set default_text_search_config in each session. Each text search function that depends on a configuration has an optional regconfig argument, so that the configuration to use can be specified explicitly. default_text_search_config is used only when this argument is omitted. To make it easier to build custom text search configurations, a configuration is built up from simpler database objects. PostgreSQL’s text search facility provides four types of configuration-related database objects: Text search parsers break documents into tokens and classify each token (for example, as words or numbers). • Text search dictionaries convert tokens to normalized form and reject stop words. • Text search templates provide the functions underlying dictionaries. (A dictionary simply specifies a template and a set of parameters for the template.) • Text search configurations select a parser and a set of dictionaries to use to normalize the tokens produced by the parser. •
Text search parsers and templates are built from low-level C functions; therefore it requires C programming ability to develop new ones, and superuser privileges to install one into a database. (There are examples of add-on parsers and templates in the contrib/ area of the PostgreSQL distribution.) Since dictionaries and configurations just parameterize and connect together some underlying parsers and templates, no special privilege is needed to create a new dictionary or configuration. Examples of creating custom dictionaries and configurations appear later in this chapter.
12.2. Tables and Indexes The examples in the previous section illustrated full text matching using simple constant strings. This section shows how to search table data, optionally using indexes.
12.2.1. Searching a Table It is possible to do a full text search without an index. A simple query to print the title of each row that contains the word friend in its body field is: SELECT title FROM pgweb WHERE to_tsvector(’english’, body) @@ to_tsquery(’english’, ’friend’);
322
Chapter 12. Full Text Search This will also find related words such as friends and friendly, since all these are reduced to the same normalized lexeme. The query above specifies that the english configuration is to be used to parse and normalize the strings. Alternatively we could omit the configuration parameters: SELECT title FROM pgweb WHERE to_tsvector(body) @@ to_tsquery(’friend’);
This query will use the configuration set by default_text_search_config. A more complex example is to select the ten most recent documents that contain create and table in the title or body: SELECT title FROM pgweb WHERE to_tsvector(title || ’ ’ || body) @@ to_tsquery(’create & table’) ORDER BY last_mod_date DESC LIMIT 10;
For clarity we omitted the coalesce function calls which would be needed to find rows that contain NULL in one of the two fields. Although these queries will work without an index, most applications will find this approach too slow, except perhaps for occasional ad-hoc searches. Practical use of text searching usually requires creating an index.
12.2.2. Creating Indexes We can create a GIN index (Section 12.9) to speed up text searches: CREATE INDEX pgweb_idx ON pgweb USING gin(to_tsvector(’english’, body));
Notice that the 2-argument version of to_tsvector is used. Only text search functions that specify a configuration name can be used in expression indexes (Section 11.7). This is because the index contents must be unaffected by default_text_search_config. If they were affected, the index contents might be inconsistent because different entries could contain tsvectors that were created with different text search configurations, and there would be no way to guess which was which. It would be impossible to dump and restore such an index correctly. Because the two-argument version of to_tsvector was used in the index above, only a query reference that uses the 2-argument version of to_tsvector with the same configuration name will use that index. That is, WHERE to_tsvector(’english’, body) @@ ’a & b’ can use the index, but WHERE to_tsvector(body) @@ ’a & b’ cannot. This ensures that an index will be used only with the same configuration used to create the index entries. It is possible to set up more complex expression indexes wherein the configuration name is specified by another column, e.g.: CREATE INDEX pgweb_idx ON pgweb USING gin(to_tsvector(config_name, body));
where config_name is a column in the pgweb table. This allows mixed configurations in the same index while recording which configuration was used for each index entry. This would be useful, for example, if the document collection contained documents in different languages. Again, queries that are meant to use the index must be phrased to match, e.g., WHERE to_tsvector(config_name, body) @@ ’a & b’.
323
Chapter 12. Full Text Search Indexes can even concatenate columns: CREATE INDEX pgweb_idx ON pgweb USING gin(to_tsvector(’english’, title || ’ ’ || body));
Another approach is to create a separate tsvector column to hold the output of to_tsvector. This example is a concatenation of title and body, using coalesce to ensure that one field will still be indexed when the other is NULL: ALTER TABLE pgweb ADD COLUMN textsearchable_index_col tsvector; UPDATE pgweb SET textsearchable_index_col = to_tsvector(’english’, coalesce(title,”) || ’ ’ || coalesce(body,”));
Then we create a GIN index to speed up the search: CREATE INDEX textsearch_idx ON pgweb USING gin(textsearchable_index_col);
Now we are ready to perform a fast full text search: SELECT title FROM pgweb WHERE textsearchable_index_col @@ to_tsquery(’create & table’) ORDER BY last_mod_date DESC LIMIT 10;
When using a separate column to store the tsvector representation, it is necessary to create a trigger to keep the tsvector column current anytime title or body changes. Section 12.4.3 explains how to do that. One advantage of the separate-column approach over an expression index is that it is not necessary to explicitly specify the text search configuration in queries in order to make use of the index. As shown in the example above, the query can depend on default_text_search_config. Another advantage is that searches will be faster, since it will not be necessary to redo the to_tsvector calls to verify index matches. (This is more important when using a GiST index than a GIN index; see Section 12.9.) The expression-index approach is simpler to set up, however, and it requires less disk space since the tsvector representation is not stored explicitly.
12.3. Controlling Text Search To implement full text searching there must be a function to create a tsvector from a document and a tsquery from a user query. Also, we need to return results in a useful order, so we need a function that compares documents with respect to their relevance to the query. It’s also important to be able to display the results nicely. PostgreSQL provides support for all of these functions.
12.3.1. Parsing Documents PostgreSQL provides the function to_tsvector for converting a document to the tsvector data type. to_tsvector([ config regconfig, ] document text) returns tsvector
324
Chapter 12. Full Text Search to_tsvector parses a textual document into tokens, reduces the tokens to lexemes, and returns a tsvector which lists the lexemes together with their positions in the document. The document is
processed according to the specified or default text search configuration. Here is a simple example: SELECT to_tsvector(’english’, ’a fat cat sat on a mat - it ate a fat rats’); to_tsvector ----------------------------------------------------’ate’:9 ’cat’:3 ’fat’:2,11 ’mat’:7 ’rat’:12 ’sat’:4
In the example above we see that the resulting tsvector does not contain the words a, on, or it, the word rats became rat, and the punctuation sign - was ignored. The to_tsvector function internally calls a parser which breaks the document text into tokens and assigns a type to each token. For each token, a list of dictionaries (Section 12.6) is consulted, where the list can vary depending on the token type. The first dictionary that recognizes the token emits one or more normalized lexemes to represent the token. For example, rats became rat because one of the dictionaries recognized that the word rats is a plural form of rat. Some words are recognized as stop words (Section 12.6.1), which causes them to be ignored since they occur too frequently to be useful in searching. In our example these are a, on, and it. If no dictionary in the list recognizes the token then it is also ignored. In this example that happened to the punctuation sign - because there are in fact no dictionaries assigned for its token type (Space symbols), meaning space tokens will never be indexed. The choices of parser, dictionaries and which types of tokens to index are determined by the selected text search configuration (Section 12.7). It is possible to have many different configurations in the same database, and predefined configurations are available for various languages. In our example we used the default configuration english for the English language. The function setweight can be used to label the entries of a tsvector with a given weight, where a weight is one of the letters A, B, C, or D. This is typically used to mark entries coming from different parts of a document, such as title versus body. Later, this information can be used for ranking of search results. Because to_tsvector(NULL) will return NULL, it is recommended to use coalesce whenever a field might be null. Here is the recommended method for creating a tsvector from a structured document: UPDATE tt SET ti = setweight(to_tsvector(coalesce(title,”)), ’A’) || setweight(to_tsvector(coalesce(keyword,”)), ’B’) || setweight(to_tsvector(coalesce(abstract,”)), ’C’) || setweight(to_tsvector(coalesce(body,”)), ’D’);
Here we have used setweight to label the source of each lexeme in the finished tsvector, and then merged the labeled tsvector values using the tsvector concatenation operator ||. (Section 12.4.1 gives details about these operations.)
12.3.2. Parsing Queries PostgreSQL provides the functions to_tsquery and plainto_tsquery for converting a query to the tsquery data type. to_tsquery offers access to more features than plainto_tsquery, but is less forgiving about its input. to_tsquery([ config regconfig, ] querytext text) returns tsquery
325
Chapter 12. Full Text Search to_tsquery creates a tsquery value from querytext, which must consist of single tokens separated by the Boolean operators & (AND), | (OR) and ! (NOT). These operators can be grouped using parentheses. In other words, the input to to_tsquery must already follow the general rules for tsquery input, as described in Section 8.11. The difference is that while basic tsquery input takes the tokens at face value, to_tsquery normalizes each token to a lexeme using the specified or default configuration, and discards any tokens that are stop words according to the configuration. For example: SELECT to_tsquery(’english’, ’The & Fat & Rats’); to_tsquery --------------’fat’ & ’rat’
As in basic tsquery input, weight(s) can be attached to each lexeme to restrict it to match only tsvector lexemes of those weight(s). For example: SELECT to_tsquery(’english’, ’Fat | Rats:AB’); to_tsquery -----------------’fat’ | ’rat’:AB
Also, * can be attached to a lexeme to specify prefix matching: SELECT to_tsquery(’supern:*A & star:A*B’); to_tsquery -------------------------’supern’:*A & ’star’:*AB
Such a lexeme will match any word in a tsvector that begins with the given string. to_tsquery can also accept single-quoted phrases. This is primarily useful when the configuration
includes a thesaurus dictionary that may trigger on such phrases. In the example below, a thesaurus contains the rule supernovae stars : sn: SELECT to_tsquery(”’supernovae stars” & !crab’); to_tsquery --------------’sn’ & !’crab’
Without quotes, to_tsquery will generate a syntax error for tokens that are not separated by an AND or OR operator. plainto_tsquery([ config regconfig, ] querytext text) returns tsquery plainto_tsquery transforms unformatted text querytext to tsquery. The text is parsed and normalized much as for to_tsvector, then the & (AND) Boolean operator is inserted between surviving
Note that plainto_tsquery cannot recognize Boolean operators, weight labels, or prefix-match labels in its input:
326
Chapter 12. Full Text Search SELECT plainto_tsquery(’english’, ’The Fat & Rats:C’); plainto_tsquery --------------------’fat’ & ’rat’ & ’c’
Here, all the input punctuation was discarded as being space symbols.
12.3.3. Ranking Search Results Ranking attempts to measure how relevant documents are to a particular query, so that when there are many matches the most relevant ones can be shown first. PostgreSQL provides two predefined ranking functions, which take into account lexical, proximity, and structural information; that is, they consider how often the query terms appear in the document, how close together the terms are in the document, and how important is the part of the document where they occur. However, the concept of relevancy is vague and very application-specific. Different applications might require additional information for ranking, e.g., document modification time. The built-in ranking functions are only examples. You can write your own ranking functions and/or combine their results with additional factors to fit your specific needs. The two ranking functions currently available are: ts_rank([ weights float4[], ] vector tsvector, query tsquery [, normalization integer ]) returns float4
Ranks vectors based on the frequency of their matching lexemes. ts_rank_cd([ weights float4[], ] vector tsvector, query tsquery [, normalization integer ]) returns float4
This function computes the cover density ranking for the given document vector and query, as described in Clarke, Cormack, and Tudhope’s "Relevance Ranking for One to Three Term Queries" in the journal "Information Processing and Management", 1999. This function requires positional information in its input. Therefore it will not work on “stripped” tsvector values — it will always return zero.
For both these functions, the optional weights argument offers the ability to weigh word instances more or less heavily depending on how they are labeled. The weight arrays specify how heavily to weigh each category of word, in the order: {D-weight, C-weight, B-weight, A-weight}
If no weights are provided, then these defaults are used: {0.1, 0.2, 0.4, 1.0}
Typically weights are used to mark words from special areas of the document, like the title or an initial abstract, so they can be treated with more or less importance than words in the document body. Since a longer document has a greater chance of containing a query term it is reasonable to take into account document size, e.g., a hundred-word document with five instances of a search word is probably more relevant than a thousand-word document with five instances. Both ranking functions take an integer normalization option that specifies whether and how a document’s length should impact its rank. The integer option controls several behaviors, so it is a bit mask: you can specify one or more behaviors using | (for example, 2|4).
327
Chapter 12. Full Text Search • • • • • • •
0 (the default) ignores the document length 1 divides the rank by 1 + the logarithm of the document length 2 divides the rank by the document length 4 divides the rank by the mean harmonic distance between extents (this is implemented only by ts_rank_cd) 8 divides the rank by the number of unique words in document 16 divides the rank by 1 + the logarithm of the number of unique words in document 32 divides the rank by itself + 1
If more than one flag bit is specified, the transformations are applied in the order listed. It is important to note that the ranking functions do not use any global information, so it is impossible to produce a fair normalization to 1% or 100% as sometimes desired. Normalization option 32 (rank/(rank+1)) can be applied to scale all ranks into the range zero to one, but of course this is just a cosmetic change; it will not affect the ordering of the search results. Here is an example that selects only the ten highest-ranked matches: SELECT title, ts_rank_cd(textsearch, query) AS rank FROM apod, to_tsquery(’neutrino|(dark & matter)’) query WHERE query @@ textsearch ORDER BY rank DESC LIMIT 10; title | rank -----------------------------------------------+---------Neutrinos in the Sun | 3.1 The Sudbury Neutrino Detector | 2.4 A MACHO View of Galactic Dark Matter | 2.01317 Hot Gas and Dark Matter | 1.91171 The Virgo Cluster: Hot Plasma and Dark Matter | 1.90953 Rafting for Solar Neutrinos | 1.9 NGC 4650A: Strange Galaxy and Dark Matter | 1.85774 Hot Gas and Dark Matter | 1.6123 Ice Fishing for Cosmic Neutrinos | 1.6 Weak Lensing Distorts the Universe | 0.818218
This is the same example using normalized ranking: SELECT title, ts_rank_cd(textsearch, query, 32 /* rank/(rank+1) */ ) AS rank FROM apod, to_tsquery(’neutrino|(dark & matter)’) query WHERE query @@ textsearch ORDER BY rank DESC LIMIT 10; title | rank -----------------------------------------------+------------------Neutrinos in the Sun | 0.756097569485493 The Sudbury Neutrino Detector | 0.705882361190954 A MACHO View of Galactic Dark Matter | 0.668123210574724 Hot Gas and Dark Matter | 0.65655958650282 The Virgo Cluster: Hot Plasma and Dark Matter | 0.656301290640973 Rafting for Solar Neutrinos | 0.655172410958162 NGC 4650A: Strange Galaxy and Dark Matter | 0.650072921219637 Hot Gas and Dark Matter | 0.617195790024749 Ice Fishing for Cosmic Neutrinos | 0.615384618911517 Weak Lensing Distorts the Universe | 0.450010798361481
328
Chapter 12. Full Text Search Ranking can be expensive since it requires consulting the tsvector of each matching document, which can be I/O bound and therefore slow. Unfortunately, it is almost impossible to avoid since practical queries often result in large numbers of matches.
12.3.4. Highlighting Results To present search results it is ideal to show a part of each document and how it is related to the query. Usually, search engines show fragments of the document with marked search terms. PostgreSQL provides a function ts_headline that implements this functionality.
ts_headline([ config regconfig, ] document text, query tsquery [, options text ]) returns text ts_headline accepts a document along with a query, and returns an excerpt from the document in
which terms from the query are highlighted. The configuration to be used to parse the document can be specified by config ; if config is omitted, the default_text_search_config configuration is used. If an options string is specified it must consist of a comma-separated list of one or more option=value pairs. The available options are: • StartSel, StopSel: the strings with which to delimit query words appearing in the document, to
• • • •
•
distinguish them from other excerpted words. You must double-quote these strings if they contain spaces or commas. MaxWords, MinWords: these numbers determine the longest and shortest headlines to output. ShortWord: words of this length or less will be dropped at the start and end of a headline. The default value of three eliminates common English articles. HighlightAll: Boolean flag; if true the whole document will be used as the headline, ignoring the preceding three parameters. MaxFragments: maximum number of text excerpts or fragments to display. The default value of zero selects a non-fragment-oriented headline generation method. A value greater than zero selects fragment-based headline generation. This method finds text fragments with as many query words as possible and stretches those fragments around the query words. As a result query words are close to the middle of each fragment and have words on each side. Each fragment will be of at most MaxWords and words of length ShortWord or less are dropped at the start and end of each fragment. If not all query words are found in the document, then a single fragment of the first MinWords in the document will be displayed. FragmentDelimiter: When more than one fragment is displayed, the fragments will be separated by this string.
Any unspecified options receive these defaults: StartSel=, StopSel=, MaxWords=35, MinWords=15, ShortWord=3, HighlightAll=FALSE, MaxFragments=0, FragmentDelimiter=" ... "
For example: SELECT ts_headline(’english’, ’The most common type of search is to find all documents containing given query terms and return them in order of their similarity to the query.’,
329
Chapter 12. Full Text Search to_tsquery(’query & similarity’)); ts_headline -----------------------------------------------------------containing given query terms and return them in order of their similarity to the query. SELECT ts_headline(’english’, ’The most common type of search is to find all documents containing given query terms and return them in order of their similarity to the query.’, to_tsquery(’query & similarity’), ’StartSel = ’); ts_headline ------------------------------------------------------containing given terms and return them in order of their to the .
ts_headline uses the original document, not a tsvector summary, so it can be slow and should be used with care. A typical mistake is to call ts_headline for every matching document when only
ten documents are to be shown. SQL subqueries can help; here is an example: SELECT id, ts_headline(body, q), rank FROM (SELECT id, body, q, ts_rank_cd(ti, q) AS rank FROM apod, to_tsquery(’stars’) q WHERE ti @@ q ORDER BY rank DESC LIMIT 10) AS foo;
12.4. Additional Features This section describes additional functions and operators that are useful in connection with text search.
12.4.1. Manipulating Documents Section 12.3.1 showed how raw textual documents can be converted into tsvector values. PostgreSQL also provides functions and operators that can be used to manipulate documents that are already in tsvector form. tsvector || tsvector
The tsvector concatenation operator returns a vector which combines the lexemes and positional information of the two vectors given as arguments. Positions and weight labels are retained during the concatenation. Positions appearing in the right-hand vector are offset by the largest position mentioned in the left-hand vector, so that the result is nearly equivalent to the result of performing to_tsvector on the concatenation of the two original document strings. (The
330
Chapter 12. Full Text Search equivalence is not exact, because any stop-words removed from the end of the left-hand argument will not affect the result, whereas they would have affected the positions of the lexemes in the right-hand argument if textual concatenation were used.) One advantage of using concatenation in the vector form, rather than concatenating text before applying to_tsvector, is that you can use different configurations to parse different sections of the document. Also, because the setweight function marks all lexemes of the given vector the same way, it is necessary to parse the text and do setweight before concatenating if you want to label different parts of the document with different weights. setweight(vector tsvector, weight "char") returns tsvector setweight returns a copy of the input vector in which every position has been labeled with the given weight, either A, B, C, or D. (D is the default for new vectors and as such is not displayed on
output.) These labels are retained when vectors are concatenated, allowing words from different parts of a document to be weighted differently by ranking functions. Note that weight labels apply to positions, not lexemes. If the input vector has been stripped of positions then setweight does nothing. length(vector tsvector) returns integer
Returns the number of lexemes stored in the vector. strip(vector tsvector) returns tsvector
Returns a vector which lists the same lexemes as the given vector, but which lacks any position or weight information. While the returned vector is much less useful than an unstripped vector for relevance ranking, it will usually be much smaller.
12.4.2. Manipulating Queries Section 12.3.2 showed how raw textual queries can be converted into tsquery values. PostgreSQL also provides functions and operators that can be used to manipulate queries that are already in tsquery form. tsquery && tsquery
Returns the AND-combination of the two given queries. tsquery || tsquery
Returns the OR-combination of the two given queries. !! tsquery
Returns the negation (NOT) of the given query. numnode(query tsquery) returns integer
Returns the number of nodes (lexemes plus operators) in a tsquery. This function is useful to determine if the query is meaningful (returns > 0), or contains only stop words (returns 0). Examples: SELECT numnode(plainto_tsquery(’the any’)); NOTICE: query contains only stopword(s) or doesn’t contain lexeme(s), ignored numnode --------0 SELECT numnode(’foo & bar’::tsquery);
331
Chapter 12. Full Text Search numnode --------3 querytree(query tsquery) returns text
Returns the portion of a tsquery that can be used for searching an index. This function is useful for detecting unindexable queries, for example those containing only stop words or only negated terms. For example: SELECT querytree(to_tsquery(’!defined’)); querytree -----------
12.4.2.1. Query Rewriting The ts_rewrite family of functions search a given tsquery for occurrences of a target subquery, and replace each occurrence with a substitute subquery. In essence this operation is a tsqueryspecific version of substring replacement. A target and substitute combination can be thought of as a query rewrite rule. A collection of such rewrite rules can be a powerful search aid. For example, you can expand the search using synonyms (e.g., new york, big apple, nyc, gotham) or narrow the search to direct the user to some hot topic. There is some overlap in functionality between this feature and thesaurus dictionaries (Section 12.6.4). However, you can modify a set of rewrite rules on-the-fly without reindexing, whereas updating a thesaurus requires reindexing to be effective. ts_rewrite (query tsquery, target tsquery, substitute tsquery) returns tsquery
This form of ts_rewrite simply applies a single rewrite rule: target is replaced by substitute wherever it appears in query . For example: SELECT ts_rewrite(’a & b’::tsquery, ’a’::tsquery, ’c’::tsquery); ts_rewrite -----------’b’ & ’c’ ts_rewrite (query tsquery, select text) returns tsquery
This form of ts_rewrite accepts a starting query and a SQL select command, which is given as a text string. The select must yield two columns of tsquery type. For each row of the select result, occurrences of the first column value (the target) are replaced by the second column value (the substitute) within the current query value. For example: CREATE TABLE aliases (t tsquery PRIMARY KEY, s tsquery); INSERT INTO aliases VALUES(’a’, ’c’); SELECT ts_rewrite(’a & b’::tsquery, ’SELECT t,s FROM aliases’); ts_rewrite -----------’b’ & ’c’
Note that when multiple rewrite rules are applied in this way, the order of application can be important; so in practice you will want the source query to ORDER BY some ordering key. Let’s consider a real-life astronomical example. We’ll expand query supernovae using table-driven rewriting rules: CREATE TABLE aliases (t tsquery primary key, s tsquery); INSERT INTO aliases VALUES(to_tsquery(’supernovae’), to_tsquery(’supernovae|sn’)); SELECT ts_rewrite(to_tsquery(’supernovae & crab’), ’SELECT * FROM aliases’);
332
Chapter 12. Full Text Search ts_rewrite --------------------------------’crab’ & ( ’supernova’ | ’sn’ )
We can change the rewriting rules just by updating the table: UPDATE aliases SET s = to_tsquery(’supernovae|sn & !nebulae’) WHERE t = to_tsquery(’supernovae’); SELECT ts_rewrite(to_tsquery(’supernovae & crab’), ’SELECT * FROM aliases’); ts_rewrite --------------------------------------------’crab’ & ( ’supernova’ | ’sn’ & !’nebula’ )
Rewriting can be slow when there are many rewriting rules, since it checks every rule for a possible match. To filter out obvious non-candidate rules we can use the containment operators for the tsquery type. In the example below, we select only those rules which might match the original query: SELECT ts_rewrite(’a & b’::tsquery, ’SELECT t,s FROM aliases WHERE ”a & b”::tsquery @> t’); ts_rewrite -----------’b’ & ’c’
12.4.3. Triggers for Automatic Updates When using a separate column to store the tsvector representation of your documents, it is necessary to create a trigger to update the tsvector column when the document content columns change. Two built-in trigger functions are available for this, or you can write your own.
These trigger functions automatically compute a tsvector column from one or more textual columns, under the control of parameters specified in the CREATE TRIGGER command. An example of their use is: CREATE TABLE messages ( title text, body text, tsv tsvector ); CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE ON messages FOR EACH ROW EXECUTE PROCEDURE tsvector_update_trigger(tsv, ’pg_catalog.english’, title, body); INSERT INTO messages VALUES(’title here’, ’the body text is here’);
333
Chapter 12. Full Text Search SELECT * FROM messages; title | body | tsv ------------+-----------------------+---------------------------title here | the body text is here | ’bodi’:4 ’text’:5 ’titl’:1 SELECT title, body FROM messages WHERE tsv @@ to_tsquery(’title & body’); title | body ------------+----------------------title here | the body text is here
Having created this trigger, any change in title or body will automatically be reflected into tsv, without the application having to worry about it. The first trigger argument must be the name of the tsvector column to be updated. The second argument specifies the text search configuration to be used to perform the conversion. For tsvector_update_trigger, the configuration name is simply given as the second trigger argument. It must be schema-qualified as shown above, so that the trigger behavior will not change with changes in search_path. For tsvector_update_trigger_column, the second trigger argument is the name of another table column, which must be of type regconfig. This allows a per-row selection of configuration to be made. The remaining argument(s) are the names of textual columns (of type text, varchar, or char). These will be included in the document in the order given. NULL values will be skipped (but the other columns will still be indexed). A limitation of these built-in triggers is that they treat all the input columns alike. To process columns differently — for example, to weight title differently from body — it is necessary to write a custom trigger. Here is an example using PL/pgSQL as the trigger language: CREATE FUNCTION messages_trigger() RETURNS trigger AS $$ begin new.tsv := setweight(to_tsvector(’pg_catalog.english’, coalesce(new.title,”)), ’A’) || setweight(to_tsvector(’pg_catalog.english’, coalesce(new.body,”)), ’D’); return new; end $$ LANGUAGE plpgsql; CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE ON messages FOR EACH ROW EXECUTE PROCEDURE messages_trigger();
Keep in mind that it is important to specify the configuration name explicitly when creating tsvector values inside triggers, so that the column’s contents will not be affected by changes to default_text_search_config. Failure to do this is likely to lead to problems such as search results changing after a dump and reload.
12.4.4. Gathering Document Statistics The function ts_stat is useful for checking your configuration and for finding stop-word candidates. ts_stat(sqlquery text, [ weights text, ] OUT word text, OUT ndoc integer, OUT nentry integer) returns setof record
334
Chapter 12. Full Text Search sqlquery is a text value containing an SQL query which must return a single tsvector column. ts_stat executes the query and returns statistics about each distinct lexeme (word) contained in the tsvector data. The columns returned are
— the value of a lexeme — number of documents (tsvectors) the word occurred in • nentry integer — total number of occurrences of the word • word text
• ndoc integer
If weights is supplied, only occurrences having one of those weights are counted. For example, to find the ten most frequent words in a document collection: SELECT * FROM ts_stat(’SELECT vector FROM apod’) ORDER BY nentry DESC, ndoc DESC, word LIMIT 10;
The same, but counting only word occurrences with weight A or B: SELECT * FROM ts_stat(’SELECT vector FROM apod’, ’ab’) ORDER BY nentry DESC, ndoc DESC, word LIMIT 10;
12.5. Parsers Text search parsers are responsible for splitting raw document text into tokens and identifying each token’s type, where the set of possible types is defined by the parser itself. Note that a parser does not modify the text at all — it simply identifies plausible word boundaries. Because of this limited scope, there is less need for application-specific custom parsers than there is for custom dictionaries. At present PostgreSQL provides just one built-in parser, which has been found to be useful for a wide range of applications. The built-in parser is named pg_catalog.default. It recognizes 23 token types, shown in Table 12-1. Table 12-1. Default Parser’s Token Types Alias
Description
Example
asciiword
Word, all ASCII letters
elephant
word
Word, all letters
mañana
numword
Word, letters and digits
beta1
asciihword
Hyphenated word, all ASCII
up-to-date
hword
Hyphenated word, all letters
lógico-matemática
numhword
Hyphenated word, letters and digits
postgresql-beta1
hword_asciipart
Hyphenated word part, all ASCII
postgresql in the context postgresql-beta1
335
Chapter 12. Full Text Search Alias
Description
Example
hword_part
Hyphenated word part, all letters
lógico or matemática in the context lógico-matemática
Improve performance of pg_timezone_names view (Tom Lane, David Rowley)
•
Fix sloppy handling of corner-case errors from lseek() and close() (Tom Lane) Neither of these system calls are likely to fail in typical situations, but if they did, fd.c could get quite confused.
•
Fix incorrect check for whether postmaster is running as a Windows service (Michael Paquier) This could result in attempting to write to the event log when that isn’t accessible, so that no logging happens at all.
•
Fix ecpg to support COMMIT PREPARED and ROLLBACK PREPARED (Masahiko Sawada)
•
Fix a double-free error when processing dollar-quoted string literals in ecpg (Michael Meskes)
•
In pg_dump, fix incorrect schema and owner marking for comments and security labels of some types of database objects (Giuseppe Broccolo, Tom Lane) In simple cases this caused no ill effects; but for example, a schema-selective restore might omit comments it should include, because they were not marked as belonging to the schema of their associated object.
•
Avoid emitting an invalid list file in pg_restore -l when SQL object names contain newlines (Tom Lane) Replace newlines by spaces, which is sufficient to make the output valid for pg_restore -L’s purposes.
•
Fix pg_upgrade to transfer comments and security labels attached to “large objects” (blobs) (Stephen Frost) Previously, blobs were correctly transferred to the new database, but any comments or security labels attached to them were lost.
•
Improve error handling in contrib/adminpack’s pg_file_write() function (Noah Misch) Notably, it failed to detect errors reported by fclose().
•
In contrib/dblink, avoid leaking the previous unnamed connection when establishing a new unnamed connection (Joe Conway)
•
Fix contrib/pg_trgm’s extraction of trigrams from regular expressions (Tom Lane) In some cases it would produce a broken data structure that could never match anything, leading to GIN or GiST indexscans that use a trigram index not finding any matches to the regular expression.
•
In contrib/postgres_fdw, transmit query cancellation requests to the remote server (Michael Paquier, Etsuro Fujita) Previously, a local query cancellation request did not cause an already-sent remote query to terminate early. This is a back-patch of work originally done for 9.6.
•
Support OpenSSL 1.1.0 (Heikki Linnakangas, Andreas Karlsson, Tom Lane)
2006
Appendix E. Release Notes This is a back-patch of work previously done in newer branches; it’s needed since many platforms are adopting newer OpenSSL versions. •
Support Tcl 8.6 in MSVC builds (Álvaro Herrera)
•
Sync our copy of the timezone library with IANA release tzcode2017b (Tom Lane) This fixes a bug affecting some DST transitions in January 2038.
•
Update time zone data files to tzdata release 2017b for DST law changes in Chile, Haiti, and Mongolia, plus historical corrections for Ecuador, Kazakhstan, Liberia, and Spain. Switch to numeric abbreviations for numerous time zones in South America, the Pacific and Indian oceans, and some Asian and Middle Eastern countries. The IANA time zone database previously provided textual abbreviations for all time zones, sometimes making up abbreviations that have little or no currency among the local population. They are in process of reversing that policy in favor of using numeric UTC offsets in zones where there is no evidence of real-world use of an English abbreviation. At least for the time being, PostgreSQL will continue to accept such removed abbreviations for timestamp input. But they will not be shown in the pg_timezone_names view nor used for output.
•
Use correct daylight-savings rules for POSIX-style time zone names in MSVC builds (David Rowley) The Microsoft MSVC build scripts neglected to install the posixrules file in the timezone directory tree. This resulted in the timezone code falling back to its built-in rule about what DST behavior to assume for a POSIX-style time zone name. For historical reasons that still corresponds to the DST rules the USA was using before 2007 (i.e., change on first Sunday in April and last Sunday in October). With this fix, a POSIX-style zone name will use the current and historical DST transition dates of the US/Eastern zone. If you don’t want that, remove the posixrules file, or replace it with a copy of some other zone file (see Section 8.5.3). Note that due to caching, you may need to restart the server to get such changes to take effect.
E.4. Release 9.3.16 Release date: 2017-02-09 This release contains a variety of fixes from 9.3.15. For information about new features in the 9.3 major release, see Section E.20.
E.4.1. Migration to Version 9.3.16 A dump/restore is not required for those running 9.3.X. However, if your installation has been affected by the bug described in the first changelog entry below, then after updating you may need to take action to repair corrupted indexes. Also, if you are upgrading from a version earlier than 9.3.15, see Section E.5.
E.4.2. Changes •
Fix a race condition that could cause indexes built with CREATE INDEX CONCURRENTLY to be corrupt (Pavan Deolasee, Tom Lane)
2007
Appendix E. Release Notes If CREATE INDEX CONCURRENTLY was used to build an index that depends on a column not previously indexed, then rows updated by transactions that ran concurrently with the CREATE INDEX command could have received incorrect index entries. If you suspect this may have happened, the most reliable solution is to rebuild affected indexes after installing this update. •
Unconditionally WAL-log creation of the “init fork” for an unlogged table (Michael Paquier) Previously, this was skipped when wal_level = minimal, but actually it’s necessary even in that case to ensure that the unlogged table is properly reset to empty after a crash.
•
If the stats collector dies during hot standby, restart it (Takayuki Tsunakawa)
•
Ensure that hot standby feedback works correctly when it’s enabled at standby server start (Ants Aasma, Craig Ringer)
•
Check for interrupts while hot standby is waiting for a conflicting query (Simon Riggs)
•
Avoid constantly respawning the autovacuum launcher in a corner case (Amit Khandekar) This fix avoids problems when autovacuum is nominally off and there are some tables that require freezing, but all such tables are already being processed by autovacuum workers.
•
Fix check for when an extension member object can be dropped (Tom Lane) Extension upgrade scripts should be able to drop member objects, but this was disallowed for serial-column sequences, and possibly other cases.
•
Make sure ALTER TABLE preserves index tablespace assignments when rebuilding indexes (Tom Lane, Michael Paquier) Previously, non-default settings of default_tablespace could result in broken indexes.
•
Prevent dropping a foreign-key constraint if there are pending trigger events for the referenced relation (Tom Lane) This avoids “could not find trigger NNN ” or “relation NNN has no triggers” errors.
•
Fix processing of OID column when a table with OIDs is associated to a parent with OIDs via ALTER TABLE ... INHERIT (Amit Langote) The OID column should be treated the same as regular user columns in this case, but it wasn’t, leading to odd behavior in later inheritance changes.
•
Report correct object identity during ALTER TEXT SEARCH CONFIGURATION (Artur Zakirov) The wrong catalog OID was reported to extensions such as logical decoding.
•
Check for serializability conflicts before reporting constraint-violation failures (Thomas Munro) When using serializable transaction isolation, it is desirable that any error due to concurrent transactions should manifest as a serialization failure, thereby cueing the application that a retry might succeed. Unfortunately, this does not reliably happen for duplicate-key failures caused by concurrent insertions. This change ensures that such an error will be reported as a serialization error if the application explicitly checked for the presence of a conflicting key (and did not find it) earlier in the transaction.
•
Prevent multicolumn expansion of foo.* in an UPDATE source expression (Tom Lane) This led to “UPDATE target count mismatch --- internal error”. Now the syntax is understood as a whole-row variable, as it would be in other contexts.
•
Ensure that column typmods are determined accurately for multi-row VALUES constructs (Tom Lane) This fixes problems occurring when the first value in a column has a determinable typmod (e.g., length for a varchar value) but later values don’t share the same limit.
2008
Appendix E. Release Notes •
Throw error for an unfinished Unicode surrogate pair at the end of a Unicode string (Tom Lane) Normally, a Unicode surrogate leading character must be followed by a Unicode surrogate trailing character, but the check for this was missed if the leading character was the last character in a Unicode string literal (U&’...’) or Unicode identifier (U&"...").
•
Ensure that a purely negative text search query, such as !foo, matches empty tsvectors (Tom Dunstan) Such matches were found by GIN index searches, but not by sequential scans or GiST index searches.
•
Prevent crash when ts_rewrite() replaces a non-top-level subtree with an empty query (Artur Zakirov)
•
Fix performance problems in ts_rewrite() (Tom Lane)
•
Fix ts_rewrite()’s handling of nested NOT operators (Tom Lane)
•
Fix array_fill() to handle empty arrays properly (Tom Lane)
•
Fix one-byte buffer overrun in quote_literal_cstr() (Heikki Linnakangas) The overrun occurred only if the input consisted entirely of single quotes and/or backslashes.
•
Prevent multiple calls of pg_start_backup() and pg_stop_backup() from running concurrently (Michael Paquier) This avoids an assertion failure, and possibly worse things, if someone tries to run these functions in parallel.
•
Avoid discarding interval-to-interval casts that aren’t really no-ops (Tom Lane) In some cases, a cast that should result in zeroing out low-order interval fields was mistakenly deemed to be a no-op and discarded. An example is that casting from INTERVAL MONTH to INTERVAL YEAR failed to clear the months field.
•
Ensure that cached plans are invalidated by changes in foreign-table options (Amit Langote, Etsuro Fujita, Ashutosh Bapat)
•
Fix pg_dump to dump user-defined casts and transforms that use built-in functions (Stephen Frost)
•
Fix possible pg_basebackup failure on standby server when including WAL files (Amit Kapila, Robert Haas)
•
Ensure that the Python exception objects we create for PL/Python are properly reference-counted (Rafa de la Torre, Tom Lane) This avoids failures if the objects are used after a Python garbage collection cycle has occurred.
•
Fix PL/Tcl to support triggers on tables that have .tupno as a column name (Tom Lane) This matches the (previously undocumented) behavior of PL/Tcl’s spi_exec and spi_execp commands, namely that a magic .tupno column is inserted only if there isn’t a real column named that.
•
Allow DOS-style line endings in ~/.pgpass files, even on Unix (Vik Fearing) This change simplifies use of the same password file across Unix and Windows machines.
•
Fix one-byte buffer overrun if ecpg is given a file name that ends with a dot (Takayuki Tsunakawa)
•
Fix psql’s tab completion for ALTER DEFAULT PRIVILEGES (Gilles Darold, Stephen Frost)
•
In psql, treat an empty or all-blank setting of the PAGER environment variable as meaning “no pager” (Tom Lane) Previously, such a setting caused output intended for the pager to vanish entirely.
2009
Appendix E. Release Notes •
Improve contrib/dblink’s reporting of low-level libpq errors, such as out-of-memory (Joe Conway)
•
Teach
contrib/dblink to ignore irrelevant server options when it uses contrib/postgres_fdw foreign server as the source of connection options (Corey Huinker)
a
Previously, if the foreign server object had options that were not also libpq connection options, an error occurred. •
On Windows, ensure that environment variable changes are propagated to DLLs built with debug options (Christian Ullrich)
•
Sync our copy of the timezone library with IANA release tzcode2016j (Tom Lane) This fixes various issues, most notably that timezone data installation failed if the target directory didn’t support hard links.
•
Update time zone data files to tzdata release 2016j for DST law changes in northern Cyprus (adding a new zone Asia/Famagusta), Russia (adding a new zone Europe/Saratov), Tonga, and Antarctica/Casey. Historical corrections for Italy, Kazakhstan, Malta, and Palestine. Switch to preferring numeric zone abbreviations for Tonga.
E.5. Release 9.3.15 Release date: 2016-10-27 This release contains a variety of fixes from 9.3.14. For information about new features in the 9.3 major release, see Section E.20.
E.5.1. Migration to Version 9.3.15 A dump/restore is not required for those running 9.3.X. However, if your installation has been affected by the bug described in the first changelog entry below, then after updating you may need to take action to repair corrupted free space maps. Also, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
E.5.2. Changes •
Fix WAL-logging of truncation of relation free space maps and visibility maps (Pavan Deolasee, Heikki Linnakangas) It was possible for these files to not be correctly restored during crash recovery, or to be written incorrectly on a standby server. Bogus entries in a free space map could lead to attempts to access pages that have been truncated away from the relation itself, typically producing errors like “could not read block XXX : read only 0 of 8192 bytes”. Checksum failures in the visibility map are also possible, if checksumming is enabled. Procedures for determining whether there is a problem and repairing it if so are discussed at https://wiki.postgresql.org/wiki/Free_Space_Map_Problems.
•
Fix SELECT FOR UPDATE/SHARE to correctly lock tuples that have been updated by a subsequently-aborted transaction (Álvaro Herrera)
2010
Appendix E. Release Notes In 9.5 and later, the SELECT would sometimes fail to return such tuples at all. A failure has not been proven to occur in earlier releases, but might be possible with concurrent updates. •
Fix EvalPlanQual rechecks involving CTE scans (Tom Lane) The recheck would always see the CTE as returning no rows, typically leading to failure to update rows that were recently updated.
•
Fix improper repetition of previous results from hashed aggregation in a subquery (Andrew Gierth) The test to see if we can reuse a previously-computed hash table of the aggregate state values neglected the possibility of an outer query reference appearing in an aggregate argument expression. A change in the value of such a reference should lead to recalculating the hash table, but did not.
•
Fix EXPLAIN to emit valid XML when track_io_timing is on (Markus Winand) Previously the XML output-format option produced syntactically invalid tags such as . That is now rendered as .
•
Suppress printing of zeroes for unmeasured times in EXPLAIN (Maksim Milyutin) Certain option combinations resulted in printing zero values for times that actually aren’t ever measured in that combination. Our general policy in EXPLAIN is not to print such fields at all, so do that consistently in all cases.
•
Fix timeout length when VACUUM is waiting for exclusive table lock so that it can truncate the table (Simon Riggs) The timeout was meant to be 50 milliseconds, but it was actually only 50 microseconds, causing VACUUM to give up on truncation much more easily than intended. Set it to the intended value.
•
Fix bugs in merging inherited CHECK constraints while creating or altering a table (Tom Lane, Amit Langote) Allow identical CHECK constraints to be added to a parent and child table in either order. Prevent merging of a valid constraint from the parent table with a NOT VALID constraint on the child. Likewise, prevent merging of a NO INHERIT child constraint with an inherited constraint.
•
Remove artificial restrictions on the values accepted by numeric_in() and numeric_recv() (Tom Lane) We allow numeric values up to the limit of the storage format (more than 1e100000), so it seems fairly pointless that numeric_in() rejected scientific-notation exponents above 1000. Likewise, it was silly for numeric_recv() to reject more than 1000 digits in an input value.
•
Avoid very-low-probability data corruption due to testing tuple visibility without holding buffer lock (Thomas Munro, Peter Geoghegan, Tom Lane)
•
Fix file descriptor leakage when truncating a temporary relation of more than 1GB (Andres Freund)
•
Disallow starting a standalone backend with standby_mode turned on (Michael Paquier) This can’t do anything useful, since there will be no WAL receiver process to fetch more WAL data; and it could result in misbehavior in code that wasn’t designed with this situation in mind.
•
Don’t try to share SSL contexts across multiple connections in libpq (Heikki Linnakangas) This led to assorted corner-case bugs, particularly when trying to use different SSL parameters for different connections.
•
Avoid corner-case memory leak in libpq (Tom Lane) The reported problem involved leaking an error report during PQreset(), but there might be related cases.
2011
Appendix E. Release Notes •
Make ecpg’s --help and --version options work consistently with our other executables (Haribabu Kommi)
•
In pg_dump, never dump range constructor functions (Tom Lane) This oversight led to pg_upgrade failures with extensions containing range types, due to duplicate creation of the constructor functions.
•
In pg_xlogdump, retry opening new WAL segments when using --follow option (Magnus Hagander) This allows for a possible delay in the server’s creation of the next segment.
•
Fix pg_xlogdump to cope with a WAL file that begins with a continuation record spanning more than one page (Pavan Deolasee)
•
Fix contrib/intarray/bench/bench.pl to print the results of the EXPLAIN it does when given the -e option (Daniel Gustafsson)
•
Update Windows time zone mapping to recognize some time zone names added in recent Windows versions (Michael Paquier)
•
Prevent failure of obsolete dynamic time zone abbreviations (Tom Lane) If a dynamic time zone abbreviation does not match any entry in the referenced time zone, treat it as equivalent to the time zone name. This avoids unexpected failures when IANA removes abbreviations from their time zone database, as they did in tzdata release 2016f and seem likely to do again in the future. The consequences were not limited to not recognizing the individual abbreviation; any mismatch caused the pg_timezone_abbrevs view to fail altogether.
•
Update time zone data files to tzdata release 2016h for DST law changes in Palestine and Turkey, plus historical corrections for Turkey and some regions of Russia. Switch to numeric abbreviations for some time zones in Antarctica, the former Soviet Union, and Sri Lanka. The IANA time zone database previously provided textual abbreviations for all time zones, sometimes making up abbreviations that have little or no currency among the local population. They are in process of reversing that policy in favor of using numeric UTC offsets in zones where there is no evidence of real-world use of an English abbreviation. At least for the time being, PostgreSQL will continue to accept such removed abbreviations for timestamp input. But they will not be shown in the pg_timezone_names view nor used for output. In this update, AMT is no longer shown as being in use to mean Armenia Time. Therefore, we have changed the Default abbreviation set to interpret it as Amazon Time, thus UTC-4 not UTC+4.
E.6. Release 9.3.14 Release date: 2016-08-11 This release contains a variety of fixes from 9.3.13. For information about new features in the 9.3 major release, see Section E.20.
E.6.1. Migration to Version 9.3.14 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
2012
Appendix E. Release Notes
E.6.2. Changes •
Fix possible mis-evaluation of nested CASE-WHEN expressions (Heikki Linnakangas, Michael Paquier, Tom Lane) A CASE expression appearing within the test value subexpression of another CASE could become confused about whether its own test value was null or not. Also, inlining of a SQL function implementing the equality operator used by a CASE expression could result in passing the wrong test value to functions called within a CASE expression in the SQL function’s body. If the test values were of different data types, a crash might result; moreover such situations could be abused to allow disclosure of portions of server memory. (CVE-2016-5423)
•
Fix client programs’ handling of special characters in database and role names (Noah Misch, Nathan Bossart, Michael Paquier) Numerous places in vacuumdb and other client programs could become confused by database and role names containing double quotes or backslashes. Tighten up quoting rules to make that safe. Also, ensure that when a conninfo string is used as a database name parameter to these programs, it is correctly treated as such throughout. Fix handling of paired double quotes in psql’s \connect and \password commands to match the documentation. Introduce a new -reuse-previous option in psql’s \connect command to allow explicit control of whether to re-use connection parameters from a previous connection. (Without this, the choice is based on whether the database name looks like a conninfo string, as before.) This allows secure handling of database names containing special characters in pg_dumpall scripts. pg_dumpall now refuses to deal with database and role names containing carriage returns or newlines, as it seems impractical to quote those characters safely on Windows. In future we may reject such names on the server side, but that step has not been taken yet. These are considered security fixes because crafted object names containing special characters could have been used to execute commands with superuser privileges the next time a superuser executes pg_dumpall or other routine maintenance operations. (CVE-2016-5424)
•
Fix corner-case misbehaviors for IS NULL/IS NOT NULL applied to nested composite values (Andrew Gierth, Tom Lane) The SQL standard specifies that IS NULL should return TRUE for a row of all null values (thus ROW(NULL,NULL) IS NULL yields TRUE), but this is not meant to apply recursively (thus ROW(NULL, ROW(NULL,NULL)) IS NULL yields FALSE). The core executor got this right, but certain planner optimizations treated the test as recursive (thus producing TRUE in both cases), and contrib/postgres_fdw could produce remote queries that misbehaved similarly.
•
Make the inet and cidr data types properly reject IPv6 addresses with too many colon-separated fields (Tom Lane)
•
Prevent crash in close_ps() (the point ## lseg operator) for NaN input coordinates (Tom Lane) Make it return NULL instead of crashing.
•
Avoid possible crash in pg_get_expr() when inconsistent values are passed to it (Michael Paquier, Thomas Munro)
•
Fix several one-byte buffer over-reads in to_number() (Peter Eisentraut)
2013
Appendix E. Release Notes In several cases the to_number() function would read one more character than it should from the input string. There is a small chance of a crash, if the input happens to be adjacent to the end of memory. •
Do not run the planner on the query contained in CREATE MATERIALIZED VIEW or CREATE TABLE AS when WITH NO DATA is specified (Michael Paquier, Tom Lane) This avoids some unnecessary failure conditions, for example if a stable function invoked by the materialized view depends on a table that doesn’t exist yet.
•
Avoid unsafe intermediate state during expensive paths through heap_update() (Masahiko Sawada, Andres Freund) Previously, these cases locked the target tuple (by setting its XMAX) but did not WAL-log that action, thus risking data integrity problems if the page were spilled to disk and then a database crash occurred before the tuple update could be completed.
•
Fix hint bit update during WAL replay of row locking operations (Andres Freund) The only known consequence of this problem is that row locks held by a prepared, but uncommitted, transaction might fail to be enforced after a crash and restart.
•
Avoid unnecessary “could not serialize access” errors when acquiring FOR KEY SHARE row locks in serializable mode (Álvaro Herrera)
•
Avoid crash in postgres -C when the specified variable has a null string value (Michael Paquier)
•
Ensure that backends see up-to-date statistics for shared catalogs (Tom Lane) The statistics collector failed to update the statistics file for shared catalogs after a request from a regular backend. This problem was partially masked because the autovacuum launcher regularly makes requests that did cause such updates; however, it became obvious with autovacuum disabled.
•
Avoid redundant writes of the statistics files when multiple backends request updates close together (Tom Lane, Tomas Vondra)
•
Avoid consuming a transaction ID during VACUUM (Alexander Korotkov) Some cases in VACUUM unnecessarily caused an XID to be assigned to the current transaction. Normally this is negligible, but if one is up against the XID wraparound limit, consuming more XIDs during anti-wraparound vacuums is a very bad thing.
•
Avoid canceling hot-standby queries during VACUUM FREEZE (Simon Riggs, Álvaro Herrera) VACUUM FREEZE on an otherwise-idle master server could result in unnecessary cancellations of
queries on its standby servers. •
Prevent possible failure when vacuuming multixact IDs in an installation that has been pg_upgrade’d from pre-9.3 (Andrew Gierth, Álvaro Herrera) The usual symptom of this bug is errors like “MultiXactId NNN has not been created yet -- apparent wraparound”.
•
When
a
manual
ANALYZE specifies a changes_since_analyze counter (Tom Lane)
column
list,
don’t
reset
the
table’s
If we’re only analyzing some columns, we should not prevent routine auto-analyze from happening for the other columns. •
Fix ANALYZE’s overestimation of n_distinct for a unique or nearly-unique column with many null entries (Tom Lane) The nulls could get counted as though they were themselves distinct values, leading to serious planner misestimates in some types of queries.
2014
Appendix E. Release Notes •
Prevent autovacuum from starting multiple workers for the same shared catalog (Álvaro Herrera) Normally this isn’t much of a problem because the vacuum doesn’t take long anyway; but in the case of a severely bloated catalog, it could result in all but one worker uselessly waiting instead of doing useful work on other tables.
•
Prevent infinite loop in GiST index build for geometric columns containing NaN component values (Tom Lane)
•
Fix contrib/btree_gin to handle the smallest possible bigint value correctly (Peter Eisentraut)
•
Teach libpq to correctly decode server version from future servers (Peter Eisentraut) It’s planned to switch to two-part instead of three-part server version numbers for releases after 9.6. Make sure that PQserverVersion() returns the correct value for such cases.
•
Fix ecpg’s code for unsigned long long array elements (Michael Meskes)
•
In pg_dump with both -c and -C options, avoid emitting an unwanted CREATE SCHEMA public command (David Johnston, Tom Lane)
•
Improve handling of SIGTERM/control-C in parallel pg_dump and pg_restore (Tom Lane) Make sure that the worker processes will exit promptly, and also arrange to send query-cancel requests to the connected backends, in case they are doing something long-running such as a CREATE INDEX.
•
Fix error reporting in parallel pg_dump and pg_restore (Tom Lane) Previously, errors reported by pg_dump or pg_restore worker processes might never make it to the user’s console, because the messages went through the master process, and there were various deadlock scenarios that would prevent the master process from passing on the messages. Instead, just print everything to stderr. In some cases this will result in duplicate messages (for instance, if all the workers report a server shutdown), but that seems better than no message.
•
Ensure that parallel pg_dump or pg_restore on Windows will shut down properly after an error (Kyotaro Horiguchi) Previously, it would report the error, but then just sit until manually stopped by the user.
•
Make pg_dump behave better when built without zlib support (Kyotaro Horiguchi) It didn’t work right for parallel dumps, and emitted some rather pointless warnings in other cases.
•
Make pg_basebackup accept -Z 0 as specifying no compression (Fujii Masao)
•
Fix makefiles’ rule for building AIX shared libraries to be safe for parallel make (Noah Misch)
•
Fix TAP tests and MSVC scripts to work when build directory’s path name contains spaces (Michael Paquier, Kyotaro Horiguchi)
•
Be more predictable about reporting “statement timeout” versus “lock timeout” (Tom Lane) On heavily loaded machines, the regression tests sometimes failed due to reporting “lock timeout” even though the statement timeout should have occurred first.
•
Make regression tests safe for Danish and Welsh locales (Jeff Janes, Tom Lane) Change some test data that triggered the unusual sorting rules of these locales.
•
Update our copy of the timezone code to match IANA’s tzcode release 2016c (Tom Lane) This is needed to cope with anticipated future changes in the time zone data files. It also fixes some corner-case bugs in coping with unusual time zones.
2015
Appendix E. Release Notes •
Update time zone data files to tzdata release 2016f for DST law changes in Kemerovo and Novosibirsk, plus historical corrections for Azerbaijan, Belarus, and Morocco.
E.7. Release 9.3.13 Release date: 2016-05-12 This release contains a variety of fixes from 9.3.12. For information about new features in the 9.3 major release, see Section E.20.
E.7.1. Migration to Version 9.3.13 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
E.7.2. Changes •
Clear the OpenSSL error queue before OpenSSL calls, rather than assuming it’s clear already; and make sure we leave it clear afterwards (Peter Geoghegan, Dave Vitek, Peter Eisentraut) This change prevents problems when there are multiple connections using OpenSSL within a single process and not all the code involved follows the same rules for when to clear the error queue. Failures have been reported specifically when a client application uses SSL connections in libpq concurrently with SSL connections using the PHP, Python, or Ruby wrappers for OpenSSL. It’s possible for similar problems to arise within the server as well, if an extension module establishes an outgoing SSL connection.
•
Fix “failed to build any N -way joins” planner error with a full join enclosed in the right-hand side of a left join (Tom Lane)
•
Fix incorrect handling of equivalence-class tests in multilevel nestloop plans (Tom Lane) Given a three-or-more-way equivalence class of variables, such as X.X = Y.Y = Z.Z, it was possible for the planner to omit some of the tests needed to enforce that all the variables are actually equal, leading to join rows being output that didn’t satisfy the WHERE clauses. For various reasons, erroneous plans were seldom selected in practice, so that this bug has gone undetected for a long time.
•
Fix possible misbehavior of TH, th, and Y,YYY format codes in to_timestamp() (Tom Lane) These could advance off the end of the input string, causing subsequent format codes to read garbage.
•
Fix dumping of rules and views in which the array argument of a value operator ANY (array ) construct is a sub-SELECT (Tom Lane)
•
Make pg_regress use a startup timeout from the PGCTLTIMEOUT environment variable, if that’s set (Tom Lane) This is for consistency with a behavior recently added to pg_ctl; it eases automated testing on slow machines.
2016
Appendix E. Release Notes •
Fix pg_upgrade to correctly restore extension membership for operator families containing only one operator class (Tom Lane) In such a case, the operator family was restored into the new database, but it was no longer marked as part of the extension. This had no immediate ill effects, but would cause later pg_dump runs to emit output that would cause (harmless) errors on restore.
•
Fix pg_upgrade to not fail when new-cluster TOAST rules differ from old (Tom Lane) pg_upgrade had special-case code to handle the situation where the new PostgreSQL version thinks that a table should have a TOAST table while the old version did not. That code was broken, so remove it, and instead do nothing in such cases; there seems no reason to believe that we can’t get along fine without a TOAST table if that was okay according to the old version’s rules.
•
Back-port 9.4-era memory-barrier code changes into 9.2 and 9.3 (Tom Lane) These changes were not originally needed in pre-9.4 branches, but we recently back-patched a fix that expected the barrier code to work properly. Only IA64 (when using icc), HPPA, and Alpha platforms are affected.
•
Reduce the number of SysV semaphores used by a build configured with --disable-spinlocks (Tom Lane)
•
Rename internal function strtoi() to strtoint() to avoid conflict with a NetBSD library function (Thomas Munro)
•
Fix reporting of errors from bind() and listen() system calls on Windows (Tom Lane)
•
Reduce verbosity of compiler output when building with Microsoft Visual Studio (Christian Ullrich)
•
Fix putenv() to work properly with Visual Studio 2013 (Michael Paquier)
•
Avoid possibly-unsafe use of Windows’ FormatMessage() function (Christian Ullrich) Use the FORMAT_MESSAGE_IGNORE_INSERTS flag where appropriate. No live bug is known to exist here, but it seems like a good idea to be careful.
•
Update time zone data files to tzdata release 2016d for DST law changes in Russia and Venezuela. There are new zone names Europe/Kirov and Asia/Tomsk to reflect the fact that these regions now have different time zone histories from adjacent regions.
E.8. Release 9.3.12 Release date: 2016-03-31 This release contains a variety of fixes from 9.3.11. For information about new features in the 9.3 major release, see Section E.20.
E.8.1. Migration to Version 9.3.12 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
2017
Appendix E. Release Notes
E.8.2. Changes •
Fix incorrect handling of NULL index entries in indexed ROW() comparisons (Tom Lane) An index search using a row comparison such as ROW(a, b) > ROW(’x’, ’y’) would stop upon reaching a NULL entry in the b column, ignoring the fact that there might be non-NULL b values associated with later values of a.
•
Avoid unlikely data-loss scenarios due to renaming files without adequate fsync() calls before and after (Michael Paquier, Tomas Vondra, Andres Freund)
•
Correctly handle cases where pg_subtrans is close to XID wraparound during server startup (Jeff Janes)
•
Fix corner-case crash due to trying to free localeconv() output strings more than once (Tom Lane)
•
Fix parsing of affix files for ispell dictionaries (Tom Lane) The code could go wrong if the affix file contained any characters whose byte length changes during case-folding, for example I in Turkish UTF8 locales.
•
Avoid use of sscanf() to parse ispell dictionary files (Artur Zakirov) This dodges a portability problem on FreeBSD-derived platforms (including macOS).
•
Avoid a crash on old Windows versions (before 7SP1/2008R2SP1) with an AVX2-capable CPU and a Postgres build done with Visual Studio 2013 (Christian Ullrich) This is a workaround for a bug in Visual Studio 2013’s runtime library, which Microsoft have stated they will not fix in that version.
•
Fix psql’s tab completion logic to handle multibyte characters properly (Kyotaro Horiguchi, Robert Haas)
•
Fix psql’s tab completion for SECURITY LABEL (Tom Lane) Pressing TAB after SECURITY LABEL might cause a crash or offering of inappropriate keywords.
•
Make pg_ctl accept a wait timeout from the PGCTLTIMEOUT environment variable, if none is specified on the command line (Noah Misch) This eases testing of slower buildfarm members by allowing them to globally specify a longer-thannormal timeout for postmaster startup and shutdown.
•
Fix incorrect test for Windows service status in pg_ctl (Manuel Mathar) The previous set of minor releases attempted to fix pg_ctl to properly determine whether to send log messages to Window’s Event Log, but got the test backwards.
•
Fix pgbench to correctly handle the combination of -C and -M prepared options (Tom Lane)
•
In pg_upgrade, skip creating a deletion script when the new data directory is inside the old data directory (Bruce Momjian) Blind application of the script in such cases would result in loss of the new data directory.
•
In PL/Perl, properly translate empty Postgres arrays into empty Perl arrays (Alex Hunsaker)
•
Make PL/Python cope with function names that aren’t valid Python identifiers (Jim Nasby)
•
Fix multiple mistakes in the statistics returned by contrib/pgstattuple’s pgstatindex() function (Tom Lane)
•
Remove dependency on psed in MSVC builds, since it’s no longer provided by core Perl (Michael Paquier, Andrew Dunstan)
2018
Appendix E. Release Notes •
Update time zone data files to tzdata release 2016c for DST law changes in Azerbaijan, Chile, Haiti, Palestine, and Russia (Altai, Astrakhan, Kirov, Sakhalin, Ulyanovsk regions), plus historical corrections for Lithuania, Moldova, and Russia (Kaliningrad, Samara, Volgograd).
E.9. Release 9.3.11 Release date: 2016-02-11 This release contains a variety of fixes from 9.3.10. For information about new features in the 9.3 major release, see Section E.20.
E.9.1. Migration to Version 9.3.11 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
E.9.2. Changes •
Fix infinite loops and buffer-overrun problems in regular expressions (Tom Lane) Very large character ranges in bracket expressions could cause infinite loops in some cases, and memory overwrites in other cases. (CVE-2016-0773)
•
Perform an immediate shutdown if the postmaster.pid file is removed (Tom Lane) The postmaster now checks every minute or so that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The main motivation for this change is to ensure that failed buildfarm runs will get cleaned up without manual intervention; but it also serves to limit the bad effects if a DBA forcibly removes postmaster.pid and then starts a new postmaster.
•
In SERIALIZABLE transaction isolation mode, serialization anomalies could be missed due to race conditions during insertions (Kevin Grittner, Thomas Munro)
•
Fix failure to emit appropriate WAL records when doing ALTER TABLE ... SET TABLESPACE for unlogged relations (Michael Paquier, Andres Freund) Even though the relation’s data is unlogged, the move must be logged or the relation will be inaccessible after a standby is promoted to master.
•
Fix possible misinitialization of unlogged relations at the end of crash recovery (Andres Freund, Michael Paquier)
•
Ensure walsender slots are fully re-initialized when being re-used (Magnus Hagander)
•
Fix ALTER COLUMN TYPE to reconstruct inherited check constraints properly (Tom Lane)
•
Fix REASSIGN OWNED to change ownership of composite types properly (Álvaro Herrera)
•
Fix REASSIGN OWNED and ALTER OWNER to correctly update granted-permissions lists when changing owners of data types, foreign data wrappers, or foreign servers (Bruce Momjian, Álvaro Herrera)
•
Fix REASSIGN OWNED to ignore foreign user mappings, rather than fail (Álvaro Herrera)
2019
Appendix E. Release Notes •
Fix possible crash after doing query rewrite for an updatable view (Stephen Frost)
•
Fix planner’s handling of LATERAL references (Tom Lane) This fixes some corner cases that led to “failed to build any N-way joins” or “could not devise a query plan” planner failures.
•
Add more defenses against bad planner cost estimates for GIN index scans when the index’s internal statistics are very out-of-date (Tom Lane)
•
Make planner cope with hypothetical GIN indexes suggested by an index advisor plug-in (Julien Rouhaud)
•
Speed up generation of unique table aliases in EXPLAIN and rule dumping, and ensure that generated aliases do not exceed NAMEDATALEN (Tom Lane)
•
Fix dumping of whole-row Vars in ROW() and VALUES() lists (Tom Lane)
•
Fix possible internal overflow in numeric division (Dean Rasheed)
•
Fix enforcement of restrictions inside parentheses within regular expression lookahead constraints (Tom Lane) Lookahead constraints aren’t allowed to contain backrefs, and parentheses within them are always considered non-capturing, according to the manual. However, the code failed to handle these cases properly inside a parenthesized subexpression, and would give unexpected results.
•
Conversion of regular expressions to indexscan bounds could produce incorrect bounds from regexps containing lookahead constraints (Tom Lane)
•
Fix regular-expression compiler to handle loops of constraint arcs (Tom Lane) The code added for CVE-2007-4772 was both incomplete, in that it didn’t handle loops involving more than one state, and incorrect, in that it could cause assertion failures (though there seem to be no bad consequences of that in a non-assert build). Multi-state loops would cause the compiler to run until the query was canceled or it reached the too-many-states error condition.
•
Improve memory-usage accounting in regular-expression compiler (Tom Lane) This causes the code to emit “regular expression is too complex” errors in some cases that previously used unreasonable amounts of time and memory.
•
Improve performance of regular-expression compiler (Tom Lane)
•
Make %h and %r escapes in log_line_prefix work for messages emitted due to log_connections (Tom Lane) Previously, %h/%r started to work just after a new session had emitted the “connection received” log message; now they work for that message too.
•
On Windows, ensure the shared-memory mapping handle gets closed in child processes that don’t need it (Tom Lane, Amit Kapila) This oversight resulted in failure to recover from crashes whenever logging_collector is turned on.
•
Fix possible failure to detect socket EOF in non-blocking mode on Windows (Tom Lane) It’s not entirely clear whether this problem can happen in pre-9.5 branches, but if it did, the symptom would be that a walsender process would wait indefinitely rather than noticing a loss of connection.
•
Avoid leaking a token handle during SSPI authentication (Christian Ullrich)
•
In psql, ensure that libreadline’s idea of the screen size is updated when the terminal window size changes (Merlin Moncure)
2020
Appendix E. Release Notes Previously, libreadline did not notice if the window was resized during query output, leading to strange behavior during later input of multiline queries. •
Fix psql’s \det command to interpret its pattern argument the same way as other \d commands with potentially schema-qualified patterns do (Reece Hart)
•
Avoid possible crash in psql’s \c command when previous connection was via Unix socket and command specifies a new hostname and same username (Tom Lane)
•
In pg_ctl start -w, test child process status directly rather than relying on heuristics (Tom Lane, Michael Paquier) Previously, pg_ctl relied on an assumption that the new postmaster would always create postmaster.pid within five seconds. But that can fail on heavily-loaded systems, causing pg_ctl to report incorrectly that the postmaster failed to start. Except on Windows, this change also means that a pg_ctl start -w done immediately after another such command will now reliably fail, whereas previously it would report success if done within two seconds of the first command.
•
In pg_ctl start -w, don’t attempt to use a wildcard listen address to connect to the postmaster (Kondo Yuta) On Windows, pg_ctl would fail to detect postmaster startup if listen_addresses is set to 0.0.0.0 or ::, because it would try to use that value verbatim as the address to connect to, which doesn’t work. Instead assume that 127.0.0.1 or ::1, respectively, is the right thing to use.
•
In pg_ctl on Windows, check service status to decide where to send output, rather than checking if standard output is a terminal (Michael Paquier)
•
In pg_dump and pg_basebackup, adopt the GNU convention for handling tar-archive members exceeding 8GB (Tom Lane) The POSIX standard for tar file format does not allow archive member files to exceed 8GB, but most modern implementations of tar support an extension that fixes that. Adopt this extension so that pg_dump with -Ft no longer fails on tables with more than 8GB of data, and so that pg_basebackup can handle files larger than 8GB. In addition, fix some portability issues that could cause failures for members between 4GB and 8GB on some platforms. Potentially these problems could cause unrecoverable data loss due to unreadable backup files.
•
Fix assorted corner-case bugs in pg_dump’s processing of extension member objects (Tom Lane)
•
Make pg_dump mark a view’s triggers as needing to be processed after its rule, to prevent possible failure during parallel pg_restore (Tom Lane)
•
Ensure that relation option values are properly quoted in pg_dump (Kouhei Sutou, Tom Lane) A reloption value that isn’t a simple identifier or number could lead to dump/reload failures due to syntax errors in CREATE statements issued by pg_dump. This is not an issue with any reloption currently supported by core PostgreSQL, but extensions could allow reloptions that cause the problem.
•
Avoid repeated password prompts during parallel pg_dump (Zeus Kronion)
•
Fix pg_upgrade’s file-copying code to handle errors properly on Windows (Bruce Momjian)
•
Install guards in pgbench against corner-case overflow conditions during evaluation of scriptspecified division or modulo operators (Fabien Coelho, Michael Paquier)
•
Fix failure to localize messages emitted by pg_receivexlog and pg_recvlogical (Ioseph Kim)
•
Avoid dump/reload problems when using both plpython2 and plpython3 (Tom Lane)
2021
Appendix E. Release Notes In principle, both versions of PL/Python can be used in the same database, though not in the same session (because the two versions of libpython cannot safely be used concurrently). However, pg_restore and pg_upgrade both do things that can fall foul of the same-session restriction. Work around that by changing the timing of the check. •
Fix PL/Python regression tests to pass with Python 3.5 (Peter Eisentraut)
•
Fix premature clearing of libpq’s input buffer when socket EOF is seen (Tom Lane) This mistake caused libpq to sometimes not report the backend’s final error message before reporting “server closed the connection unexpectedly”.
•
Prevent certain PL/Java parameters from being set by non-superusers (Noah Misch) This change mitigates a PL/Java security bug (CVE-2016-0766), which was fixed in PL/Java by marking these parameters as superuser-only. To fix the security hazard for sites that update PostgreSQL more frequently than PL/Java, make the core code aware of them also.
•
Improve libpq’s handling of out-of-memory situations (Michael Paquier, Amit Kapila, Heikki Linnakangas)
•
Fix order of arguments in ecpg-generated typedef statements (Michael Meskes)
•
Use %g not %f format in ecpg’s PGTYPESnumeric_from_double() (Tom Lane)
•
Fix ecpg-supplied header files to not contain comments continued from a preprocessor directive line onto the next line (Michael Meskes) Such a comment is rejected by ecpg. It’s not yet clear whether ecpg itself should be changed.
•
Fix hstore_to_json_loose()’s test for whether an hstore value can be converted to a JSON number (Tom Lane) Previously this function could be fooled by non-alphanumeric trailing characters, leading to emitting syntactically-invalid JSON.
•
Ensure that contrib/pgcrypto’s crypt() function can be interrupted by query cancel (Andreas Karlsson)
•
Accept flex versions later than 2.5.x (Tom Lane, Michael Paquier) Now that flex 2.6.0 has been released, the version checks in our build scripts needed to be adjusted.
•
Improve reproducibility of build output by ensuring filenames are given to the linker in a fixed order (Christoph Berg) This avoids possible bitwise differences in the produced executable files from one build to the next.
•
Install our missing script where PGXS builds can find it (Jim Nasby) This allows sane behavior in a PGXS build done on a machine where build tools such as bison are missing.
•
Ensure that dynloader.h is included in the installed header files in MSVC builds (Bruce Momjian, Michael Paquier)
•
Add variant regression test expected-output file to match behavior of current libxml2 (Tom Lane) The fix for libxml2’s CVE-2015-7499 causes it not to output error context reports in some cases where it used to do so. This seems to be a bug, but we’ll probably have to live with it for some time, so work around it.
•
Update time zone data files to tzdata release 2016a for DST law changes in Cayman Islands, Metlakatla, and Trans-Baikal Territory (Zabaykalsky Krai), plus historical corrections for Pakistan.
2022
Appendix E. Release Notes
E.10. Release 9.3.10 Release date: 2015-10-08 This release contains a variety of fixes from 9.3.9. For information about new features in the 9.3 major release, see Section E.20.
E.10.1. Migration to Version 9.3.10 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.9, see Section E.11.
E.10.2. Changes •
Guard against stack overflows in json parsing (Oskari Saarenmaa) If an application constructs PostgreSQL json or jsonb values from arbitrary user input, the application’s users can reliably crash the PostgreSQL server, causing momentary denial of service. (CVE-2015-5289)
•
Fix contrib/pgcrypto to detect and report too-short crypt() salts (Josh Kupershmidt) Certain invalid salt arguments crashed the server or disclosed a few bytes of server memory. We have not ruled out the viability of attacks that arrange for presence of confidential information in the disclosed bytes, but they seem unlikely. (CVE-2015-5288)
•
Fix subtransaction cleanup after a portal (cursor) belonging to an outer subtransaction fails (Tom Lane, Michael Paquier) A function executed in an outer-subtransaction cursor could cause an assertion failure or crash by referencing a relation created within an inner subtransaction.
•
Ensure all relations referred to by an updatable view are properly locked during an update statement (Dean Rasheed)
•
Fix insertion of relations into the relation cache “init file” (Tom Lane) An
oversight
in
a
patch
in
the
most
recent
minor
releases
caused
pg_trigger_tgrelid_tgname_index to be omitted from the init file. Subsequent sessions
detected this, then deemed the init file to be broken and silently ignored it, resulting in a significant degradation in session startup time. In addition to fixing the bug, install some guards so that any similar future mistake will be more obvious. •
Avoid O(N^2) behavior when inserting many tuples into a SPI query result (Neil Conway)
•
Improve LISTEN startup time when there are many unread notifications (Matt Newell)
•
Fix performance problem when a session alters large numbers of foreign key constraints (Jan Wieck, Tom Lane) This was seen primarily when restoring pg_dump output for databases with many thousands of tables.
•
Disable SSL renegotiation by default (Michael Paquier, Andres Freund) While use of SSL renegotiation is a good idea in theory, we have seen too many bugs in practice, both in the underlying OpenSSL library and in our usage of it. Renegotiation will
2023
Appendix E. Release Notes be removed entirely in 9.5 and later. In the older branches, just change the default value of ssl_renegotiation_limit to zero (disabled). •
Lower the minimum values of the *_freeze_max_age parameters (Andres Freund) This is mainly to make tests of related behavior less time-consuming, but it may also be of value for installations with limited disk space.
•
Limit the maximum value of wal_buffers to 2GB to avoid server crashes (Josh Berkus)
•
Avoid logging complaints when a parameter that can only be set at server start appears multiple times in postgresql.conf, and fix counting of line numbers after an include_dir directive (Tom Lane)
•
Fix rare internal overflow in multiplication of numeric values (Dean Rasheed)
•
Guard against hard-to-reach stack overflows involving record types, range types, json, jsonb, tsquery, ltxtquery and query_int (Noah Misch)
•
Fix handling of DOW and DOY in datetime input (Greg Stark) These tokens aren’t meant to be used in datetime values, but previously they resulted in opaque internal error messages rather than “invalid input syntax”.
•
Add more query-cancel checks to regular expression matching (Tom Lane)
•
Add recursion depth protections to regular expression, SIMILAR TO, and LIKE matching (Tom Lane) Suitable search patterns and a low stack depth limit could lead to stack-overrun crashes.
•
Fix potential infinite loop in regular expression execution (Tom Lane) A search pattern that can apparently match a zero-length string, but actually doesn’t match because of a back reference, could lead to an infinite loop.
•
In regular expression execution, correctly record match data for capturing parentheses within a quantifier even when the match is zero-length (Tom Lane)
•
Fix low-memory failures in regular expression compilation (Andreas Seltenreich)
•
Fix low-probability memory leak during regular expression execution (Tom Lane)
•
Fix rare low-memory failure in lock cleanup during transaction abort (Tom Lane)
•
Fix “unexpected out-of-memory situation during sort” errors when using tuplestores with small work_mem settings (Tom Lane)
•
Fix very-low-probability stack overrun in qsort (Tom Lane)
•
Fix “invalid memory alloc request size” failure in hash joins with large work_mem settings (Tomas Vondra, Tom Lane)
•
Fix assorted planner bugs (Tom Lane) These mistakes could lead to incorrect query plans that would give wrong answers, or to assertion failures in assert-enabled builds, or to odd planner errors such as “could not devise a query plan for the given query”, “could not find pathkey item to sort”, “plan should not reference subplan’s variable”, or “failed to assign all NestLoopParams to plan nodes”. Thanks are due to Andreas Seltenreich and Piotr Stefaniak for fuzz testing that exposed these problems.
•
Improve planner’s performance for UPDATE/DELETE on large inheritance sets (Tom Lane, Dean Rasheed)
•
Ensure standby promotion trigger files are removed at postmaster startup (Michael Paquier, Fujii Masao)
2024
Appendix E. Release Notes This prevents unwanted promotion from occurring if these files appear in a database backup that is used to initialize a new standby server. •
During postmaster shutdown, ensure that per-socket lock files are removed and listen sockets are closed before we remove the postmaster.pid file (Tom Lane) This avoids race-condition failures if an external script attempts to start a new postmaster as soon as pg_ctl stop returns.
•
Fix postmaster’s handling of a startup-process crash during crash recovery (Tom Lane) If, during a crash recovery cycle, the startup process crashes without having restored database consistency, we’d try to launch a new startup process, which typically would just crash again, leading to an infinite loop.
•
Make emergency autovacuuming for multixact wraparound more robust (Andres Freund)
•
Do not print a WARNING when an autovacuum worker is already gone when we attempt to signal it, and reduce log verbosity for such signals (Tom Lane)
•
Prevent autovacuum launcher from sleeping unduly long if the server clock is moved backwards a large amount (Álvaro Herrera)
•
Ensure that cleanup of a GIN index’s pending-insertions list is interruptable by cancel requests (Jeff Janes)
•
Allow all-zeroes pages in GIN indexes to be reused (Heikki Linnakangas) Such a page might be left behind after a crash.
•
Fix handling of all-zeroes pages in SP-GiST indexes (Heikki Linnakangas) VACUUM attempted to recycle such pages, but did so in a way that wasn’t crash-safe.
•
Fix off-by-one error that led to otherwise-harmless warnings about “apparent wraparound” in subtrans/multixact truncation (Thomas Munro)
•
Fix misreporting of CONTINUE and MOVE statement types in PL/pgSQL’s error context messages (Pavel Stehule, Tom Lane)
Fix PL/Python crash when returning the string representation of a record result (Tom Lane)
•
Fix some places in PL/Tcl that neglected to check for failure of malloc() calls (Michael Paquier, Álvaro Herrera)
•
In contrib/isn, fix output of ISBN-13 numbers that begin with 979 (Fabien Coelho) EANs beginning with 979 (but not 9790) are considered ISBNs, but they must be printed in the new 13-digit format, not the 10-digit format.
•
Improve contrib/postgres_fdw’s handling of collation-related decisions (Tom Lane) The main user-visible effect is expected to be that comparisons involving varchar columns will be sent to the remote server for execution in more cases than before.
•
Improve libpq’s handling of out-of-memory conditions (Michael Paquier, Heikki Linnakangas)
•
Fix memory leaks and missing out-of-memory checks in ecpg (Michael Paquier)
•
Fix psql’s code for locale-aware formatting of numeric output (Tom Lane) The formatting code invoked by \pset numericlocale on did the wrong thing for some uncommon cases such as numbers with an exponent but no decimal point. It could also mangle already-localized output from the money data type.
•
Prevent crash in psql’s \c command when there is no current connection (Noah Misch)
2025
Appendix E. Release Notes •
Make pg_dump handle inherited NOT VALID check constraints correctly (Tom Lane)
•
Fix selection of default zlib compression level in pg_dump’s directory output format (Andrew Dunstan)
•
Ensure that temporary files created during a pg_dump run with tar-format output are not worldreadable (Michael Paquier)
•
Fix pg_dump and pg_upgrade to support cases where the postgres or template1 database is in a non-default tablespace (Marti Raudsepp, Bruce Momjian)
•
Fix pg_dump to handle object privileges sanely when dumping from a server too old to have a particular privilege type (Tom Lane) When dumping data types from pre-9.2 servers, and when dumping functions or procedural languages from pre-7.3 servers, pg_dump would produce GRANT/REVOKE commands that revoked the owner’s grantable privileges and instead granted all privileges to PUBLIC. Since the privileges involved are just USAGE and EXECUTE, this isn’t a security problem, but it’s certainly a surprising representation of the older systems’ behavior. Fix it to leave the default privilege state alone in these cases.
•
Fix pg_dump to dump shell types (Tom Lane) Shell types (that is, not-yet-fully-defined types) aren’t useful for much, but nonetheless pg_dump should dump them.
•
Fix assorted minor memory leaks in pg_dump and other client-side programs (Michael Paquier)
•
Fix spinlock assembly code for PPC hardware to be compatible with AIX’s native assembler (Tom Lane) Building with gcc didn’t work if gcc had been configured to use the native assembler, which is becoming more common.
•
On AIX, test the -qlonglong compiler option rather than just assuming it’s safe to use (Noah Misch)
•
On AIX, use -Wl,-brtllib link option to allow symbols to be resolved at runtime (Noah Misch) Perl relies on this ability in 5.8.0 and later.
•
Avoid use of inline functions when compiling with 32-bit xlc, due to compiler bugs (Noah Misch)
•
Use librt for sched_yield() when necessary, which it is on some Solaris versions (Oskari Saarenmaa)
•
Fix Windows install.bat script to handle target directory names that contain spaces (Heikki Linnakangas)
•
Make the numeric form of the PostgreSQL version number (e.g., 90405) readily available to extension Makefiles, as a variable named VERSION_NUM (Michael Paquier)
•
Update time zone data files to tzdata release 2015g for DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, and Uruguay. There is a new zone name America/Fort_Nelson for the Canadian Northern Rockies.
E.11. Release 9.3.9 Release date: 2015-06-12
2026
Appendix E. Release Notes This release contains a small number of fixes from 9.3.8. For information about new features in the 9.3 major release, see Section E.20.
E.11.1. Migration to Version 9.3.9 A dump/restore is not required for those running 9.3.X. However, if you are upgrading an installation that was previously upgraded using a pg_upgrade version between 9.3.0 and 9.3.4 inclusive, see the first changelog entry below. Also, if you are upgrading from a version earlier than 9.3.7, see Section E.13.
E.11.2. Changes •
Fix possible failure to recover from an inconsistent database state (Robert Haas) Recent PostgreSQL releases introduced mechanisms to protect against multixact wraparound, but some of that code did not account for the possibility that it would need to run during crash recovery, when the database may not be in a consistent state. This could result in failure to restart after a crash, or failure to start up a secondary server. The lingering effects of a previously-fixed bug in pg_upgrade could also cause such a failure, in installations that had used pg_upgrade versions between 9.3.0 and 9.3.4. The pg_upgrade bug in question was that it would set oldestMultiXid to 1 in pg_control even if the true value should be higher. With the fixes introduced in this release, such a situation will result in immediate emergency autovacuuming until a correct oldestMultiXid value can be determined. If that would pose a hardship, users can avoid it by doing manual vacuuming before upgrading to this release. In detail: 1. Check whether pg_controldata reports “Latest checkpoint’s oldestMultiXid” to be 1. If not, there’s nothing to do. 2. Look in PGDATA/pg_multixact/offsets to see if there’s a file named 0000. If there is, there’s nothing to do. 3. Otherwise, for each table that has pg_class.relminmxid equal to 1, VACUUM that table with both vacuum_multixact_freeze_min_age and vacuum_multixact_freeze_table_age set to zero. (You can use the vacuum cost delay parameters described in Section 18.4.4 to reduce the performance consequences for concurrent sessions.) You must use PostgreSQL 9.3.5 or later to perform this step.
•
Fix rare failure to invalidate relation cache init file (Tom Lane) With just the wrong timing of concurrent activity, a VACUUM FULL on a system catalog might fail to update the “init file” that’s used to avoid cache-loading work for new sessions. This would result in later sessions being unable to access that catalog at all. This is a very ancient bug, but it’s so hard to trigger that no reproducible case had been seen until recently.
•
Avoid deadlock between incoming sessions and CREATE/DROP DATABASE (Tom Lane) A new session starting in a database that is the target of a DROP DATABASE command, or is the template for a CREATE DATABASE command, could cause the command to wait for five seconds and then fail, even if the new session would have exited before that.
2027
Appendix E. Release Notes •
Improve planner’s cost estimates for semi-joins and anti-joins with inner indexscans (Tom Lane, Tomas Vondra) This type of plan is quite cheap when all the join clauses are used as index scan conditions, even if the inner scan would nominally fetch many rows, because the executor will stop after obtaining one row. The planner only partially accounted for that effect, and would therefore overestimate the cost, leading it to possibly choose some other much less efficient plan type.
E.12. Release 9.3.8 Release date: 2015-06-04 This release contains a small number of fixes from 9.3.7. For information about new features in the 9.3 major release, see Section E.20.
E.12.1. Migration to Version 9.3.8 A dump/restore is not required for those running 9.3.X. However, if you are upgrading from a version earlier than 9.3.7, see Section E.13.
E.12.2. Changes •
Avoid failures while fsync’ing data directory during crash restart (Abhijit Menon-Sen, Tom Lane) In the previous minor releases we added a patch to fsync everything in the data directory after a crash. Unfortunately its response to any error condition was to fail, thereby preventing the server from starting up, even when the problem was quite harmless. An example is that an unwritable file in the data directory would prevent restart on some platforms; but it is common to make SSL certificate files unwritable by the server. Revise this behavior so that permissions failures are ignored altogether, and other types of failures are logged but do not prevent continuing. Also apply the same rules in initdb --sync-only. This case is less critical but it should act similarly.
•
Fix pg_get_functiondef() to show functions’ LEAKPROOF property, if set (Jeevan Chalke)
•
Remove configure’s check prohibiting linking to a threaded libpython on OpenBSD (Tom Lane) The failure this restriction was meant to prevent seems to not be a problem anymore on current OpenBSD versions.
•
Allow libpq to use TLS protocol versions beyond v1 (Noah Misch) For a long time, libpq was coded so that the only SSL protocol it would allow was TLS v1. Now that newer TLS versions are becoming popular, allow it to negotiate the highest commonly-supported TLS version with the server. (PostgreSQL servers were already capable of such negotiation, so no change is needed on the server side.) This is a back-patch of a change already released in 9.4.0.
2028
Appendix E. Release Notes
E.13. Release 9.3.7 Release date: 2015-05-22 This release contains a variety of fixes from 9.3.6. For information about new features in the 9.3 major release, see Section E.20.
E.13.1. Migration to Version 9.3.7 A dump/restore is not required for those running 9.3.X. However, if you use contrib/citext’s regexp_matches() functions, see the changelog entry below about that. Also, if you are upgrading from a version earlier than 9.3.6, see Section E.14.
E.13.2. Changes •
Avoid possible crash when client disconnects just before the authentication timeout expires (Benkocs Norbert Attila) If the timeout interrupt fired partway through the session shutdown sequence, SSL-related state would be freed twice, typically causing a crash and hence denial of service to other sessions. Experimentation shows that an unauthenticated remote attacker could trigger the bug somewhat consistently, hence treat as security issue. (CVE-2015-3165)
•
Improve detection of system-call failures (Noah Misch) Our replacement implementation of snprintf() failed to check for errors reported by the underlying system library calls; the main case that might be missed is out-of-memory situations. In the worst case this might lead to information exposure, due to our code assuming that a buffer had been overwritten when it hadn’t been. Also, there were a few places in which security-relevant calls of other system library functions did not check for failure. It remains possible that some calls of the *printf() family of functions are vulnerable to information disclosure if an out-of-memory error occurs at just the wrong time. We judge the risk to not be large, but will continue analysis in this area. (CVE-2015-3166)
•
In contrib/pgcrypto, uniformly report decryption failures as “Wrong key or corrupt data” (Noah Misch) Previously, some cases of decryption with an incorrect key could report other error message texts. It has been shown that such variance in error reports can aid attackers in recovering keys from other systems. While it’s unknown whether pgcrypto’s specific behaviors are likewise exploitable, it seems better to avoid the risk by using a one-size-fits-all message. (CVE-2015-3167)
•
Protect against wraparound of multixact member IDs (Álvaro Herrera, Robert Haas, Thomas Munro) Under certain usage patterns, the existing defenses against this might be insufficient, allowing pg_multixact/members files to be removed too early, resulting in data loss. The fix for this includes modifying the server to fail transactions that would result in overwriting old multixact member ID data, and improving autovacuum to ensure it will act proactively to prevent multixact member ID wraparound, as it does for transaction ID wraparound.
•
Fix incorrect declaration of contrib/citext’s regexp_matches() functions (Tom Lane)
2029
Appendix E. Release Notes These functions should return setof text[], like the core functions they are wrappers for; but they were incorrectly declared as returning just text[]. This mistake had two results: first, if there was no match you got a scalar null result, whereas what you should get is an empty set (zero rows). Second, the g flag was effectively ignored, since you would get only one result array even if there were multiple matches. While the latter behavior is clearly a bug, there might be applications depending on the former behavior; therefore the function declarations will not be changed by default until PostgreSQL 9.5. In pre-9.5 branches, the old behavior exists in version 1.0 of the citext extension, while we have provided corrected declarations in version 1.1 (which is not installed by default). To adopt the fix in pre-9.5 branches, execute ALTER EXTENSION citext UPDATE TO ’1.1’ in each database in which citext is installed. (You can also “update” back to 1.0 if you need to undo that.) Be aware that either update direction will require dropping and recreating any views or rules that use citext’s regexp_matches() functions. •
Fix incorrect checking of deferred exclusion constraints after a HOT update (Tom Lane) If a new row that potentially violates a deferred exclusion constraint is HOT-updated (that is, no indexed columns change and the row can be stored back onto the same table page) later in the same transaction, the exclusion constraint would be reported as violated when the check finally occurred, even if the row(s) the new row originally conflicted with had been deleted.
•
Fix planning of star-schema-style queries (Tom Lane) Sometimes, efficient scanning of a large table requires that index parameters be provided from more than one other table (commonly, dimension tables whose keys are needed to index a large fact table). The planner should be able to find such plans, but an overly restrictive search heuristic prevented it.
•
Prevent improper reordering of antijoins (NOT EXISTS joins) versus other outer joins (Tom Lane) This oversight in the planner has been observed to cause “could not find RelOptInfo for given relids” errors, but it seems possible that sometimes an incorrect query plan might get past that consistency check and result in silently-wrong query output.
•
Fix incorrect matching of subexpressions in outer-join plan nodes (Tom Lane) Previously, if textually identical non-strict subexpressions were used both above and below an outer join, the planner might try to re-use the value computed below the join, which would be incorrect because the executor would force the value to NULL in case of an unmatched outer row.
•
Fix GEQO planner to cope with failure of its join order heuristic (Tom Lane) This oversight has been seen to lead to “failed to join all relations together” errors in queries involving LATERAL, and that might happen in other cases as well.
•
Fix possible deadlock at startup when max_prepared_transactions is too small (Heikki Linnakangas)
•
Don’t archive useless preallocated WAL files after a timeline switch (Heikki Linnakangas)
•
Recursively fsync() the data directory after a crash (Abhijit Menon-Sen, Robert Haas) This ensures consistency if another crash occurs shortly later. (The second crash would have to be a system-level crash, not just a database crash, for there to be a problem.)
•
Fix autovacuum launcher’s possible failure to shut down, if an error occurs after it receives SIGTERM (Álvaro Herrera)
•
Cope with unexpected signals in LockBufferForCleanup() (Andres Freund) This oversight could result in spurious errors about “multiple backends attempting to wait for pincount 1”.
2030
Appendix E. Release Notes •
Fix crash when doing COPY IN to a table with check constraints that contain whole-row references (Tom Lane) The known failure case only crashes in 9.4 and up, but there is very similar code in 9.3 and 9.2, so back-patch those branches as well.
•
Avoid waiting for WAL flush or synchronous replication during commit of a transaction that was read-only so far as the user is concerned (Andres Freund) Previously, a delay could occur at commit in transactions that had written WAL due to HOT page pruning, leading to undesirable effects such as sessions getting stuck at startup if all synchronous replicas are down. Sessions have also been observed to get stuck in catchup interrupt processing when using synchronous replication; this will fix that problem as well.
•
Fix crash when manipulating hash indexes on temporary tables (Heikki Linnakangas)
•
Fix possible failure during hash index bucket split, if other processes are modifying the index concurrently (Tom Lane)
•
Check for interrupts while analyzing index expressions (Jeff Janes) ANALYZE executes index expressions many times; if there are slow functions in such an expression, it’s desirable to be able to cancel the ANALYZE before that loop finishes.
•
Ensure tableoid of a foreign table is reported correctly when a READ COMMITTED recheck occurs after locking rows in SELECT FOR UPDATE, UPDATE, or DELETE (Etsuro Fujita)
•
Add the name of the target server to object description strings for foreign-server user mappings (Álvaro Herrera)
•
Include the schema name in object identity strings for conversions (Álvaro Herrera)
•
Recommend setting include_realm to 1 when using Kerberos/GSSAPI/SSPI authentication (Stephen Frost) Without this, identically-named users from different realms cannot be distinguished. For the moment this is only a documentation change, but it will become the default setting in PostgreSQL 9.5.
•
Remove code for matching IPv4 pg_hba.conf entries to IPv4-in-IPv6 addresses (Tom Lane) This hack was added in 2003 in response to a report that some Linux kernels of the time would report IPv4 connections as having IPv4-in-IPv6 addresses. However, the logic was accidentally broken in 9.0. The lack of any field complaints since then shows that it’s not needed anymore. Now we have reports that the broken code causes crashes on some systems, so let’s just remove it rather than fix it. (Had we chosen to fix it, that would make for a subtle and potentially security-sensitive change in the effective meaning of IPv4 pg_hba.conf entries, which does not seem like a good thing to do in minor releases.)
•
Report WAL flush, not insert, position in IDENTIFY_SYSTEM replication command (Heikki Linnakangas) This avoids a possible startup failure in pg_receivexlog.
•
While shutting down service on Windows, periodically send status updates to the Service Control Manager to prevent it from killing the service too soon; and ensure that pg_ctl will wait for shutdown (Krystian Bigaj)
•
Reduce risk of network deadlock when using libpq’s non-blocking mode (Heikki Linnakangas) When sending large volumes of data, it’s important to drain the input buffer every so often, in case the server has sent enough response data to cause it to block on output. (A typical scenario is that the server is sending a stream of NOTICE messages during COPY FROM STDIN.) This worked
2031
Appendix E. Release Notes properly in the normal blocking mode, but not so much in non-blocking mode. We’ve modified libpq to opportunistically drain input when it can, but a full defense against this problem requires application cooperation: the application should watch for socket read-ready as well as write-ready conditions, and be sure to call PQconsumeInput() upon read-ready. •
In libpq, fix misparsing of empty values in URI connection strings (Thomas Fanghaenel)
•
Fix array handling in ecpg (Michael Meskes)
•
Fix psql to sanely handle URIs and conninfo strings as the first parameter to \connect (David Fetter, Andrew Dunstan, Álvaro Herrera) This syntax has been accepted (but undocumented) for a long time, but previously some parameters might be taken from the old connection instead of the given string, which was agreed to be undesirable.
•
Suppress incorrect complaints from psql on some platforms that it failed to write ~/.psql_history at exit (Tom Lane) This misbehavior was caused by a workaround for a bug in very old (pre-2006) versions of libedit. We fixed it by removing the workaround, which will cause a similar failure to appear for anyone still using such versions of libedit. Recommendation: upgrade that library, or use libreadline.
•
Fix pg_dump’s rule for deciding which casts are system-provided casts that should not be dumped (Tom Lane)
•
In pg_dump, fix failure to honor -Z compression level option together with -Fd (Michael Paquier)
•
Make pg_dump consider foreign key relationships between extension configuration tables while choosing dump order (Gilles Darold, Michael Paquier, Stephen Frost) This oversight could result in producing dumps that fail to reload because foreign key constraints are transiently violated.
•
Avoid possible pg_dump failure when concurrent sessions are creating and dropping temporary functions (Tom Lane)
•
Fix dumping of views that are just VALUES(...) but have column aliases (Tom Lane)
•
In pg_upgrade, force timeline 1 in the new cluster (Bruce Momjian) This change prevents upgrade failures caused by bogus complaints about missing WAL history files.
•
In pg_upgrade, check for improperly non-connectable databases before proceeding (Bruce Momjian)
•
In pg_upgrade, quote directory paths properly in the generated delete_old_cluster script (Bruce Momjian)
•
In pg_upgrade, preserve database-level freezing info properly (Bruce Momjian) This oversight could cause missing-clog-file errors for tables within the postgres and template1 databases.
•
Run pg_upgrade and pg_resetxlog with restricted privileges on Windows, so that they don’t fail when run by an administrator (Muhammad Asif Naeem)
•
Improve handling of readdir() failures when scanning directories in initdb and pg_basebackup (Marco Nenciarini)
•
Fix slow sorting algorithm in contrib/intarray (Tom Lane)
•
Fix compile failure on Sparc V8 machines (Rob Rowan)
•
Silence some build warnings on macOS (Tom Lane)
2032
Appendix E. Release Notes •
Update time zone data files to tzdata release 2015d for DST law changes in Egypt, Mongolia, and Palestine, plus historical changes in Canada and Chile. Also adopt revised zone abbreviations for the America/Adak zone (HST/HDT not HAST/HADT).
E.14. Release 9.3.6 Release date: 2015-02-05 This release contains a variety of fixes from 9.3.5. For information about new features in the 9.3 major release, see Section E.20.
E.14.1. Migration to Version 9.3.6 A dump/restore is not required for those running 9.3.X. However, if you are a Windows user and are using the “Norwegian (Bokmål)” locale, manual action is needed after the upgrade to replace any “Norwegian (Bokmål)_Norway” locale names stored in PostgreSQL system catalogs with the plain-ASCII alias “Norwegian_Norway”. For details see http://wiki.postgresql.org/wiki/Changes_To_Norwegian_Locale Also, if you are upgrading from a version earlier than 9.3.5, see Section E.15.
E.14.2. Changes •
Fix buffer overruns in to_char() (Bruce Momjian) When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241)
•
Fix buffer overrun in replacement *printf() functions (Tom Lane) PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well. This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242)
•
Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch) Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243)
•
Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas)
2033
Appendix E. Release Notes If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message’s data as a new protocol message. An attacker able to submit crafted binary data within a command parameter might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244) •
Fix information leak via constraint-violation error messages (Stephen Frost) Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161)
•
Lock down regression testing’s temporary installations on Windows (Noah Misch) Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067)
•
Cope with the Windows locale named “Norwegian (Bokmål)” (Heikki Linnakangas) Non-ASCII locale names are problematic since it’s not clear what encoding they should be represented in. Map the troublesome locale name to a plain-ASCII alias, “Norwegian_Norway”.
•
Avoid possible data corruption if ALTER DATABASE SET TABLESPACE is used to move a database to a new tablespace and then shortly later move it back to its original tablespace (Tom Lane)
•
Avoid corrupting tables when ANALYZE inside a transaction is rolled back (Andres Freund, Tom Lane, Michael Paquier) If the failing transaction had earlier removed the last index, rule, or trigger from the table, the table would be left in a corrupted state with the relevant pg_class flags not set though they should be.
•
Ensure that unlogged tables are copied correctly during CREATE DATABASE or ALTER DATABASE SET TABLESPACE (Pavan Deolasee, Andres Freund)
•
Fix incorrect processing of CreateEventTrigStmt.eventname (Petr Jelinek) This could result in misbehavior if CREATE EVENT TRIGGER were executed as a prepared query, or via extended query protocol.
•
Fix DROP’s dependency searching to correctly handle the case where a table column is recursively visited before its table (Petr Jelinek, Tom Lane) This case is only known to arise when an extension creates both a datatype and a table using that datatype. The faulty code might refuse a DROP EXTENSION unless CASCADE is specified, which should not be required.
•
Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane) In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug.
•
Avoid possible deadlock while trying to acquire tuple locks in EvalPlanQual processing (Álvaro Herrera, Mark Kirkwood)
•
Fix failure to wait when a transaction tries to acquire a FOR NO KEY EXCLUSIVE tuple lock, while multiple other transactions currently hold FOR SHARE locks (Álvaro Herrera)
2034
Appendix E. Release Notes •
Fix planning of SELECT FOR UPDATE when using a partial index on a child table (Kyotaro Horiguchi) In READ COMMITTED mode, SELECT FOR UPDATE must also recheck the partial index’s WHERE condition when rechecking a recently-updated row to see if it still satisfies the query’s WHERE condition. This requirement was missed if the index belonged to an inheritance child table, so that it was possible to incorrectly return rows that no longer satisfy the query condition.
•
Fix corner case wherein SELECT FOR UPDATE could return a row twice, and possibly miss returning other rows (Tom Lane) In READ COMMITTED mode, a SELECT FOR UPDATE that is scanning an inheritance tree could incorrectly return a row from a prior child table instead of the one it should return from a later child table.
•
Improve performance of EXPLAIN with large range tables (Tom Lane)
•
Reject duplicate column names in the referenced-columns list of a FOREIGN KEY declaration (David Rowley) This restriction is per SQL standard. Previously we did not reject the case explicitly, but later on the code would fail with bizarre-looking errors.
•
Re-enable error for SELECT ... OFFSET -1 (Tom Lane) A negative offset value has been an error since 8.4, but an optimization added in 9.3 accidentally turned the case into a no-op. Restore the expected behavior.
•
Restore previous behavior of conversion of domains to JSON (Tom Lane) This change causes domains over numeric and boolean to be treated like their base types for purposes of conversion to JSON. It worked like that before 9.3.5 and 9.2.9, but was unintentionally changed while fixing a related problem.
•
Fix json_agg() to not return extra trailing right brackets in its result (Tom Lane)
•
Fix bugs in raising a numeric value to a large integral power (Tom Lane) The previous code could get a wrong answer, or consume excessive amounts of time and memory before realizing that the answer must overflow.
•
In numeric_recv(), truncate away any fractional digits that would be hidden according to the value’s dscale field (Tom Lane) A numeric value’s display scale (dscale) should never be less than the number of nonzero fractional digits; but apparently there’s at least one broken client application that transmits binary numeric values in which that’s true. This leads to strange behavior since the extra digits are taken into account by arithmetic operations even though they aren’t printed. The least risky fix seems to be to truncate away such “hidden” digits on receipt, so that the value is indeed what it prints as.
•
Fix incorrect search for shortest-first regular expression matches (Tom Lane) Matching would often fail when the number of allowed iterations is limited by a ? quantifier or a bound expression.
•
Reject out-of-range numeric timezone specifications (Tom Lane) Simple numeric timezone specifications exceeding +/- 168 hours (one week) would be accepted, but could then cause null-pointer dereference crashes in certain operations. There’s no use-case for such large UTC offsets, so reject them.
•
Fix bugs in tsquery @> tsquery operator (Heikki Linnakangas)
2035
Appendix E. Release Notes Two different terms would be considered to match if they had the same CRC. Also, if the second operand had more terms than the first, it would be assumed not to be contained in the first; which is wrong since it might contain duplicate terms. •
Improve ispell dictionary’s defenses against bad affix files (Tom Lane)
•
Allow more than 64K phrases in a thesaurus dictionary (David Boutin) The previous coding could crash on an oversize dictionary, so this was deemed a back-patchable bug fix rather than a feature addition.
•
Fix namespace handling in xpath() (Ali Akbar) Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation.
•
Ensure that whole-row variables expose nonempty column names to functions that pay attention to column names within composite arguments (Tom Lane) In some contexts, constructs like row_to_json(tab.*) may not produce the expected column names. This is fixed properly as of 9.4; in older branches, just ensure that we produce some nonempty name. (In some cases this will be the underlying table’s column name rather than the query-assigned alias that should theoretically be visible.)
•
Fix mishandling of system columns, particularly tableoid, in FDW queries (Etsuro Fujita)
•
Fix assorted oversights in range-operator selectivity estimation (Emre Hasegeli) This patch fixes corner-case “unexpected operator NNNN” planner errors, and improves the selectivity estimates for some other cases.
•
Avoid doing indexed_column = ANY (array ) as an index qualifier if that leads to an inferior plan (Andrew Gierth) In some cases, = ANY conditions applied to non-first index columns would be done as index conditions even though it would be better to use them as simple filter conditions.
•
Fix “variable not found in subplan target list” planner failure when an inline-able SQL function taking a composite argument is used in a LATERAL subselect and the composite argument is a lateral reference (Tom Lane)
•
Fix planner problems with nested append relations, such as inherited tables within UNION ALL subqueries (Tom Lane)
•
Fail cleanly when a GiST index tuple doesn’t fit on a page, rather than going into infinite recursion (Andrew Gierth)
•
Exempt tables that have per-table cost_limit and/or cost_delay settings from autovacuum’s global cost balancing rules (Álvaro Herrera) The previous behavior resulted in basically ignoring these per-table settings, which was unintended. Now, a table having such settings will be vacuumed using those settings, independently of what is going on in other autovacuum workers. This may result in heavier total I/O load than before, so such settings should be re-examined for sanity.
•
Avoid wholesale autovacuuming when autovacuum is nominally off (Tom Lane) Even when autovacuum is nominally off, we will still launch autovacuum worker processes to vacuum tables that are at risk of XID wraparound. However, such a worker process then proceeded to vacuum all tables in the target database, if they met the usual thresholds for autovacuuming. This
2036
Appendix E. Release Notes is at best pretty unexpected; at worst it delays response to the wraparound threat. Fix it so that if autovacuum is turned off, workers only do anti-wraparound vacuums and not any other work. •
During crash recovery, ensure that unlogged relations are rewritten as empty and are synced to disk before recovery is considered complete (Abhijit Menon-Sen, Andres Freund) This prevents scenarios in which unlogged relations might contain garbage data following database crash recovery.
•
Fix race condition between hot standby queries and replaying a full-page image (Heikki Linnakangas) This mistake could result in transient errors in queries being executed in hot standby.
•
Fix several cases where recovery logic improperly ignored WAL records for COMMIT/ABORT PREPARED (Heikki Linnakangas)
The most notable oversight was that recovery_target_xid could not be used to stop at a twophase commit. •
Prevent latest WAL file from being archived a second time at completion of crash recovery (Fujii Masao)
•
Avoid creating unnecessary .ready marker files for timeline history files (Fujii Masao)
•
Fix possible null pointer dereference when an empty prepared statement is used and the log_statement setting is mod or ddl (Fujii Masao)
•
Change “pgstat wait timeout” warning message to be LOG level, and rephrase it to be more understandable (Tom Lane) This message was originally thought to be essentially a can’t-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads “using stale statistics instead of current ones because stats collector is not responding”.
•
Fix possible corruption of postmaster’s list of dynamic background workers (Andres Freund)
•
Fix SPARC spinlock implementation to ensure correctness if the CPU is being run in a non-TSO coherency mode, as some non-Solaris kernels do (Andres Freund)
•
Warn if macOS’s setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch)
•
Fix processing of repeated dbname parameters in PQconnectdbParams() (Alex Shulgin) Unexpected behavior ensued if the first occurrence of dbname contained a connection string or URI to be expanded.
•
Ensure that libpq reports a suitable error message on unexpected socket EOF (Marko Tiikkaja, Tom Lane) Depending on kernel behavior, libpq might return an empty error string rather than something useful when the server unexpectedly closed the socket.
•
Clear any old error message during PQreset() (Heikki Linnakangas) If PQreset() is called repeatedly, and the connection cannot be re-established, error messages from the failed connection attempts kept accumulating in the PGconn’s error string.
•
Properly handle out-of-memory conditions while parsing connection options in libpq (Alex Shulgin, Heikki Linnakangas)
•
Fix array overrun in ecpg’s version of ParseDateTime() (Michael Paquier)
2037
Appendix E. Release Notes •
In initdb, give a clearer error message if a password file is specified but is empty (Mats Erik Andersson)
•
Fix psql’s \s command to work nicely with libedit, and add pager support (Stepan Rutz, Tom Lane) When using libedit rather than readline, \s printed the command history in a fairly unreadable encoded format, and on recent libedit versions might fail altogether. Fix that by printing the history ourselves rather than having the library do it. A pleasant side-effect is that the pager is used if appropriate. This patch also fixes a bug that caused newline encoding to be applied inconsistently when saving the command history with libedit. Multiline history entries written by older psql versions will be read cleanly with this patch, but perhaps not vice versa, depending on the exact libedit versions involved.
•
Improve consistency of parsing of psql’s special variables (Tom Lane) Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors.
• •
Make psql’s \watch command display nulls as specified by \pset null (Fujii Masao) Fix psql’s expanded-mode display to work consistently when using border = 3 and linestyle = ascii or unicode (Stephen Frost)
•
Fix pg_dump to handle comments on event triggers without failing (Tom Lane)
•
Allow parallel pg_dump to use --serializable-deferrable (Kevin Grittner)
•
Improve performance of pg_dump when the database contains many instances of multiple dependency paths between the same two objects (Tom Lane)
•
Fix pg_dumpall to restore its ability to dump from pre-8.1 servers (Gilles Darold)
•
Fix possible deadlock during parallel restore of a schema-only dump (Robert Haas, Tom Lane)
•
Fix core dump in pg_dump --binary-upgrade on zero-column composite type (Rushabh Lathia)
•
Fix failure to fsync tables in nondefault tablespaces during pg_upgrade (Abhijit Menon-Sen, Andres Freund) With an operating system crash and some bad luck, this could result in data loss during an upgrade.
•
In pg_upgrade, cope with cases where the new cluster creates a TOAST table for a table that didn’t previously have one (Bruce Momjian) Previously this could result in failures due to OID conflicts.
•
In pg_upgrade, don’t try to set autovacuum_multixact_freeze_max_age for the old cluster (Bruce Momjian) This could result in failure because not all 9.3.X versions have that parameter. Fortunately, we don’t actually need to set it at all.
•
In pg_upgrade, preserve the transaction ID epoch (Bruce Momjian) This oversight did not bother PostgreSQL proper, but could confuse some external replication tools.
•
Prevent WAL files created by pg_basebackup -x/-X from being archived again when the standby is promoted (Andres Freund)
•
Fix memory leak in pg_receivexlog (Fujii Masao)
2038
Appendix E. Release Notes •
Fix unintended suppression of pg_receivexlog verbose messages (Fujii Masao)
•
Fix failure of contrib/auto_explain to print per-node timing information when doing EXPLAIN ANALYZE (Tom Lane)
•
Fix upgrade-from-unpackaged script for contrib/citext (Tom Lane)
•
Avoid integer overflow and buffer overrun in contrib/hstore’s hstore_to_json() (Heikki Linnakangas)
•
Fix recognition of numbers in hstore_to_json_loose(), so that JSON numbers and strings are correctly distinguished (Andrew Dunstan)
•
Fix block number checking in contrib/pageinspect’s get_raw_page() (Tom Lane) The incorrect checking logic could prevent access to some pages in non-main relation forks.
•
Fix contrib/pgcrypto’s pgp_sym_decrypt() to not fail on messages whose length is 6 less than a power of 2 (Marko Tiikkaja)
•
Fix file descriptor leak in contrib/pg_test_fsync (Jeff Janes) This could cause failure to remove temporary files on Windows.
•
Handle unexpected query results, especially NULLs, safely in contrib/tablefunc’s connectby() (Michael Paquier) connectby() previously crashed if it encountered a NULL key value. It now prints that row but
doesn’t recurse further. •
Avoid a possible crash in contrib/xml2’s xslt_process() (Mark Simonetti) libxslt seems to have an undocumented dependency on the order in which resources are freed; reorder our calls to avoid a crash.
•
Mark some contrib I/O functions with correct volatility properties (Tom Lane) The previous over-conservative marking was immaterial in normal use, but could cause optimization problems or rejection of valid index expression definitions. Since the consequences are not large, we’ve just adjusted the function definitions in the extension modules’ scripts, without changing version numbers.
•
Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier) These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues.
•
Fix setup of background workers in EXEC_BACKEND builds, eg Windows (Robert Haas)
•
Detect incompatible OpenLDAP versions during build (Noah Misch) With OpenLDAP versions 2.4.24 through 2.4.31, inclusive, PostgreSQL backends can crash at exit. Raise a warning during configure based on the compile-time OpenLDAP version number, and test the crashing scenario in the contrib/dblink regression test.
•
In non-MSVC Windows builds, ensure libpq.dll is installed with execute permissions (Noah Misch)
•
Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane) This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations.
•
Support time zone abbreviations that change UTC offset from time to time (Tom Lane)
2039
Appendix E. Release Notes Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date. •
Update time zone abbreviations lists (Tom Lane) Add CST (China Standard Time) to our lists. Remove references to ADT as “Arabia Daylight Time”, an abbreviation that’s been out of use since 2007; therefore, claiming there is a conflict with “Atlantic Daylight Time” doesn’t seem especially helpful. Fix entirely incorrect GMT offsets for CKT (Cook Islands), FJT, and FJST (Fiji); we didn’t even have them on the proper side of the date line.
•
Update time zone data files to tzdata release 2015a. The IANA timezone database has adopted abbreviations of the form Ax ST/Ax DT for all Australian time zones, reflecting what they believe to be current majority practice Down Under. These names do not conflict with usage elsewhere (other than ACST for Acre Summer Time, which has been in disuse since 1994). Accordingly, adopt these names into our “Default” timezone abbreviation set. The “Australia” abbreviation set now contains only CST, EAST, EST, SAST, SAT, and WST, all of which are thought to be mostly historical usage. Note that SAST has also been changed to be South Africa Standard Time in the “Default” abbreviation set. Also, add zone abbreviations SRET (Asia/Srednekolymsk) and XJT (Asia/Urumqi), and use WSST/WSDT for western Samoa. Also, there were DST law changes in Chile, Mexico, the Turks & Caicos Islands (America/Grand_Turk), and Fiji. There is a new zone Pacific/Bougainville for portions of Papua New Guinea. Also, numerous corrections for historical (pre-1970) time zone data.
E.15. Release 9.3.5 Release date: 2014-07-24 This release contains a variety of fixes from 9.3.4. For information about new features in the 9.3 major release, see Section E.20.
E.15.1. Migration to Version 9.3.5 A dump/restore is not required for those running 9.3.X. However, this release corrects a logic error in pg_upgrade, as well as an index corruption problem in some GiST indexes. See the first two changelog entries below to find out whether your installation has been affected and what steps you should take if so. Also, if you are upgrading from a version earlier than 9.3.4, see Section E.16.
2040
Appendix E. Release Notes
E.15.2. Changes •
In pg_upgrade, remove pg_multixact files left behind by initdb (Bruce Momjian) If you used a pre-9.3.5 version of pg_upgrade to upgrade a database cluster to 9.3, it might have left behind a file $PGDATA/pg_multixact/offsets/0000 that should not be there and will eventually cause problems in VACUUM. However, in common cases this file is actually valid and must not be removed. To determine whether your installation has this problem, run this query as superuser, in any database of the cluster: WITH list(file) AS (SELECT * FROM pg_ls_dir(’pg_multixact/offsets’)) SELECT EXISTS (SELECT * FROM list WHERE file = ’0000’) AND NOT EXISTS (SELECT * FROM list WHERE file = ’0001’) AND NOT EXISTS (SELECT * FROM list WHERE file = ’FFFF’) AND EXISTS (SELECT * FROM list WHERE file != ’0000’) AS file_0000_removal_required; If this query returns t, manually remove the file $PGDATA/pg_multixact/offsets/0000. Do nothing if the query returns f.
•
Correctly initialize padding bytes in contrib/btree_gist indexes on bit columns (Heikki Linnakangas) This error could result in incorrect query results due to values that should compare equal not being seen as equal. Users with GiST indexes on bit or bit varying columns should REINDEX those indexes after installing this update.
•
Protect against torn pages when deleting GIN list pages (Heikki Linnakangas) This fix prevents possible index corruption if a system crash occurs while the page update is being written to disk.
•
Don’t clear the right-link of a GiST index page while replaying updates from WAL (Heikki Linnakangas) This error could lead to transiently wrong answers from GiST index scans performed in Hot Standby.
•
Fix corner-case infinite loop during insertion into an SP-GiST text index (Tom Lane)
•
Fix incorrect answers from SP-GiST index searches with -|- (range adjacency) operator (Heikki Linnakangas)
•
Fix wraparound handling for pg_multixact/members (Álvaro Herrera)
•
Truncate pg_multixact during checkpoints, not during VACUUM (Álvaro Herrera) This change ensures that pg_multixact segments can’t be removed if they’d still be needed during WAL replay after a crash.
•
Fix possible inconsistency of all-visible flags after WAL recovery (Heikki Linnakangas)
•
Fix
possibly-incorrect
cache
invalidation
during
nested
calls
to
ReceiveSharedInvalidMessages (Andres Freund) •
Fix race condition when updating a tuple concurrently locked by another process (Andres Freund, Álvaro Herrera)
•
Fix “could not find pathkey item to sort” planner failures with UNION ALL over subqueries reading from tables with inheritance children (Tom Lane)
•
Don’t assume a subquery’s output is unique if there’s a set-returning function in its targetlist (David Rowley)
2041
Appendix E. Release Notes This oversight could lead to misoptimization of constructs like WHERE x IN (SELECT y, generate_series(1,10) FROM t GROUP BY y). •
Improve planner to drop constant-NULL inputs of AND/OR when possible (Tom Lane) This change fixes some cases where the more aggressive parameter substitution done by 9.2 and later can lead to a worse plan than older versions produced.
•
Ensure that the planner sees equivalent VARIADIC and non-VARIADIC function calls as equivalent (Tom Lane) This bug could for example result in failure to use expression indexes involving variadic functions. It might be necessary to re-create such indexes, and/or re-create views including variadic function calls that should match the indexes, for the fix to be effective for existing 9.3 installations.
•
Fix handling of nested JSON objects in json_populate_recordset() and friends (Michael Paquier, Tom Lane) A nested JSON object could result in previous fields of the parent object not being shown in the output.
•
Fix identification of input type category in to_json() and friends (Tom Lane) This is known to have led to inadequate quoting of money fields in the JSON result, and there may have been wrong results for other data types as well.
•
Fix failure to detoast fields in composite elements of structured types (Tom Lane) This corrects cases where TOAST pointers could be copied into other tables without being dereferenced. If the original data is later deleted, it would lead to errors like “missing chunk number 0 for toast value ...” when the now-dangling pointer is used.
•
Fix “record type has not been registered” failures with whole-row references to the output of Append plan nodes (Tom Lane)
•
Fix possible crash when invoking a user-defined function while rewinding a cursor (Tom Lane)
•
Fix query-lifespan memory leak while evaluating the arguments for a function in FROM (Tom Lane)
•
Fix session-lifespan memory leaks in regular-expression processing (Tom Lane, Arthur O’Dwyer, Greg Stark)
•
Fix data encoding error in hungarian.stop (Tom Lane)
•
Prevent foreign tables from being created with OIDS when default_with_oids is true (Etsuro Fujita)
•
Fix liveness checks for rows that were inserted in the current transaction and then deleted by a now-rolled-back subtransaction (Andres Freund) This could cause problems (at least spurious warnings, and at worst an infinite loop) if CREATE INDEX or CLUSTER were done later in the same transaction.
•
Clear pg_stat_activity.xact_start during PREPARE TRANSACTION (Andres Freund) After the PREPARE, the originating session is no longer in a transaction, so it should not continue to display a transaction start time.
•
Fix REASSIGN OWNED to not fail for text search objects (Álvaro Herrera)
•
Prevent pg_class.relminmxid values from going backwards during VACUUM FULL (Álvaro Herrera)
•
Reduce indentation in rule/view dumps to improve readability and avoid excessive whitespace (Greg Stark, Tom Lane)
2042
Appendix E. Release Notes This change reduces the amount of indentation applied to nested constructs, including some cases that the user probably doesn’t think of as nested, such as UNION lists. Previously, deeply nested constructs were printed with an amount of whitespace growing as O(N^2), which created a performance problem and even risk of out-of-memory failures. Now the indentation is reduced modulo 40, which is initially odd to look at but seems to preserve readability better than simply limiting the indentation would do. Redundant parenthesization of UNION lists has been reduced as well. •
Fix dumping of rules/views when subsequent addition of a column has resulted in multiple input columns matching a USING specification (Tom Lane)
•
Repair view printing for some cases involving functions in FROM that return a composite type containing dropped columns (Tom Lane)
•
Block signals during postmaster startup (Tom Lane) This ensures that the postmaster will properly clean up after itself if, for example, it receives SIGINT while still starting up.
•
Fix client host name lookup when processing pg_hba.conf entries that specify host names instead of IP addresses (Tom Lane) Ensure that reverse-DNS lookup failures are reported, instead of just silently not matching such entries. Also ensure that we make only one reverse-DNS lookup attempt per connection, not one per host name entry, which is what previously happened if the lookup attempts failed.
•
Allow the root user to use postgres -C variable and postgres --describe-config (MauMau) The prohibition on starting the server as root does not need to extend to these operations, and relaxing it prevents failure of pg_ctl in some scenarios.
•
Secure Unix-domain sockets of temporary postmasters started during make check (Noah Misch) Any local user able to access the socket file could connect as the server’s bootstrap superuser, then proceed to execute arbitrary code as the operating-system user running the test, as we previously noted in CVE-2014-0067. This change defends against that risk by placing the server’s socket in a temporary, mode 0700 subdirectory of /tmp. The hazard remains however on platforms where Unix sockets are not supported, notably Windows, because then the temporary postmaster must accept local TCP connections. A useful side effect of this change is to simplify make check testing in builds that override DEFAULT_PGSOCKET_DIR. Popular non-default values like /var/run/postgresql are often not writable by the build user, requiring workarounds that will no longer be necessary.
•
Fix tablespace creation WAL replay to work on Windows (MauMau)
•
Fix detection of socket creation failures on Windows (Bruce Momjian)
•
On Windows, allow new sessions to absorb values of PGC_BACKEND parameters (such as log_connections) from the configuration file (Amit Kapila) Previously, if such a parameter were changed in the file post-startup, the change would have no effect.
•
Properly quote executable path names on Windows (Nikhil Deshpande) This oversight could cause initdb and pg_upgrade to fail on Windows, if the installation path contained both spaces and @ signs.
•
Fix linking of libpython on macOS (Tom Lane) The method we previously used can fail with the Python library supplied by Xcode 5.0 and later.
2043
Appendix E. Release Notes •
Avoid buffer bloat in libpq when the server consistently sends data faster than the client can absorb it (Shin-ichi Morita, Tom Lane) libpq could be coerced into enlarging its input buffer until it runs out of memory (which would be reported misleadingly as “lost synchronization with server”). Under ordinary circumstances it’s quite far-fetched that data could be continuously transmitted more quickly than the recv() loop can absorb it, but this has been observed when the client is artificially slowed by scheduler constraints.
•
Ensure that LDAP lookup attempts in libpq time out as intended (Laurenz Albe)
•
Fix ecpg to do the right thing when an array of char * is the target for a FETCH statement returning more than one row, as well as some other array-handling fixes (Ashutosh Bapat)
•
Fix pg_dump to cope with a materialized view that depends on a table’s primary key (Tom Lane) This occurs if the view’s query relies on functional dependency to abbreviate a GROUP BY list. pg_dump got sufficiently confused that it dumped the materialized view as a regular view.
•
Fix parsing of pg_dumpall’s -i switch (Tom Lane)
•
Fix pg_restore’s processing of old-style large object comments (Tom Lane) A direct-to-database restore from an archive file generated by a pre-9.0 version of pg_dump would usually fail if the archive contained more than a few comments for large objects.
•
Fix pg_upgrade for cases where the new server creates a TOAST table but the old version did not (Bruce Momjian) This rare situation would manifest as “relation OID mismatch” errors.
•
In pg_upgrade, preserve pg_database.datminmxid and pg_class.relminmxid values from the old cluster, or insert reasonable values when upgrading from pre-9.3; also defend against unreasonable values in the core server (Bruce Momjian, Álvaro Herrera, Tom Lane) These changes prevent scenarios in which autovacuum might insist on scanning the entire cluster’s contents immediately upon starting the new cluster, or in which tracking of unfrozen MXID values might be disabled completely.
•
Prevent contrib/auto_explain from changing the output of a user’s EXPLAIN (Tom Lane) If auto_explain is active, it could cause an EXPLAIN (ANALYZE, TIMING OFF) command to nonetheless print timing information.
•
Fix query-lifespan memory leak in contrib/dblink (MauMau, Joe Conway)
•
In contrib/pgcrypto functions, ensure sensitive information is cleared from stack variables before returning (Marko Kreen)
•
Prevent use of already-freed memory in contrib/pgstattuple’s pgstat_heap() (Noah Misch)
•
In contrib/uuid-ossp, cache the state of the OSSP UUID library across calls (Tom Lane) This improves the efficiency of UUID generation and reduces the amount of entropy drawn from /dev/urandom, on platforms that have that.
•
Update time zone data files to tzdata release 2014e for DST law changes in Crimea, Egypt, and Morocco.
2044
Appendix E. Release Notes
E.16. Release 9.3.4 Release date: 2014-03-20 This release contains a variety of fixes from 9.3.3. For information about new features in the 9.3 major release, see Section E.20.
E.16.1. Migration to Version 9.3.4 A dump/restore is not required for those running 9.3.X. However, the error fixed in the first changelog entry below could have resulted in corrupt data on standby servers. It may be prudent to reinitialize standby servers from fresh base backups after installing this update. Also, if you are upgrading from a version earlier than 9.3.3, see Section E.17.
E.16.2. Changes •
Fix WAL replay of locking an already-updated tuple (Andres Freund, Álvaro Herrera) This error caused updated rows to not be found by index scans, resulting in inconsistent query results depending on whether an index scan was used. Subsequent processing could result in constraint violations, since the previously updated row would not be found by later index searches, thus possibly allowing conflicting rows to be inserted. Since this error is in WAL replay, it would only manifest during crash recovery or on standby servers. The improperly-replayed case most commonly arises when a table row that is referenced by a foreign-key constraint is updated concurrently with creation of a referencing row.
•
Restore GIN metapages unconditionally to avoid torn-page risk (Heikki Linnakangas) Although this oversight could theoretically result in a corrupted index, it is unlikely to have caused any problems in practice, since the active part of a GIN metapage is smaller than a standard 512byte disk sector.
•
Avoid race condition in checking transaction commit status during receipt of a NOTIFY message (Marko Tiikkaja) This prevents a scenario wherein a sufficiently fast client might respond to a notification before database updates made by the notifier have become visible to the recipient.
•
Allow materialized views to be referenced in UPDATE and DELETE commands (Michael Paquier) Previously such queries failed with a complaint about not being able to lock rows in the materialized view.
•
Allow regular-expression operators to be terminated early by query cancel requests (Tom Lane) This prevents scenarios wherein a pathological regular expression could lock up a server process uninterruptibly for a long time.
•
Remove incorrect code that tried to allow OVERLAPS with single-element row arguments (Joshua Yanovski) This code never worked correctly, and since the case is neither specified by the SQL standard nor documented, it seemed better to remove it than fix it.
•
Avoid getting more than AccessShareLock when de-parsing a rule or view (Dean Rasheed)
2045
Appendix E. Release Notes This oversight resulted in pg_dump unexpectedly acquiring RowExclusiveLock locks on tables mentioned as the targets of INSERT/UPDATE/DELETE commands in rules. While usually harmless, that could interfere with concurrent transactions that tried to acquire, for example, ShareLock on those tables. •
Improve performance of index endpoint probes during planning (Tom Lane) This change fixes a significant performance problem that occurred when there were many not-yetcommitted rows at the end of the index, which is a common situation for indexes on sequentiallyassigned values such as timestamps or sequence-generated identifiers.
•
Use non-default selectivity estimates for value IN (list) and value operator ANY (array ) expressions when the righthand side is a stable expression (Tom Lane)
•
Remove the correct per-database statistics file during DROP DATABASE (Tomas Vondra) This fix prevents a permanent leak of statistics file space. Users who have done many DROP DATABASE commands since upgrading to PostgreSQL 9.3 may wish to check their statistics directory and delete statistics files that do not correspond to any existing database. Please note that db_0.stat should not be removed.
•
Fix walsender ping logic to avoid inappropriate disconnects under continuous load (Andres Freund, Heikki Linnakangas) walsender failed to send ping messages to the client if it was constantly busy sending WAL data; but it expected to see ping responses despite that, and would therefore disconnect once wal_sender_timeout elapsed.
•
Fix walsender’s failure to shut down cleanly when client is pg_receivexlog (Fujii Masao)
•
Check WAL level and hot standby parameters correctly when doing crash recovery that will be followed by archive recovery (Heikki Linnakangas)
•
Fix test to see if hot standby connections can be allowed immediately after a crash (Heikki Linnakangas)
•
Add read-only data_checksums parameter to display whether page checksums are enabled (Heikki Linnakangas) Without this parameter, determining the state of checksum processing was difficult.
•
Prevent interrupts while reporting non-ERROR messages (Tom Lane) This guards against rare server-process freezeups due to recursive entry to syslog(), and perhaps other related problems.
•
Fix memory leak in PL/Perl when returning a composite result, including multiple-OUT-parameter cases (Alex Hunsaker)
•
Fix tracking of psql script line numbers during \copy from out-of-line data (Kumar Rajeev Rastogi, Amit Khandekar) \copy ... from incremented the script file line number for each data line, even if the data was
not coming from the script file. This mistake resulted in wrong line numbers being reported for any errors occurring later in the same script file. •
Fix contrib/postgres_fdw to handle multiple join conditions properly (Tom Lane) This oversight could result in sending WHERE clauses to the remote server for execution even though the clauses are not known to have the same semantics on the remote server (for example, clauses that use non-built-in operators). The query might succeed anyway, but it could also fail with errors from the remote server, or worse give silently wrong answers.
2046
Appendix E. Release Notes •
Prevent intermittent “could not reserve shared memory region” failures on recent Windows versions (MauMau)
•
Update time zone data files to tzdata release 2014a for DST law changes in Fiji and Turkey, plus historical changes in Israel and Ukraine.
E.17. Release 9.3.3 Release date: 2014-02-20 This release contains a variety of fixes from 9.3.2. For information about new features in the 9.3 major release, see Section E.20.
E.17.1. Migration to Version 9.3.3 A dump/restore is not required for those running 9.3.X. However, several of the issues corrected in this release could have resulted in corruption of foreignkey constraints; that is, there might now be referencing rows for which there is no matching row in the referenced table. It may be worthwhile to recheck such constraints after installing this update. The simplest way to do that is to drop and recreate each suspect constraint; however, that will require taking an exclusive lock on both tables, so it is unlikely to be acceptable in production databases. Alternatively, you can do a manual join query between the two tables to look for unmatched rows. Note also the requirement for replication standby servers to be upgraded before their master server is upgraded. Also, if you are upgrading from a version earlier than 9.3.2, see Section E.18.
E.17.2. Changes •
Shore up GRANT ... WITH ADMIN OPTION restrictions (Noah Misch) Granting a role without ADMIN OPTION is supposed to prevent the grantee from adding or removing members from the granted role, but this restriction was easily bypassed by doing SET ROLE first. The security impact is mostly that a role member can revoke the access of others, contrary to the wishes of his grantor. Unapproved role member additions are a lesser concern, since an uncooperative role member could provide most of his rights to others anyway by creating views or SECURITY DEFINER functions. (CVE-2014-0060)
•
Prevent privilege escalation via manual calls to PL validator functions (Andres Freund) The primary role of PL validator functions is to be called implicitly during CREATE FUNCTION, but they are also normal SQL functions that a user can call explicitly. Calling a validator on a function actually written in some other language was not checked for and could be exploited for privilege-escalation purposes. The fix involves adding a call to a privilege-checking function in each validator function. Non-core procedural languages will also need to make this change to their own validator functions, if any. (CVE-2014-0061)
•
Avoid multiple name lookups during table and index DDL (Robert Haas, Andres Freund) If the name lookups come to different conclusions due to concurrent activity, we might perform some parts of the DDL on a different table than other parts. At least in the case of CREATE INDEX,
2047
Appendix E. Release Notes this can be used to cause the permissions checks to be performed against a different table than the index creation, allowing for a privilege escalation attack. (CVE-2014-0062) •
Prevent buffer overrun with long datetime strings (Noah Misch) The MAXDATELEN constant was too small for the longest possible value of type interval, allowing a buffer overrun in interval_out(). Although the datetime input functions were more careful about avoiding buffer overrun, the limit was short enough to cause them to reject some valid inputs, such as input containing a very long timezone name. The ecpg library contained these vulnerabilities along with some of its own. (CVE-2014-0063)
•
Prevent buffer overrun due to integer overflow in size calculations (Noah Misch, Heikki Linnakangas) Several functions, mostly type input functions, calculated an allocation size without checking for overflow. If overflow did occur, a too-small buffer would be allocated and then written past. (CVE2014-0064)
•
Prevent overruns of fixed-size buffers (Peter Eisentraut, Jozef Mlich) Use strlcpy() and related functions to provide a clear guarantee that fixed-size buffers are not overrun. Unlike the preceding items, it is unclear whether these cases really represent live issues, since in most cases there appear to be previous constraints on the size of the input string. Nonetheless it seems prudent to silence all Coverity warnings of this type. (CVE-2014-0065)
•
Avoid crashing if crypt() returns NULL (Honza Horak, Bruce Momjian) There are relatively few scenarios in which crypt() could return NULL, but contrib/chkpass would crash if it did. One practical case in which this could be an issue is if libc is configured to refuse to execute unapproved hashing algorithms (e.g., “FIPS mode”). (CVE-2014-0066)
•
Document risks of make check in the regression testing instructions (Noah Misch, Tom Lane) Since the temporary server started by make check uses “trust” authentication, another user on the same machine could connect to it as database superuser, and then potentially exploit the privileges of the operating-system user who started the tests. A future release will probably incorporate changes in the testing procedure to prevent this risk, but some public discussion is needed first. So for the moment, just warn people against using make check when there are untrusted users on the same machine. (CVE-2014-0067)
•
Rework tuple freezing protocol (Álvaro Herrera, Andres Freund) The logic for tuple freezing was unable to handle some cases involving freezing of multixact IDs, with the practical effect that shared row-level locks might be forgotten once old enough. Fixing this required changing the WAL record format for tuple freezing. While this is no issue for standalone servers, when using replication it means that standby servers must be upgraded to 9.3.3 or later before their masters are. An older standby will be unable to interpret freeze records generated by a newer master, and will fail with a PANIC message. (In such a case, upgrading the standby should be sufficient to let it resume execution.)
•
Create separate GUC parameters to control multixact freezing (Álvaro Herrera) 9.3 requires multixact tuple labels to be frozen before they grow too old, in the same fashion as plain transaction ID labels have been frozen for some time. Previously, the transaction ID freezing parameters were used for multixact IDs too; but since the consumption rates of transaction IDs and multixact IDs can be quite different, this did not work very well. Introduce new settings vacuum_multixact_freeze_min_age, vacuum_multixact_freeze_table_age, and autovacuum_multixact_freeze_max_age to control when to freeze multixacts.
•
Account for remote row locks propagated by local updates (Álvaro Herrera)
2048
Appendix E. Release Notes If a row was locked by transaction A, and transaction B updated it, the new version of the row created by B would be locked by A, yet visible only to B. If transaction B then again updated the row, A’s lock wouldn’t get checked, thus possibly allowing B to complete when it shouldn’t. This case is new in 9.3 since prior versions did not have any types of row locking that would permit another transaction to update the row at all. This oversight could allow referential integrity checks to give false positives (for instance, allow deletes that should have been rejected). Applications using the new commands SELECT FOR KEY SHARE and SELECT FOR NO KEY UPDATE might also have suffered locking failures of this kind. •
Prevent “forgetting” valid row locks when one of several holders of a row lock aborts (Álvaro Herrera) This was yet another mechanism by which a shared row lock could be lost, thus possibly allowing updates that should have been prevented by foreign-key constraints.
•
Fix incorrect logic during update chain locking (Álvaro Herrera) This mistake could result in spurious “could not serialize access due to concurrent update” errors in REPEATABLE READ and SERIALIZABLE transaction isolation modes.
•
Handle wraparound correctly during extension or truncation of pg_multixact/members (Andres Freund, Álvaro Herrera)
•
Fix handling of 5-digit filenames in pg_multixact/members (Álvaro Herrera) As of 9.3, these names can be more than 4 digits, but the directory cleanup code ignored such files.
•
Improve performance of multixact cache code (Álvaro Herrera)
•
Optimize updating a row that’s already locked by the same transaction (Andres Freund, Álvaro Herrera) This fixes a performance regression from pre-9.3 versions when doing SELECT FOR UPDATE followed by UPDATE/DELETE.
•
During archive recovery, prefer highest timeline number when WAL segments with the same ID are present in both the archive and pg_xlog/ (Kyotaro Horiguchi) Previously, not-yet-archived segments could get ignored during recovery. This reverts an undesirable behavioral change in 9.3.0 back to the way things worked pre-9.3.
•
Fix possible mis-replay of WAL records when some segments of a relation aren’t full size (Greg Stark, Tom Lane) The WAL update could be applied to the wrong page, potentially many pages past where it should have been. Aside from corrupting data, this error has been observed to result in significant “bloat” of standby servers compared to their masters, due to updates being applied far beyond where the end-of-file should have been. This failure mode does not appear to be a significant risk during crash recovery, only when initially synchronizing a standby created from a base backup taken from a quickly-changing master.
•
Fix bug in determining when recovery has reached consistency (Tomonari Katsumata, Heikki Linnakangas) In some cases WAL replay would mistakenly conclude that the database was already consistent at the start of replay, thus possibly allowing hot-standby queries before the database was really consistent. Other symptoms such as “PANIC: WAL contains references to invalid pages” were also possible.
•
Fix WAL logging of visibility map changes (Heikki Linnakangas)
2049
Appendix E. Release Notes •
Fix improper locking of btree index pages while replaying a VACUUM operation in hot-standby mode (Andres Freund, Heikki Linnakangas, Tom Lane) This error could result in “PANIC: WAL contains references to invalid pages” failures.
•
Ensure that insertions into non-leaf GIN index pages write a full-page WAL record when appropriate (Heikki Linnakangas) The previous coding risked index corruption in the event of a partial-page write during a system crash.
•
When pause_at_recovery_target and recovery_target_inclusive are both set, ensure the target record is applied before pausing, not after (Heikki Linnakangas)
•
Ensure walreceiver sends hot-standby feedback messages on time even when there is a continuous stream of data (Andres Freund, Amit Kapila)
•
Prevent
timeout
interrupts
from
taking
control
away
from
mainline
code
unless
ImmediateInterruptOK is set (Andres Freund, Tom Lane)
This is a serious issue for any application making use of statement timeouts, as it could cause all manner of strange failures after a timeout occurred. We have seen reports of “stuck” spinlocks, ERRORs being unexpectedly promoted to PANICs, unkillable backends, and other misbehaviors. •
Fix race conditions during server process exit (Robert Haas) Ensure that signal handlers don’t attempt to use the process’s MyProc pointer after it’s no longer valid.
•
Fix race conditions in walsender shutdown logic and walreceiver SIGHUP signal handler (Tom Lane)
•
Fix unsafe references to errno within error reporting logic (Christian Kruse) This would typically lead to odd behaviors such as missing or inappropriate HINT fields.
•
Fix possible crashes from using ereport() too early during server startup (Tom Lane) The principal case we’ve seen in the field is a crash if the server is started in a directory it doesn’t have permission to read.
•
Clear retry flags properly in OpenSSL socket write function (Alexander Kukushkin) This omission could result in a server lockup after unexpected loss of an SSL-encrypted connection.
•
Fix length checking for Unicode identifiers (U&"..." syntax) containing escapes (Tom Lane) A spurious truncation warning would be printed for such identifiers if the escaped form of the identifier was too long, but the identifier actually didn’t need truncation after de-escaping.
•
Fix parsing of Unicode literals and identifiers just before the end of a command string or function body (Tom Lane)
•
Allow keywords that are type names to be used in lists of roles (Stephen Frost) A previous patch allowed such keywords to be used without quoting in places such as role identifiers; but it missed cases where a list of role identifiers was permitted, such as DROP ROLE.
•
Fix parser crash for EXISTS(SELECT * FROM zero_column_table) (Tom Lane)
•
Fix possible crash due to invalid plan for nested sub-selects, such as WHERE (... x IN (SELECT ...) ...) IN (SELECT ...) (Tom Lane)
•
Fix mishandling of WHERE conditions pulled up from a LATERAL subquery (Tom Lane) The typical symptom of this bug was a “JOIN qualification cannot refer to other relations” error, though subtle logic errors in created plans seem possible as well.
2050
Appendix E. Release Notes •
Disallow LATERAL references to the target table of an UPDATE/DELETE (Tom Lane) While this might be allowed in some future release, it was unintentional in 9.3, and didn’t work quite right anyway.
•
Fix UPDATE/DELETE of an inherited target table that has UNION ALL subqueries (Tom Lane) Without this fix, UNION ALL subqueries aren’t correctly inserted into the update plans for inheritance child tables after the first one, typically resulting in no update happening for those child table(s).
•
Fix ANALYZE to not fail on a column that’s a domain over a range type (Tom Lane)
•
Ensure that ANALYZE creates statistics for a table column even when all the values in it are “too wide” (Tom Lane) ANALYZE intentionally omits very wide values from its histogram and most-common-values cal-
culations, but it neglected to do something sane in the case that all the sampled entries are too wide. •
In ALTER TABLE ... SET TABLESPACE, allow the database’s default tablespace to be used without a permissions check (Stephen Frost) CREATE TABLE has always allowed such usage, but ALTER TABLE didn’t get the memo.
•
Fix support for extensions containing event triggers (Tom Lane)
•
Fix “cannot accept a set” error when some arms of a CASE return a set and others don’t (Tom Lane)
•
Fix memory leakage in JSON functions (Craig Ringer)
•
Properly distinguish numbers from non-numbers when generating JSON output (Andrew Dunstan)
•
Fix checks for all-zero client addresses in pgstat functions (Kevin Grittner)
•
Fix possible misclassification of multibyte characters by the text search parser (Tom Lane) Non-ASCII characters could be misclassified when using C locale with a multibyte encoding. On Cygwin, non-C locales could fail as well.
•
Fix possible misbehavior in plainto_tsquery() (Heikki Linnakangas) Use memmove() not memcpy() for copying overlapping memory regions. There have been no field reports of this actually causing trouble, but it’s certainly risky.
•
Fix placement of permissions checks in pg_start_backup() and pg_stop_backup() (Andres Freund, Magnus Hagander) The previous coding might attempt to do catalog access when it shouldn’t.
•
Accept SHIFT_JIS as an encoding name for locale checking purposes (Tatsuo Ishii)
•
Fix *-qualification of named parameters in SQL-language functions (Tom Lane) Given a composite-type parameter named foo, $1.* worked fine, but foo.* not so much.
•
Fix misbehavior of PQhost() on Windows (Fujii Masao) It should return localhost if no host has been specified.
•
Improve error handling in libpq and psql for failures during COPY TO STDOUT/FROM STDIN (Tom Lane) In particular this fixes an infinite loop that could occur in 9.2 and up if the server connection was lost during COPY FROM STDIN. Variants of that scenario might be possible in older versions, or with other client applications.
•
Fix incorrect translation handling in some psql \d commands (Peter Eisentraut, Tom Lane)
2051
Appendix E. Release Notes •
Ensure pg_basebackup’s background process is killed when exiting its foreground process (Magnus Hagander)
•
Fix possible incorrect printing of filenames in pg_basebackup’s verbose mode (Magnus Hagander)
•
Avoid including tablespaces inside PGDATA twice in base backups (Dimitri Fontaine, Magnus Hagander)
•
Fix misaligned descriptors in ecpg (MauMau)
•
In ecpg, handle lack of a hostname in the connection parameters properly (Michael Meskes)
•
Fix performance regression in contrib/dblink connection startup (Joe Conway) Avoid an unnecessary round trip when client and server encodings match.
•
In contrib/isn, fix incorrect calculation of the check digit for ISMN values (Fabien Coelho)
•
Fix contrib/pgbench’s progress logging to avoid overflow when the scale factor is large (Tatsuo Ishii)
•
Fix contrib/pg_stat_statement’s handling of CURRENT_DATE and related constructs (Kyotaro Horiguchi)
•
Improve lost-connection error handling in contrib/postgres_fdw (Tom Lane)
•
Ensure client-code-only installation procedure works as documented (Peter Eisentraut)
•
In Mingw and Cygwin builds, install the libpq DLL in the bin directory (Andrew Dunstan) This duplicates what the MSVC build has long done. It should fix problems with programs like psql failing to start because they can’t find the DLL.
•
Avoid using the deprecated dllwrap tool in Cygwin builds (Marco Atzeri)
•
Enable building with Visual Studio 2013 (Brar Piening)
•
Don’t generate plain-text HISTORY and src/test/regress/README files anymore (Tom Lane) These text files duplicated the main HTML and PDF documentation formats. The trouble involved in maintaining them greatly outweighs the likely audience for plain-text format. Distribution tarballs will still contain files by these names, but they’ll just be stubs directing the reader to consult the main documentation. The plain-text INSTALL file will still be maintained, as there is arguably a use-case for that.
•
Update time zone data files to tzdata release 2013i for DST law changes in Jordan and historical changes in Cuba. In addition, the zones Asia/Riyadh87, Asia/Riyadh88, and Asia/Riyadh89 have been removed, as they are no longer maintained by IANA, and never represented actual civil timekeeping practice.
E.18. Release 9.3.2 Release date: 2013-12-05 This release contains a variety of fixes from 9.3.1. For information about new features in the 9.3 major release, see Section E.20.
2052
Appendix E. Release Notes
E.18.1. Migration to Version 9.3.2 A dump/restore is not required for those running 9.3.X. However, this release corrects a number of potential data corruption issues. See the first three changelog entries below to find out whether your installation has been affected and what steps you can take if so. Also, if you are upgrading from a version earlier than 9.3.1, see Section E.19.
E.18.2. Changes •
Fix VACUUM’s tests to see whether it can update relfrozenxid (Andres Freund) In some cases VACUUM (either manual or autovacuum) could incorrectly advance a table’s relfrozenxid value, allowing tuples to escape freezing, causing those rows to become invisible once 2^31 transactions have elapsed. The probability of data loss is fairly low since multiple incorrect advancements would need to happen before actual loss occurs, but it’s not zero. In 9.2.0 and later, the probability of loss is higher, and it’s also possible to get “could not access status of transaction” errors as a consequence of this bug. Users upgrading from releases 9.0.4 or 8.4.8 or earlier are not affected, but all later versions contain the bug. The issue can be ameliorated by, after upgrading, vacuuming all tables in all databases while having vacuum_freeze_table_age set to zero. This will fix any latent corruption but will not be able to fix all pre-existing data errors. However, an installation can be presumed safe after performing this vacuuming if it has executed fewer than 2^31 update transactions in its lifetime (check this with SELECT txid_current() < 2^31).
•
Fix multiple bugs in MultiXactId freezing (Andres Freund, Álvaro Herrera) These bugs could lead to “could not access status of transaction” errors, or to duplicate or vanishing rows. Users upgrading from releases prior to 9.3.0 are not affected. The issue can be ameliorated by, after upgrading, vacuuming all tables in all databases while having vacuum_freeze_table_age set to zero. This will fix latent corruption but will not be able to fix all pre-existing data errors. As a separate issue, these bugs can also cause standby servers to get out of sync with the primary, thus exhibiting data errors that are not in the primary. Therefore, it’s recommended that 9.3.0 and 9.3.1 standby servers be re-cloned from the primary (e.g., with a new base backup) after upgrading.
•
Fix initialization of pg_clog and pg_subtrans during hot standby startup (Andres Freund, Heikki Linnakangas) This bug can cause data loss on standby servers at the moment they start to accept hot-standby queries, by marking committed transactions as uncommitted. The likelihood of such corruption is small unless, at the time of standby startup, the primary server has executed many updating transactions since its last checkpoint. Symptoms include missing rows, rows that should have been deleted being still visible, and obsolete versions of updated rows being still visible alongside their newer versions. This bug was introduced in versions 9.3.0, 9.2.5, 9.1.10, and 9.0.14. Standby servers that have only been running earlier releases are not at risk. It’s recommended that standby servers that have ever run any of the buggy releases be re-cloned from the primary (e.g., with a new base backup) after upgrading.
Appendix E. Release Notes These bugs could result in incorrect behavior, such as locking or even updating the wrong row, in the presence of concurrent updates. Spurious “unable to fetch updated version of tuple” errors were also possible. •
Fix dangling-pointer problem in fast-path locking (Tom Lane) This could lead to corruption of the lock data structures in shared memory, causing “lock already held” and other odd errors.
•
Fix assorted race conditions in timeout management (Tom Lane) These errors could result in a server process becoming unresponsive because it had blocked SIGALRM and/or SIGINT.
•
Truncate pg_multixact contents during WAL replay (Andres Freund) This avoids ever-increasing disk space consumption in standby servers.
•
Ensure an anti-wraparound VACUUM counts a page as scanned when it’s only verified that no tuples need freezing (Sergey Burladyan, Jeff Janes) This bug could result in failing to advance relfrozenxid, so that the table would still be thought to need another anti-wraparound vacuum. In the worst case the database might even shut down to prevent wraparound.
•
Fix full-table-vacuum request mechanism for MultiXactIds (Andres Freund) This bug could result in large amounts of useless autovacuum activity.
•
Fix race condition in GIN index posting tree page deletion (Heikki Linnakangas) This could lead to transient wrong answers or query failures.
•
Fix “unexpected spgdoinsert() failure” error during SP-GiST index creation (Teodor Sigaev)
•
Fix assorted bugs in materialized views (Kevin Grittner, Andres Freund)
•
Re-allow duplicate table aliases if they’re within aliased JOINs (Tom Lane) Historically PostgreSQL has accepted queries like SELECT ... FROM tab1 x CROSS JOIN (tab2 x CROSS JOIN tab3 y) z
although a strict reading of the SQL standard would forbid the duplicate usage of table alias x. A misguided change in 9.3.0 caused it to reject some such cases that were formerly accepted. Restore the previous behavior. •
Avoid flattening a subquery whose SELECT list contains a volatile function wrapped inside a subSELECT (Tom Lane) This avoids unexpected results due to extra evaluations of the volatile function.
•
Fix planner’s processing of non-simple-variable subquery outputs nested within outer joins (Tom Lane) This error could lead to incorrect plans for queries involving multiple levels of subqueries within JOIN syntax.
•
Fix incorrect planning in cases where the same non-strict expression appears in multiple WHERE and outer JOIN equality clauses (Tom Lane)
•
Fix planner crash with whole-row reference to a subquery (Tom Lane)
•
Fix incorrect generation of optimized MIN()/MAX() plans for inheritance trees (Tom Lane) The planner could fail in cases where the MIN()/MAX() argument was an expression rather than a simple variable.
•
Fix premature deletion of temporary files (Andres Freund)
2054
Appendix E. Release Notes •
Prevent intra-transaction memory leak when printing range values (Tom Lane) This fix actually cures transient memory leaks in any datatype output function, but range types are the only ones known to have had a significant problem.
•
Fix memory leaks when reloading configuration files (Heikki Linnakangas, Hari Babu)
•
Prevent incorrect display of dropped columns in NOT NULL and CHECK constraint violation messages (Michael Paquier and Tom Lane)
•
Allow default arguments and named-argument notation for window functions (Tom Lane) Previously, these cases were likely to crash.
•
Suppress trailing whitespace on each line when pretty-printing rules and views (Tom Lane) 9.3.0 generated such whitespace in many more cases than previous versions did. To reduce unexpected behavioral changes, suppress unnecessary whitespace in all cases.
•
Fix possible read past end of memory in rule printing (Peter Eisentraut)
•
Fix array slicing of int2vector and oidvector values (Tom Lane) Expressions of this kind are now implicitly promoted to regular int2 or oid arrays.
•
Return a valid JSON value when converting an empty hstore value to json (Oskari Saarenmaa)
•
Fix incorrect behaviors when using a SQL-standard, simple GMT offset timezone (Tom Lane) In some cases, the system would use the simple GMT offset value when it should have used the regular timezone setting that had prevailed before the simple offset was selected. This change also causes the timeofday function to honor the simple GMT offset zone.
•
Prevent possible misbehavior when logging translations of Windows error codes (Tom Lane)
•
Properly quote generated command lines in pg_ctl (Naoya Anzai and Tom Lane) This fix applies only to Windows.
•
Fix pg_dumpall to work when a source database sets default_transaction_read_only via ALTER DATABASE SET (Kevin Grittner) Previously, the generated script would fail during restore.
•
Fix pg_isready to handle its -d option properly (Fabrízio de Royes Mello and Fujii Masao)
•
Fix parsing of WAL file names in pg_receivexlog (Heikki Linnakangas) This error made pg_receivexlog unable to restart streaming after stopping, once at least 4 GB of WAL had been written.
•
Report out-of-disk-space failures properly in pg_upgrade (Peter Eisentraut)
•
Make ecpg search for quoted cursor names case-sensitively (Zoltán Böszörményi)
•
Fix ecpg’s processing of lists of variables declared varchar (Zoltán Böszörményi)
•
Make contrib/lo defend against incorrect trigger definitions (Marc Cousin)
•
Update time zone data files to tzdata release 2013h for DST law changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco, and Palestine. Also, new timezone abbreviations WIB, WIT, WITA for Indonesia.
2055
Appendix E. Release Notes
E.19. Release 9.3.1 Release date: 2013-10-10 This release contains a variety of fixes from 9.3.0. For information about new features in the 9.3 major release, see Section E.20.
E.19.1. Migration to Version 9.3.1 A dump/restore is not required for those running 9.3.X. However, if you use the hstore extension, see the first changelog entry.
E.19.2. Changes •
Ensure new-in-9.3 JSON functionality is added to the hstore extension during an update (Andrew Dunstan) Users who upgraded a pre-9.3 database containing hstore should execute ALTER EXTENSION hstore UPDATE;
after installing 9.3.1, to add two new JSON functions and a cast. (If hstore is already up to date, this command does nothing.) •
Fix memory leak when creating B-tree indexes on range columns (Heikki Linnakangas)
•
Fix memory leak caused by lo_open() failure (Heikki Linnakangas)
•
Serializable snapshot fixes (Kevin Grittner, Heikki Linnakangas)
•
Fix deadlock bug in libpq when using SSL (Stephen Frost)
•
Fix timeline handling bugs in pg_receivexlog (Heikki Linnakangas, Andrew Gierth)
•
Prevent CREATE FUNCTION from checking SET variables unless function body checking is enabled (Tom Lane)
•
Remove rare inaccurate warning during vacuum of index-less tables (Heikki Linnakangas)
E.20. Release 9.3 Release date: 2013-09-09
E.20.1. Overview Major enhancements in PostgreSQL 9.3 include: •
Add materialized views
•
Make simple views auto-updatable
•
Add many features for the JSON data type, including operators and functions to extract elements from JSON values
•
Implement SQL-standard LATERAL option for FROM-clause subqueries and function calls
2056
Appendix E. Release Notes •
Allow foreign data wrappers to support writes (inserts/updates/deletes) on foreign tables
•
Add a Postgres foreign data wrapper to allow access to other Postgres servers
•
Add support for event triggers
•
Add optional ability to checksum data pages and report corruption
•
Prevent non-key-field row updates from blocking foreign key checks
•
Greatly reduce System V shared memory requirements
The above items are explained in more detail in the sections below.
E.20.2. Migration to Version 9.3 A dump/restore using pg_dumpall, or use of pg_upgrade, is required for those wishing to migrate data from any previous release. Version 9.3 contains a number of changes that may affect compatibility with previous releases. Observe the following incompatibilities:
E.20.2.1. Server Settings •
Rename replication_timeout to wal_sender_timeout (Amit Kapila) This setting controls the WAL sender timeout.
•
Require superuser privileges to set commit_delay because it can now potentially delay other sessions (Simon Riggs)
•
Allow in-memory sorts to use their full memory allocation (Jeff Janes) Users who have set work_mem based on the previous behavior may need to revisit that setting.
E.20.2.2. Other •
Throw an error if a tuple to be updated or deleted has already been updated or deleted by a BEFORE trigger (Kevin Grittner) Formerly, the originally-intended update was silently skipped, resulting in logical inconsistency since the trigger might have propagated data to other places based on the intended update. Now an error is thrown to prevent the inconsistent results from being committed. If this change affects your application, the best solution is usually to move the data-propagation actions to an AFTER trigger. This error will also be thrown if a query invokes a volatile function that modifies rows that are later modified by the query itself. Such cases likewise previously resulted in silently skipping updates.
•
Change multicolumn ON UPDATE SET NULL/SET DEFAULT foreign key actions to affect all columns of the constraint, not just those changed in the UPDATE (Tom Lane) Previously, we would set only those referencing columns that correspond to referenced columns that were changed by the UPDATE. This was what was required by SQL-92, but more recent editions of the SQL standard specify the new behavior.
•
Force cached plans to be replanned if the search_path changes (Tom Lane)
2057
Appendix E. Release Notes Previously, cached plans already generated in the current session were not redone if the query was re-executed with a new search_path setting, resulting in surprising behavior. •
Fix to_number() to properly handle a period used as a thousands separator (Tom Lane) Previously, a period was considered to be a decimal point even when the locale says it isn’t and the D format code is used to specify use of the locale-specific decimal point. This resulted in wrong answers if FM format was also used.
•
Fix STRICT non-set-returning functions that have set-returning functions in their arguments to properly return null rows (Tom Lane) A null value passed to the strict function should result in a null output, but instead, that output row was suppressed entirely.
•
Store WAL in a continuous stream, rather than skipping the last 16MB segment every 4GB (Heikki Linnakangas) Previously, WAL files with names ending in FF were not used because of this skipping. If you have WAL backup or restore scripts that took this behavior into account, they will need to be adjusted.
•
In pg_constraint.confmatchtype, store the default foreign key match type (non-FULL, nonPARTIAL) as s for “simple” (Tom Lane) Previously this case was represented by u for “unspecified”.
E.20.3. Changes Below you will find a detailed account of the changes between PostgreSQL 9.3 and the previous major release.
E.20.3.1. Server E.20.3.1.1. Locking •
Prevent non-key-field row updates from blocking foreign key checks (Álvaro Herrera, Noah Misch, Andres Freund, Alexander Shulgin, Marti Raudsepp, Alexander Shulgin) This change improves concurrency and reduces the probability of deadlocks when updating tables involved in a foreign-key constraint. UPDATEs that do not change any columns referenced in a foreign key now take the new NO KEY UPDATE lock mode on the row, while foreign key checks use the new KEY SHARE lock mode, which does not conflict with NO KEY UPDATE. So there is no blocking unless a foreign-key column is changed.
•
Add configuration variable lock_timeout to allow limiting how long a session will wait to acquire any one lock (Zoltán Böszörményi)
E.20.3.1.2. Indexes •
Add SP-GiST support for range data types (Alexander Korotkov)
•
Allow GiST indexes to be unlogged (Jeevan Chalke)
•
Improve performance of GiST index insertion by randomizing the choice of which page to descend to when there are multiple equally good alternatives (Heikki Linnakangas)
2058
Appendix E. Release Notes •
Improve concurrency of hash index operations (Robert Haas)
E.20.3.1.3. Optimizer •
Collect and use histograms of upper and lower bounds, as well as range lengths, for range types (Alexander Korotkov)
•
Improve optimizer’s cost estimation for index access (Tom Lane)
•
Improve optimizer’s hash table size estimate for doing DISTINCT via hash aggregation (Tom Lane)
•
Suppress no-op Result and Limit plan nodes (Kyotaro Horiguchi, Amit Kapila, Tom Lane)
•
Reduce optimizer overhead by not keeping plans on the basis of cheap startup cost when the optimizer only cares about total cost overall (Tom Lane)
E.20.3.1.4. General Performance •
Add COPY FREEZE option to avoid the overhead of marking tuples as frozen later (Simon Riggs, Jeff Davis)
•
Improve performance of NUMERIC calculations (Kyotaro Horiguchi)
•
Improve synchronization of sessions waiting for commit_delay (Peter Geoghegan) This greatly improves the usefulness of commit_delay.
•
Improve performance of the CREATE TEMPORARY TABLE ... ON COMMIT DELETE ROWS option by not truncating such temporary tables in transactions that haven’t touched any temporary tables (Heikki Linnakangas)
•
Make vacuum recheck visibility after it has removed expired tuples (Pavan Deolasee) This increases the chance of a page being marked as all-visible.
•
Add per-resource-owner lock caches (Jeff Janes) This speeds up lock bookkeeping at statement completion in multi-statement transactions that hold many locks; it is particularly useful for pg_dump.
•
Avoid scanning the entire relation cache at commit of a transaction that creates a new relation (Jeff Janes) This speeds up sessions that create many tables in successive small transactions, such as a pg_restore run.
•
Improve performance of transactions that drop many relations (Tomas Vondra)
E.20.3.1.5. Monitoring •
Add optional ability to checksum data pages and report corruption (Simon Riggs, Jeff Davis, Greg Smith, Ants Aasma) The checksum option can be set during initdb.
•
Split the statistics collector’s data file into separate global and per-database files (Tomas Vondra) This reduces the I/O required for statistics tracking.
2059
Appendix E. Release Notes •
Fix the statistics collector to operate properly in cases where the system clock goes backwards (Tom Lane) Previously, statistics collection would stop until the time again reached the latest time previously recorded.
•
Emit an informative message to postmaster standard error when we are about to stop logging there (Tom Lane) This should help reduce user confusion about where to look for log output in common configurations that log to standard error only during postmaster startup.
E.20.3.1.6. Authentication •
When an authentication failure occurs, log the relevant pg_hba.conf line, to ease debugging of unintended failures (Magnus Hagander)
•
Improve LDAP error reporting and documentation (Peter Eisentraut)
•
Add support for specifying LDAP authentication parameters in URL format, per RFC 4516 (Peter Eisentraut)
•
Change the ssl_ciphers parameter to start with DEFAULT, rather than ALL, then remove insecure ciphers (Magnus Hagander) This should yield a more appropriate SSL cipher set.
•
Parse and load pg_ident.conf once, not during each connection (Amit Kapila) This is similar to how pg_hba.conf is processed.
E.20.3.1.7. Server Settings •
Greatly reduce System V shared memory requirements (Robert Haas) On Unix-like systems, mmap() is now used for most of PostgreSQL’s shared memory. For most users, this will eliminate any need to adjust kernel parameters for shared memory.
•
Allow the postmaster to listen on multiple Unix-domain sockets (Honza Horák) The
configuration parameter unix_socket_directory unix_socket_directories, which accepts a list of directories.
•
is
replaced
by
Allow a directory of configuration files to be processed (Magnus Hagander, Greg Smith, Selena Deckelmann) Such a directory is specified with include_dir in the server configuration file.
•
Increase the maximum initdb-configured value for shared_buffers to 128MB (Robert Haas) This is the maximum value that initdb will attempt to set in postgresql.conf; the previous maximum was 32MB.
•
Remove the external PID file, if any, on postmaster exit (Peter Eisentraut)
E.20.3.2. Replication and Recovery •
Allow a streaming replication standby to follow a timeline switch (Heikki Linnakangas)
2060
Appendix E. Release Notes This allows streaming standby servers to receive WAL data from a slave newly promoted to master status. Previously, other standbys would require a resync to begin following the new master. •
Add SQL functions pg_is_in_backup() and pg_backup_start_time() (Gilles Darold) These functions report the status of base backups.
•
Improve performance of streaming log shipping with synchronous_commit disabled (Andres Freund)
•
Allow much faster promotion of a streaming standby to primary (Simon Riggs, Kyotaro Horiguchi)
•
Add the last checkpoint’s redo location to pg_controldata’s output (Fujii Masao) This information is useful for determining which WAL files are needed for restore.
•
Allow tools like pg_receivexlog to run on computers with different architectures (Heikki Linnakangas) WAL files can still only be replayed on servers with the same architecture as the primary; but they can now be transmitted to and stored on machines of any architecture, since the streaming replication protocol is now machine-independent.
•
Make pg_basebackup --write-recovery-conf output a minimal recovery.conf file (Zoltán Böszörményi, Magnus Hagander) This simplifies setting up a standby server.
•
Allow pg_receivexlog and pg_basebackup --xlog-method to handle streaming timeline switches (Heikki Linnakangas)
•
Add wal_receiver_timeout parameter to control the WAL receiver’s timeout (Amit Kapila) This allows more rapid detection of connection failure.
•
Change the WAL record format to allow splitting the record header across pages (Heikki Linnakangas) The new format is slightly more compact, and is more efficient to write.
E.20.3.3. Queries •
Implement SQL-standard LATERAL option for FROM-clause subqueries and function calls (Tom Lane) This feature allows subqueries and functions in FROM to reference columns from other tables in the FROM clause. The LATERAL keyword is optional for functions.
•
Add support for piping COPY and psql \copy data to/from an external program (Etsuro Fujita)
•
Allow a multirow VALUES clause in a rule to reference OLD/NEW (Tom Lane)
E.20.3.4. Object Manipulation •
Add support for event triggers (Dimitri Fontaine, Robert Haas, Álvaro Herrera) This allows server-side functions written in event-enabled languages to be called when DDL commands are run.
•
Allow foreign data wrappers to support writes (inserts/updates/deletes) on foreign tables (KaiGai Kohei)
2061
Appendix E. Release Notes •
Add CREATE SCHEMA ... IF NOT EXISTS clause (Fabrízio de Royes Mello)
•
Make REASSIGN OWNED also change ownership of shared objects (Álvaro Herrera)
•
Make CREATE AGGREGATE complain if the given initial value string is not valid input for the transition datatype (Tom Lane)
•
Suppress CREATE TABLE’s messages about implicit index and sequence creation (Robert Haas) These messages now appear at DEBUG1 verbosity, so that they will not be shown by default.
•
Allow DROP TABLE IF EXISTS to succeed when a non-existent schema is specified in the table name (Bruce Momjian) Previously, it threw an error if the schema did not exist.
•
Provide clients with constraint violation details as separate fields (Pavel Stehule) This allows clients to retrieve table, column, data type, or constraint name error details. Previously such information had to be extracted from error strings. Client library support is required to access these fields.
E.20.3.4.1. ALTER •
Support IF NOT EXISTS option in ALTER TYPE ... ADD VALUE (Andrew Dunstan) This is useful for conditionally adding values to enumerated types.
•
Add ALTER ROLE ALL SET to establish settings for all users (Peter Eisentraut) This allows settings to apply to all users in all databases. ALTER DATABASE SET already allowed addition of settings for all users in a single database. postgresql.conf has a similar effect.
•
Add support for ALTER RULE ... RENAME (Ali Dar)
E.20.3.4.2. VIEWs •
Add materialized views (Kevin Grittner) Unlike ordinary views, where the base tables are read on every access, materialized views create physical tables at creation or refresh time. Access to the materialized view then reads from its physical table. There is not yet any facility for incrementally refreshing materialized views or autoaccessing them via base table access.
•
Make simple views auto-updatable (Dean Rasheed) Simple views that reference some or all columns from a single base table are now updatable by default. More complex views can be made updatable using INSTEAD OF triggers or INSTEAD rules.
•
Add CREATE RECURSIVE VIEW syntax (Peter Eisentraut) Internally this is translated into CREATE VIEW ... WITH RECURSIVE ....
•
Improve view/rule printing code to handle cases where referenced tables are renamed, or columns are renamed, added, or dropped (Tom Lane) Table and column renamings can produce cases where, if we merely substitute the new name into the original text of a rule or view, the result is ambiguous. This change fixes the rule-dumping code to insert manufactured table and column aliases when needed to preserve the original semantics.
2062
Appendix E. Release Notes
E.20.3.5. Data Types •
Increase the maximum size of large objects from 2GB to 4TB (Nozomi Anzai, Yugo Nagata) This change includes adding 64-bit-capable large object access functions, both in the server and in libpq.
•
Allow text timezone designations, e.g. “America/Chicago”, in the “T” field of ISO-format timestamptz input (Bruce Momjian)
E.20.3.5.1. JSON •
Add operators and functions to extract elements from JSON values (Andrew Dunstan)
•
Allow JSON values to be converted into records (Andrew Dunstan)
•
Add functions to convert scalars, records, and hstore values to JSON (Andrew Dunstan)
E.20.3.6. Functions •
Add array_remove() and array_replace() functions (Marco Nenciarini, Gabriele Bartolini)
•
Allow concat() and format() to properly expand VARIADIC-labeled arguments (Pavel Stehule)
•
Improve format() to provide field width and left/right alignment options (Pavel Stehule)
•
Make to_char(), to_date(), and to_timestamp() handle negative (BC) century values properly (Bruce Momjian) Previously the behavior was either wrong or inconsistent with positive/AD handling, e.g. with the format mask “IYYY-IW-DY”.
•
Make to_date() and to_timestamp() return proper results when mixing ISO and Gregorian week/day designations (Bruce Momjian)
•
Cause pg_get_viewdef() to start a new line by default after each SELECT target list entry and FROM entry (Marko Tiikkaja) This reduces line length in view printing, for instance in pg_dump output.
•
Fix map_sql_value_to_xml_value() to print values of domain types the same way their base type would be printed (Pavel Stehule) There are special formatting rules for certain built-in types such as boolean; these rules now also apply to domains over these types.
E.20.3.7. Server-Side Languages E.20.3.7.1. PL/pgSQL Server-Side Language •
Allow PL/pgSQL to use RETURN with a composite-type expression (Asif Rehman) Previously, in a function returning a composite type, RETURN could only reference a variable of that type.
•
Allow PL/pgSQL to access constraint violation details as separate fields (Pavel Stehule)
2063
Appendix E. Release Notes •
Allow PL/pgSQL to access the number of rows processed by COPY (Pavel Stehule) A COPY executed in a PL/pgSQL function now updates the value retrieved by GET DIAGNOSTICS x = ROW_COUNT.
•
Allow unreserved keywords to be used as identifiers everywhere in PL/pgSQL (Tom Lane) In certain places in the PL/pgSQL grammar, keywords had to be quoted to be used as identifiers, even if they were nominally unreserved.
E.20.3.7.2. PL/Python Server-Side Language •
Add PL/Python result object string handler (Peter Eisentraut) This allows plpy.debug(rv) to output something reasonable.
•
Make PL/Python convert OID values to a proper Python numeric type (Peter Eisentraut)
•
Handle SPI errors raised explicitly (with PL/Python’s RAISE) the same as internal SPI errors (Oskari Saarenmaa and Jan Urbanski)
E.20.3.8. Server Programming Interface (SPI) •
Prevent leakage of SPI tuple tables during subtransaction abort (Tom Lane) At the end of any failed subtransaction, the core SPI code now releases any SPI tuple tables that were created during that subtransaction. This avoids the need for SPI-using code to keep track of such tuple tables and release them manually in error-recovery code. Failure to do so caused a number of transaction-lifespan memory leakage issues in PL/pgSQL and perhaps other SPI clients. SPI_freetuptable() now protects itself against multiple freeing requests, so any existing code that did take care to clean up shouldn’t be broken by this change.
•
Allow SPI functions to access the number of rows processed by COPY (Pavel Stehule)
E.20.3.9. Client Applications •
Add command-line utility pg_isready to check if the server is ready to accept connections (Phil Sorber)
•
Support multiple --table arguments for pg_restore, clusterdb, reindexdb, and vacuumdb (Josh Kupershmidt) This is similar to the way pg_dump’s --table option works.
•
Add --dbname option to pg_dumpall, pg_basebackup, and pg_receivexlog to allow specifying a connection string (Amit Kapila)
•
Add libpq function PQconninfo() to return connection information (Zoltán Böszörményi, Magnus Hagander)
E.20.3.9.1. psql •
Adjust function cost settings so psql tab completion and pattern searching are more efficient (Tom Lane)
2064
Appendix E. Release Notes •
Improve psql’s tab completion coverage (Jeff Janes, Dean Rasheed, Peter Eisentraut, Magnus Hagander)
•
Allow the psql --single-transaction mode to work when reading from standard input (Fabien Coelho, Robert Haas) Previously this option only worked when reading from a file.
•
Remove psql warning when connecting to an older server (Peter Eisentraut) A warning is still issued when connecting to a server of a newer major version than psql’s.
E.20.3.9.1.1. Backslash Commands •
Add psql command \watch to repeatedly execute a SQL command (Will Leinweber)
•
Add psql command \gset to store query results in psql variables (Pavel Stehule)
•
Add SSL information to psql’s \conninfo command (Alastair Turner)
•
Add “Security” column to psql’s \df+ output (Jon Erdman)
•
Allow psql command \l to accept a database name pattern (Peter Eisentraut)
•
In psql, do not allow \connect to use defaults if there is no active connection (Bruce Momjian) This might be the case if the server had crashed.
•
Properly reset state after failure of a SQL command executed with psql’s \g file (Tom Lane) Previously, the output from subsequent SQL commands would unexpectedly continue to go to the same file.
E.20.3.9.1.2. Output •
Add a latex-longtable output format to psql (Bruce Momjian) This format allows tables to span multiple pages.
•
Add a border=3 output mode to the psql latex format (Bruce Momjian)
•
In psql’s tuples-only and expanded output modes, no longer emit “(No rows)” for zero rows (Peter Eisentraut)
•
In psql’s unaligned, expanded output mode, no longer print an empty line for zero rows (Peter Eisentraut)
E.20.3.9.2. pg_dump •
Add pg_dump --jobs option to dump tables in parallel (Joachim Wieland)
•
Make pg_dump output functions in a more predictable order (Joel Jacobson)
•
Fix tar files emitted by pg_dump to be POSIX conformant (Brian Weaver, Tom Lane)
•
Add --dbname option to pg_dump, for consistency with other client commands (Heikki Linnakangas) The database name could already be supplied last without a flag.
2065
Appendix E. Release Notes E.20.3.9.3. initdb •
Make initdb fsync the newly created data directory (Jeff Davis) This insures data integrity in event of a system crash shortly after initdb. This can be disabled by using --nosync.
•
Add initdb --sync-only option to sync the data directory to durable storage (Bruce Momjian) This is used by pg_upgrade.
•
Make initdb issue a warning about placing the data directory at the top of a file system mount point (Bruce Momjian)
E.20.3.10. Source Code •
Add infrastructure to allow plug-in background worker processes (Álvaro Herrera)
•
Create a centralized timeout API (Zoltán Böszörményi)
•
Create libpgcommon and move pg_malloc() and other functions there (Álvaro Herrera, Andres Freund) This allows libpgport to be used solely for portability-related code.
•
Add support for list links embedded in larger structs (Andres Freund)
•
Use SA_RESTART for all signals, including SIGALRM (Tom Lane)
•
Ensure that the correct text domain is used when translating errcontext() messages (Heikki Linnakangas)
•
Standardize naming of client-side memory allocation functions (Tom Lane)
•
Provide support for “static assertions” that will fail at compile time if some compile-time-constant condition is not met (Andres Freund, Tom Lane)
•
Support Assert() in client-side code (Andrew Dunstan)
•
Add decoration to inform the C compiler that some ereport() and elog() calls do not return (Peter Eisentraut, Andres Freund, Tom Lane, Heikki Linnakangas)
•
Allow options to be passed to the regression test output comparison utility via PG_REGRESS_DIFF_OPTS (Peter Eisentraut)
•
Add isolation tests for CREATE INDEX CONCURRENTLY (Abhijit Menon-Sen)
•
Remove typedefs for int2/int4 as they are better represented as int16/int32 (Peter Eisentraut)
•
Fix install-strip on Mac OS X (Peter Eisentraut)
•
Remove configure flag --disable-shared, as it is no longer supported (Bruce Momjian)
•
Rewrite pgindent in Perl (Andrew Dunstan)
•
Provide Emacs macro to set Perl formatting to match PostgreSQL’s perltidy settings (Peter Eisentraut)
•
Run tool to check the keyword list whenever the backend grammar is changed (Tom Lane)
•
Change the way UESCAPE is lexed, to significantly reduce the size of the lexer tables (Heikki Linnakangas)
•
Centralize flex and bison make rules (Peter Eisentraut)
2066
Appendix E. Release Notes This is useful for pgxs authors. •
Change many internal backend functions to return object OIDs rather than void (Dimitri Fontaine) This is useful for event triggers.
•
Invent pre-commit/pre-prepare/pre-subcommit events for transaction callbacks (Tom Lane) Loadable modules that use transaction callbacks might need modification to handle these new event types.
•
Add function pg_identify_object() to produce a machine-readable description of a database object (Álvaro Herrera)
•
Add post-ALTER-object server hooks (KaiGai Kohei)
•
Implement a generic binary heap and use it for Merge-Append operations (Abhijit Menon-Sen)
•
Provide a tool to help detect timezone abbreviation changes when updating the src/timezone/data files (Tom Lane)
•
Add pkg-config support for libpq and ecpg libraries (Peter Eisentraut)
•
Remove src/tools/backend, now that the content is on the PostgreSQL wiki (Bruce Momjian)
•
Split out WAL reading as an independent facility (Heikki Linnakangas, Andres Freund)
•
Use a 64-bit integer to represent WAL positions (XLogRecPtr) instead of two 32-bit integers (Heikki Linnakangas) Generally, tools that need to read the WAL format will need to be adjusted.
•
Allow PL/Python to support platform-specific include directories (Peter Eisentraut)
•
Allow PL/Python on OS X to build against custom versions of Python (Peter Eisentraut)
E.20.3.11. Additional Modules •
Add a Postgres foreign data wrapper contrib module to allow access to other Postgres servers (Shigeru Hanada) This foreign data wrapper supports writes.
•
Add pg_xlogdump contrib program (Andres Freund)
•
Add support for indexing of regular-expression searches in pg_trgm (Alexander Korotkov)
•
Improve pg_trgm’s handling of multibyte characters (Tom Lane) On a platform that does not have the wcstombs() or towlower() library functions, this could result in an incompatible change in the contents of pg_trgm indexes for non-ASCII data. In such cases, REINDEX those indexes to ensure correct search results.
•
Add a pgstattuple function to report the size of the pending-insertions list of a GIN index (Fujii Masao)
•
Make oid2name, pgbench, and vacuumlo set fallback_application_name (Amit Kapila)
•
Improve output of pg_test_timing (Bruce Momjian)
•
Improve output of pg_test_fsync (Peter Geoghegan)
•
Create a dedicated foreign data wrapper, with its own option validator function, for dblink (Shigeru Hanada)
2067
Appendix E. Release Notes When using this FDW to define the target of a dblink connection, instead of using a hard-wired list of connection options, the underlying libpq library is consulted to see what connection options it supports. E.20.3.11.1. pg_upgrade •
Allow pg_upgrade to do dumps and restores in parallel (Bruce Momjian, Andrew Dunstan) This allows parallel schema dump/restore of databases, as well as parallel copy/link of data files per tablespace. Use the --jobs option to specify the level of parallelism.
•
Make pg_upgrade create Unix-domain sockets in the current directory (Bruce Momjian, Tom Lane) This reduces the possibility that someone will accidentally connect during the upgrade.
•
Make pg_upgrade --check mode properly detect the location of non-default socket directories (Bruce Momjian, Tom Lane)
•
Improve performance of pg_upgrade for databases with many tables (Bruce Momjian)
•
Improve pg_upgrade’s logs by showing executed commands (Álvaro Herrera)
•
Improve pg_upgrade’s status display during copy/link (Bruce Momjian)
E.20.3.11.2. pgbench •
Add --foreign-keys option to pgbench (Jeff Janes) This adds foreign key constraints to the standard tables created by pgbench, for use in foreign key performance testing.
•
Allow
pgbench
to
aggregate
performance
statistics
and
produce
output
every
--aggregate-interval seconds (Tomas Vondra) •
Add pgbench --sampling-rate option to control the percentage of transactions logged (Tomas Vondra)
•
Reduce and improve the status message output of pgbench’s initialization mode (Robert Haas, Peter Eisentraut)
•
Add pgbench -q mode to print one output line every five seconds (Tomas Vondra)
•
Output pgbench elapsed and estimated remaining time during initialization (Tomas Vondra)
•
Allow pgbench to use much larger scale factors, by changing relevant columns from integer to bigint when the requested scale factor exceeds 20000 (Greg Smith)
E.20.3.12. Documentation •
Allow EPUB-format documentation to be created (Peter Eisentraut)
E.21. Release 9.2.23 Release date: 2017-08-31 This release contains a small number of fixes from 9.2.22. For information about new features in the 9.2 major release, see Section E.44. The PostgreSQL community will stop releasing updates for the 9.2.X release series in September 2017. Users are encouraged to update to a newer release branch soon.
E.21.1. Migration to Version 9.2.23 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.22, see Section E.22.
E.21.2. Changes •
Show foreign tables in information_schema.table_privileges view (Peter Eisentraut) All other relevant information_schema views include foreign tables, but this one ignored them. Since this view definition is installed by initdb, merely upgrading will not fix the problem. If you need to fix this in an existing installation, you can, as a superuser, do this in psql: BEGIN; DROP SCHEMA information_schema CASCADE; \i SHAREDIR/information_schema.sql COMMIT; (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) This must be repeated in
each database to be fixed. •
Clean up handling of a fatal exit (e.g., due to receipt of SIGTERM) that occurs while trying to execute a ROLLBACK of a failed transaction (Tom Lane) This situation could result in an assertion failure. In production builds, the exit would still occur, but it would log an unexpected message about “cannot drop active portal”.
•
Remove assertion that could trigger during a fatal exit (Tom Lane)
•
Correctly identify columns that are of a range type or domain type over a composite type or domain type being searched for (Tom Lane) Certain ALTER commands that change the definition of a composite type or domain type are supposed to fail if there are any stored values of that type in the database, because they lack the infrastructure needed to update or check such values. Previously, these checks could miss relevant values that are wrapped inside range types or sub-domains, possibly allowing the database to become inconsistent.
•
Change ecpg’s parser to allow RETURNING clauses without attached C variables (Michael Meskes) This allows ecpg programs to contain SQL constructs that use RETURNING internally (for example, inside a CTE) rather than using it to define values to be returned to the client.
•
Improve selection of compiler flags for PL/Perl on Windows (Tom Lane) This fix avoids possible crashes of PL/Perl due to inconsistent assumptions about the width of time_t values. A side-effect that may be visible to extension developers is that
2069
Appendix E. Release Notes _USE_32BIT_TIME_T is no longer defined globally in PostgreSQL Windows builds. This is not expected to cause problems, because type time_t is not used in any PostgreSQL API definitions.
E.22. Release 9.2.22 Release date: 2017-08-10 This release contains a variety of fixes from 9.2.21. For information about new features in the 9.2 major release, see Section E.44. The PostgreSQL community will stop releasing updates for the 9.2.X release series in September 2017. Users are encouraged to update to a newer release branch soon.
E.22.1. Migration to Version 9.2.22 A dump/restore is not required for those running 9.2.X. However, if you use foreign data servers that make use of user passwords for authentication, see the first changelog entry below. Also, if you are upgrading from a version earlier than 9.2.20, see Section E.24.
E.22.2. Changes •
Further restrict visibility of pg_user_mappings.umoptions, to protect passwords stored as user mapping options (Noah Misch) The fix for CVE-2017-7486 was incorrect: it allowed a user to see the options in her own user mapping, even if she did not have USAGE permission on the associated foreign server. Such options might include a password that had been provided by the server owner rather than the user herself. Since information_schema.user_mapping_options does not show the options in such cases, pg_user_mappings should not either. (CVE-2017-7547) By itself, this patch will only fix the behavior in newly initdb’d databases. If you wish to apply this change in an existing database, you will need to do the following: 1.
Restart the postmaster after adding allow_system_table_mods = true to postgresql.conf. (In versions supporting ALTER SYSTEM, you can use that to make the configuration change, but you’ll still need a restart.)
2.
In each database of the cluster, run the following commands as superuser: SET search_path = pg_catalog; CREATE OR REPLACE VIEW pg_user_mappings AS SELECT U.oid AS umid, S.oid AS srvid, S.srvname AS srvname, U.umuser AS umuser, CASE WHEN U.umuser = 0 THEN ’public’ ELSE
2070
Appendix E. Release Notes
A.rolname END AS usename, CASE WHEN (U.umuser 0 AND A.rolname = current_user AND (pg_has_role(S.srvowner, ’USAGE’) OR has_server_privilege(S.oid, ’USAGE’))) OR (U.umuser = 0 AND pg_has_role(S.srvowner, ’USAGE’)) OR (SELECT rolsuper FROM pg_authid WHERE rolname = current_user THEN U.umoptions ELSE NULL END AS umoptions FROM pg_user_mapping U LEFT JOIN pg_authid A ON (A.oid = U.umuser) JOIN pg_foreign_server S ON (U.umserver = S.oid);
3.
Do not forget to include the template0 and template1 databases, or the vulnerability will still exist in databases you create later. To fix template0, you’ll need to temporarily make it accept connections. In PostgreSQL 9.5 and later, you can use ALTER DATABASE template0 WITH ALLOW_CONNECTIONS true; and then after fixing template0, undo that with ALTER DATABASE template0 WITH ALLOW_CONNECTIONS false;
In prior versions, instead use UPDATE pg_database SET datallowconn = true WHERE datname = ’template0’; UPDATE pg_database SET datallowconn = false WHERE datname = ’template0’;
4.
•
Finally, remove the allow_system_table_mods configuration setting, and again restart the postmaster.
Disallow empty passwords in all password-based authentication methods (Heikki Linnakangas) libpq ignores empty password specifications, and does not transmit them to the server. So, if a user’s password has been set to the empty string, it’s impossible to log in with that password via psql or other libpq-based clients. An administrator might therefore believe that setting the password to empty is equivalent to disabling password login. However, with a modified or non-libpq-based client, logging in could be possible, depending on which authentication method is configured. In particular the most common method, md5, accepted empty passwords. Change the server to reject empty passwords in all cases. (CVE-2017-7546)
•
On Windows, retry process creation if we fail to reserve the address range for our shared memory in the new process (Tom Lane, Amit Kapila) This is expected to fix infrequent child-process-launch failures that are probably due to interference from antivirus products.
•
Fix low-probability corruption of shared predicate-lock hash table in Windows builds (Thomas Munro, Tom Lane)
•
Avoid logging clean closure of an SSL connection as though it were a connection reset (Michael Paquier)
•
Prevent sending SSL session tickets to clients (Tom Lane) This fix prevents reconnection failures with ticket-aware client-side SSL code.
•
Fix code for setting tcp_keepalives_idle on Solaris (Tom Lane)
•
Fix statistics collector to honor inquiry messages issued just after a postmaster shutdown and immediate restart (Tom Lane)
2071
Appendix E. Release Notes Statistics inquiries issued within half a second of the previous postmaster shutdown were effectively ignored. •
Ensure that the statistics collector’s receive buffer size is at least 100KB (Tom Lane) This reduces the risk of dropped statistics data on older platforms whose default receive buffer size is less than that.
•
Fix possible creation of an invalid WAL segment when a standby is promoted just after it processes an XLOG_SWITCH WAL record (Andres Freund)
•
Fix SIGHUP and SIGUSR1 handling in walsender processes (Petr Jelinek, Andres Freund)
•
Fix unnecessarily slow restarts of walreceiver processes due to race condition in postmaster (Tom Lane)
•
Fix cases where an INSERT or UPDATE assigns to more than one element of a column that is of domain-over-array type (Tom Lane)
•
Move autogenerated array types out of the way during ALTER ... RENAME (Vik Fearing) Previously, we would rename a conflicting autogenerated array type out of the way during CREATE; this fix extends that behavior to renaming operations.
•
Ensure that ALTER USER ... SET accepts all the syntax variants that ALTER ROLE ... SET does (Peter Eisentraut)
•
Properly update dependency info when changing a datatype I/O function’s argument or return type from opaque to the correct type (Heikki Linnakangas) CREATE TYPE updates I/O functions declared in this long-obsolete style, but it forgot to record a dependency on the type, allowing a subsequent DROP TYPE to leave broken function definitions
behind. •
Reduce memory usage when ANALYZE processes a tsvector column (Heikki Linnakangas)
•
Fix unnecessary precision loss and sloppy rounding when multiplying or dividing money values by integers or floats (Tom Lane)
•
Tighten checks for whitespace in functions that parse identifiers, such as regprocedurein() (Tom Lane) Depending on the prevailing locale, these functions could misinterpret fragments of multibyte characters as whitespace.
•
Use relevant #define symbols from Perl while compiling PL/Perl (Ashutosh Sharma, Tom Lane) This avoids portability problems, typically manifesting as a “handshake” mismatch during library load, when working with recent Perl versions.
•
In psql, fix failure when COPY FROM STDIN is ended with a keyboard EOF signal and then another COPY FROM STDIN is attempted (Thomas Munro) This misbehavior was observed on BSD-derived platforms (including macOS), but not on most others.
•
Fix pg_dump to not emit invalid SQL for an empty operator class (Daniel Gustafsson)
•
Fix pg_dump output to stdout on Windows (Kuntal Ghosh) A compressed plain-text dump written to stdout would contain corrupt data due to failure to put the file descriptor into binary mode.
•
Fix pg_get_ruledef() to print correct output for the ON SELECT rule of a view whose columns have been renamed (Tom Lane)
2072
Appendix E. Release Notes In some corner cases, pg_dump relies on pg_get_ruledef() to dump views, so that this error could result in dump/reload failures. •
Fix dumping of function expressions in the FROM clause in cases where the expression does not deparse into something that looks like a function call (Tom Lane)
•
Fix pg_basebackup output to stdout on Windows (Haribabu Kommi) A backup written to stdout would contain corrupt data due to failure to put the file descriptor into binary mode.
•
Fix pg_upgrade to ensure that the ending WAL record does not have wal_level = minimum (Bruce Momjian) This condition could prevent upgraded standby servers from reconnecting.
•
Always use -fPIC, not -fpic, when building shared libraries with gcc (Tom Lane) This supports larger extension libraries on platforms where it makes a difference.
•
Fix unescaped-braces issue in our build scripts for Microsoft MSVC, to avoid a warning or error from recent Perl versions (Andrew Dunstan)
•
In MSVC builds, handle the case where the openssl library is not within a VC subdirectory (Andrew Dunstan)
•
In MSVC builds, add proper include path for libxml2 header files (Andrew Dunstan) This fixes a former need to move things around in standard Windows installations of libxml2.
•
In MSVC builds, recognize a Tcl library that is named tcl86.lib (Noah Misch)
E.23. Release 9.2.21 Release date: 2017-05-11 This release contains a variety of fixes from 9.2.20. For information about new features in the 9.2 major release, see Section E.44. The PostgreSQL community will stop releasing updates for the 9.2.X release series in September 2017. Users are encouraged to update to a newer release branch soon.
E.23.1. Migration to Version 9.2.21 A dump/restore is not required for those running 9.2.X. However, if you use foreign data servers that make use of user passwords for authentication, see the first changelog entry below. Also, if you are upgrading from a version earlier than 9.2.20, see Section E.24.
E.23.2. Changes •
Restrict visibility of pg_user_mappings.umoptions, to protect passwords stored as user mapping options (Michael Paquier, Feike Steenbergen)
2073
Appendix E. Release Notes The previous coding allowed the owner of a foreign server object, or anyone he has granted server USAGE permission to, to see the options for all user mappings associated with that server. This might well include passwords for other users. Adjust the view definition to match the behavior of information_schema.user_mapping_options, namely that these options are visible to the user being mapped, or if the mapping is for PUBLIC and the current user is the server owner, or if the current user is a superuser. (CVE-2017-7486) By itself, this patch will only fix the behavior in newly initdb’d databases. If you wish to apply this change in an existing database, follow the corrected procedure shown in the changelog entry for CVE-2017-7547, in Section E.22. •
Prevent exposure of statistical information via leaky operators (Peter Eisentraut) Some selectivity estimation functions in the planner will apply user-defined operators to values obtained from pg_statistic, such as most common values and histogram entries. This occurs before table permissions are checked, so a nefarious user could exploit the behavior to obtain these values for table columns he does not have permission to read. To fix, fall back to a default estimate if the operator’s implementation function is not certified leak-proof and the calling user does not have permission to read the table column whose statistics are needed. At least one of these criteria is satisfied in most cases in practice. (CVE-2017-7484)
•
Fix possible corruption of “init forks” of unlogged indexes (Robert Haas, Michael Paquier) This could result in an unlogged index being set to an invalid state after a crash and restart. Such a problem would persist until the index was dropped and rebuilt.
•
Fix incorrect reconstruction of pg_subtrans entries when a standby server replays a prepared but uncommitted two-phase transaction (Tom Lane) In most cases this turned out to have no visible ill effects, but in corner cases it could result in circular references in pg_subtrans, potentially causing infinite loops in queries that examine rows modified by the two-phase transaction.
•
Ensure parsing of queries in extension scripts sees the results of immediately-preceding DDL (Julien Rouhaud, Tom Lane) Due to lack of a cache flush step between commands in an extension script file, non-utility queries might not see the effects of an immediately preceding catalog change, such as ALTER TABLE ... RENAME.
•
Skip tablespace privilege checks when ALTER TABLE ... ALTER COLUMN TYPE rebuilds an existing index (Noah Misch) The command failed if the calling user did not currently have CREATE privilege for the tablespace containing the index. That behavior seems unhelpful, so skip the check, allowing the index to be rebuilt where it is.
•
Fix ALTER TABLE ... VALIDATE CONSTRAINT to not recurse to child tables when the constraint is marked NO INHERIT (Amit Langote) This fix prevents unwanted “constraint does not exist” failures when no matching constraint is present in the child tables.
•
Fix VACUUM to account properly for pages that could not be scanned due to conflicting page pins (Andrew Gierth) This tended to lead to underestimation of the number of tuples in the table. In the worst case of a small heavily-contended table, VACUUM could incorrectly report that the table contained no tuples, leading to very bad planning choices.
2074
Appendix E. Release Notes •
Ensure that bulk-tuple-transfer loops within a hash join are interruptible by query cancel requests (Tom Lane, Thomas Munro)
•
Fix cursor_to_xml() to produce valid output with tableforest = false (Thomas Munro, Peter Eisentraut) Previously it failed to produce a wrapping
element.
•
Improve performance of pg_timezone_names view (Tom Lane, David Rowley)
•
Fix sloppy handling of corner-case errors from lseek() and close() (Tom Lane) Neither of these system calls are likely to fail in typical situations, but if they did, fd.c could get quite confused.
•
Fix incorrect check for whether postmaster is running as a Windows service (Michael Paquier) This could result in attempting to write to the event log when that isn’t accessible, so that no logging happens at all.
•
Fix ecpg to support COMMIT PREPARED and ROLLBACK PREPARED (Masahiko Sawada)
•
Fix a double-free error when processing dollar-quoted string literals in ecpg (Michael Meskes)
•
In pg_dump, fix incorrect schema and owner marking for comments and security labels of some types of database objects (Giuseppe Broccolo, Tom Lane) In simple cases this caused no ill effects; but for example, a schema-selective restore might omit comments it should include, because they were not marked as belonging to the schema of their associated object.
•
Avoid emitting an invalid list file in pg_restore -l when SQL object names contain newlines (Tom Lane) Replace newlines by spaces, which is sufficient to make the output valid for pg_restore -L’s purposes.
•
Fix pg_upgrade to transfer comments and security labels attached to “large objects” (blobs) (Stephen Frost) Previously, blobs were correctly transferred to the new database, but any comments or security labels attached to them were lost.
•
Improve error handling in contrib/adminpack’s pg_file_write() function (Noah Misch) Notably, it failed to detect errors reported by fclose().
•
In contrib/dblink, avoid leaking the previous unnamed connection when establishing a new unnamed connection (Joe Conway)
•
Support OpenSSL 1.1.0 (Heikki Linnakangas, Andreas Karlsson, Tom Lane) This is a back-patch of work previously done in newer branches; it’s needed since many platforms are adopting newer OpenSSL versions.
•
Support Tcl 8.6 in MSVC builds (Álvaro Herrera)
•
Sync our copy of the timezone library with IANA release tzcode2017b (Tom Lane) This fixes a bug affecting some DST transitions in January 2038.
•
Update time zone data files to tzdata release 2017b for DST law changes in Chile, Haiti, and Mongolia, plus historical corrections for Ecuador, Kazakhstan, Liberia, and Spain. Switch to numeric abbreviations for numerous time zones in South America, the Pacific and Indian oceans, and some Asian and Middle Eastern countries.
2075
Appendix E. Release Notes The IANA time zone database previously provided textual abbreviations for all time zones, sometimes making up abbreviations that have little or no currency among the local population. They are in process of reversing that policy in favor of using numeric UTC offsets in zones where there is no evidence of real-world use of an English abbreviation. At least for the time being, PostgreSQL will continue to accept such removed abbreviations for timestamp input. But they will not be shown in the pg_timezone_names view nor used for output. •
Use correct daylight-savings rules for POSIX-style time zone names in MSVC builds (David Rowley) The Microsoft MSVC build scripts neglected to install the posixrules file in the timezone directory tree. This resulted in the timezone code falling back to its built-in rule about what DST behavior to assume for a POSIX-style time zone name. For historical reasons that still corresponds to the DST rules the USA was using before 2007 (i.e., change on first Sunday in April and last Sunday in October). With this fix, a POSIX-style zone name will use the current and historical DST transition dates of the US/Eastern zone. If you don’t want that, remove the posixrules file, or replace it with a copy of some other zone file (see Section 8.5.3). Note that due to caching, you may need to restart the server to get such changes to take effect.
E.24. Release 9.2.20 Release date: 2017-02-09 This release contains a variety of fixes from 9.2.19. For information about new features in the 9.2 major release, see Section E.44.
E.24.1. Migration to Version 9.2.20 A dump/restore is not required for those running 9.2.X. However, if your installation has been affected by the bug described in the first changelog entry below, then after updating you may need to take action to repair corrupted indexes. Also, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.24.2. Changes •
Fix a race condition that could cause indexes built with CREATE INDEX CONCURRENTLY to be corrupt (Pavan Deolasee, Tom Lane) If CREATE INDEX CONCURRENTLY was used to build an index that depends on a column not previously indexed, then rows updated by transactions that ran concurrently with the CREATE INDEX command could have received incorrect index entries. If you suspect this may have happened, the most reliable solution is to rebuild affected indexes after installing this update.
•
Unconditionally WAL-log creation of the “init fork” for an unlogged table (Michael Paquier) Previously, this was skipped when wal_level = minimal, but actually it’s necessary even in that case to ensure that the unlogged table is properly reset to empty after a crash.
•
Fix WAL page header validation when re-reading segments (Takayuki Tsunakawa, Amit Kapila) In corner cases, a spurious “out-of-sequence TLI” error could be reported during recovery.
2076
Appendix E. Release Notes •
If the stats collector dies during hot standby, restart it (Takayuki Tsunakawa)
•
Check for interrupts while hot standby is waiting for a conflicting query (Simon Riggs)
•
Avoid constantly respawning the autovacuum launcher in a corner case (Amit Khandekar) This fix avoids problems when autovacuum is nominally off and there are some tables that require freezing, but all such tables are already being processed by autovacuum workers.
•
Fix check for when an extension member object can be dropped (Tom Lane) Extension upgrade scripts should be able to drop member objects, but this was disallowed for serial-column sequences, and possibly other cases.
•
Make sure ALTER TABLE preserves index tablespace assignments when rebuilding indexes (Tom Lane, Michael Paquier) Previously, non-default settings of default_tablespace could result in broken indexes.
•
Prevent dropping a foreign-key constraint if there are pending trigger events for the referenced relation (Tom Lane) This avoids “could not find trigger NNN ” or “relation NNN has no triggers” errors.
•
Fix processing of OID column when a table with OIDs is associated to a parent with OIDs via ALTER TABLE ... INHERIT (Amit Langote) The OID column should be treated the same as regular user columns in this case, but it wasn’t, leading to odd behavior in later inheritance changes.
•
Check for serializability conflicts before reporting constraint-violation failures (Thomas Munro) When using serializable transaction isolation, it is desirable that any error due to concurrent transactions should manifest as a serialization failure, thereby cueing the application that a retry might succeed. Unfortunately, this does not reliably happen for duplicate-key failures caused by concurrent insertions. This change ensures that such an error will be reported as a serialization error if the application explicitly checked for the presence of a conflicting key (and did not find it) earlier in the transaction.
•
Ensure that column typmods are determined accurately for multi-row VALUES constructs (Tom Lane) This fixes problems occurring when the first value in a column has a determinable typmod (e.g., length for a varchar value) but later values don’t share the same limit.
•
Throw error for an unfinished Unicode surrogate pair at the end of a Unicode string (Tom Lane) Normally, a Unicode surrogate leading character must be followed by a Unicode surrogate trailing character, but the check for this was missed if the leading character was the last character in a Unicode string literal (U&’...’) or Unicode identifier (U&"...").
•
Ensure that a purely negative text search query, such as !foo, matches empty tsvectors (Tom Dunstan) Such matches were found by GIN index searches, but not by sequential scans or GiST index searches.
•
Prevent crash when ts_rewrite() replaces a non-top-level subtree with an empty query (Artur Zakirov)
•
Fix performance problems in ts_rewrite() (Tom Lane)
•
Fix ts_rewrite()’s handling of nested NOT operators (Tom Lane)
•
Fix array_fill() to handle empty arrays properly (Tom Lane)
2077
Appendix E. Release Notes •
Fix one-byte buffer overrun in quote_literal_cstr() (Heikki Linnakangas) The overrun occurred only if the input consisted entirely of single quotes and/or backslashes.
•
Prevent multiple calls of pg_start_backup() and pg_stop_backup() from running concurrently (Michael Paquier) This avoids an assertion failure, and possibly worse things, if someone tries to run these functions in parallel.
•
Avoid discarding interval-to-interval casts that aren’t really no-ops (Tom Lane) In some cases, a cast that should result in zeroing out low-order interval fields was mistakenly deemed to be a no-op and discarded. An example is that casting from INTERVAL MONTH to INTERVAL YEAR failed to clear the months field.
•
Fix pg_dump to dump user-defined casts and transforms that use built-in functions (Stephen Frost)
•
Fix possible pg_basebackup failure on standby server when including WAL files (Amit Kapila, Robert Haas)
•
Ensure that the Python exception objects we create for PL/Python are properly reference-counted (Rafa de la Torre, Tom Lane) This avoids failures if the objects are used after a Python garbage collection cycle has occurred.
•
Fix PL/Tcl to support triggers on tables that have .tupno as a column name (Tom Lane) This matches the (previously undocumented) behavior of PL/Tcl’s spi_exec and spi_execp commands, namely that a magic .tupno column is inserted only if there isn’t a real column named that.
•
Allow DOS-style line endings in ~/.pgpass files, even on Unix (Vik Fearing) This change simplifies use of the same password file across Unix and Windows machines.
•
Fix one-byte buffer overrun if ecpg is given a file name that ends with a dot (Takayuki Tsunakawa)
•
Fix psql’s tab completion for ALTER DEFAULT PRIVILEGES (Gilles Darold, Stephen Frost)
•
In psql, treat an empty or all-blank setting of the PAGER environment variable as meaning “no pager” (Tom Lane) Previously, such a setting caused output intended for the pager to vanish entirely.
•
Improve contrib/dblink’s reporting of low-level libpq errors, such as out-of-memory (Joe Conway)
•
On Windows, ensure that environment variable changes are propagated to DLLs built with debug options (Christian Ullrich)
•
Sync our copy of the timezone library with IANA release tzcode2016j (Tom Lane) This fixes various issues, most notably that timezone data installation failed if the target directory didn’t support hard links.
•
Update time zone data files to tzdata release 2016j for DST law changes in northern Cyprus (adding a new zone Asia/Famagusta), Russia (adding a new zone Europe/Saratov), Tonga, and Antarctica/Casey. Historical corrections for Italy, Kazakhstan, Malta, and Palestine. Switch to preferring numeric zone abbreviations for Tonga.
2078
Appendix E. Release Notes
E.25. Release 9.2.19 Release date: 2016-10-27 This release contains a variety of fixes from 9.2.18. For information about new features in the 9.2 major release, see Section E.44.
E.25.1. Migration to Version 9.2.19 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.25.2. Changes •
Fix EvalPlanQual rechecks involving CTE scans (Tom Lane) The recheck would always see the CTE as returning no rows, typically leading to failure to update rows that were recently updated.
•
Fix improper repetition of previous results from hashed aggregation in a subquery (Andrew Gierth) The test to see if we can reuse a previously-computed hash table of the aggregate state values neglected the possibility of an outer query reference appearing in an aggregate argument expression. A change in the value of such a reference should lead to recalculating the hash table, but did not.
•
Fix EXPLAIN to emit valid XML when track_io_timing is on (Markus Winand) Previously the XML output-format option produced syntactically invalid tags such as . That is now rendered as .
•
Suppress printing of zeroes for unmeasured times in EXPLAIN (Maksim Milyutin) Certain option combinations resulted in printing zero values for times that actually aren’t ever measured in that combination. Our general policy in EXPLAIN is not to print such fields at all, so do that consistently in all cases.
•
Fix timeout length when VACUUM is waiting for exclusive table lock so that it can truncate the table (Simon Riggs) The timeout was meant to be 50 milliseconds, but it was actually only 50 microseconds, causing VACUUM to give up on truncation much more easily than intended. Set it to the intended value.
•
Fix bugs in merging inherited CHECK constraints while creating or altering a table (Tom Lane, Amit Langote) Allow identical CHECK constraints to be added to a parent and child table in either order. Prevent merging of a valid constraint from the parent table with a NOT VALID constraint on the child. Likewise, prevent merging of a NO INHERIT child constraint with an inherited constraint.
•
Remove artificial restrictions on the values accepted by numeric_in() and numeric_recv() (Tom Lane) We allow numeric values up to the limit of the storage format (more than 1e100000), so it seems fairly pointless that numeric_in() rejected scientific-notation exponents above 1000. Likewise, it was silly for numeric_recv() to reject more than 1000 digits in an input value.
•
Avoid very-low-probability data corruption due to testing tuple visibility without holding buffer lock (Thomas Munro, Peter Geoghegan, Tom Lane)
2079
Appendix E. Release Notes •
Fix file descriptor leakage when truncating a temporary relation of more than 1GB (Andres Freund)
•
Disallow starting a standalone backend with standby_mode turned on (Michael Paquier) This can’t do anything useful, since there will be no WAL receiver process to fetch more WAL data; and it could result in misbehavior in code that wasn’t designed with this situation in mind.
•
Don’t try to share SSL contexts across multiple connections in libpq (Heikki Linnakangas) This led to assorted corner-case bugs, particularly when trying to use different SSL parameters for different connections.
•
Avoid corner-case memory leak in libpq (Tom Lane) The reported problem involved leaking an error report during PQreset(), but there might be related cases.
•
Make ecpg’s --help and --version options work consistently with our other executables (Haribabu Kommi)
•
In pg_dump, never dump range constructor functions (Tom Lane) This oversight led to pg_upgrade failures with extensions containing range types, due to duplicate creation of the constructor functions.
•
Fix contrib/intarray/bench/bench.pl to print the results of the EXPLAIN it does when given the -e option (Daniel Gustafsson)
•
Update Windows time zone mapping to recognize some time zone names added in recent Windows versions (Michael Paquier)
•
Prevent failure of obsolete dynamic time zone abbreviations (Tom Lane) If a dynamic time zone abbreviation does not match any entry in the referenced time zone, treat it as equivalent to the time zone name. This avoids unexpected failures when IANA removes abbreviations from their time zone database, as they did in tzdata release 2016f and seem likely to do again in the future. The consequences were not limited to not recognizing the individual abbreviation; any mismatch caused the pg_timezone_abbrevs view to fail altogether.
•
Update time zone data files to tzdata release 2016h for DST law changes in Palestine and Turkey, plus historical corrections for Turkey and some regions of Russia. Switch to numeric abbreviations for some time zones in Antarctica, the former Soviet Union, and Sri Lanka. The IANA time zone database previously provided textual abbreviations for all time zones, sometimes making up abbreviations that have little or no currency among the local population. They are in process of reversing that policy in favor of using numeric UTC offsets in zones where there is no evidence of real-world use of an English abbreviation. At least for the time being, PostgreSQL will continue to accept such removed abbreviations for timestamp input. But they will not be shown in the pg_timezone_names view nor used for output. In this update, AMT is no longer shown as being in use to mean Armenia Time. Therefore, we have changed the Default abbreviation set to interpret it as Amazon Time, thus UTC-4 not UTC+4.
E.26. Release 9.2.18 Release date: 2016-08-11 This release contains a variety of fixes from 9.2.17. For information about new features in the 9.2 major release, see Section E.44.
2080
Appendix E. Release Notes
E.26.1. Migration to Version 9.2.18 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.26.2. Changes •
Fix possible mis-evaluation of nested CASE-WHEN expressions (Heikki Linnakangas, Michael Paquier, Tom Lane) A CASE expression appearing within the test value subexpression of another CASE could become confused about whether its own test value was null or not. Also, inlining of a SQL function implementing the equality operator used by a CASE expression could result in passing the wrong test value to functions called within a CASE expression in the SQL function’s body. If the test values were of different data types, a crash might result; moreover such situations could be abused to allow disclosure of portions of server memory. (CVE-2016-5423)
•
Fix client programs’ handling of special characters in database and role names (Noah Misch, Nathan Bossart, Michael Paquier) Numerous places in vacuumdb and other client programs could become confused by database and role names containing double quotes or backslashes. Tighten up quoting rules to make that safe. Also, ensure that when a conninfo string is used as a database name parameter to these programs, it is correctly treated as such throughout. Fix handling of paired double quotes in psql’s \connect and \password commands to match the documentation. Introduce a new -reuse-previous option in psql’s \connect command to allow explicit control of whether to re-use connection parameters from a previous connection. (Without this, the choice is based on whether the database name looks like a conninfo string, as before.) This allows secure handling of database names containing special characters in pg_dumpall scripts. pg_dumpall now refuses to deal with database and role names containing carriage returns or newlines, as it seems impractical to quote those characters safely on Windows. In future we may reject such names on the server side, but that step has not been taken yet. These are considered security fixes because crafted object names containing special characters could have been used to execute commands with superuser privileges the next time a superuser executes pg_dumpall or other routine maintenance operations. (CVE-2016-5424)
•
Fix corner-case misbehaviors for IS NULL/IS NOT NULL applied to nested composite values (Andrew Gierth, Tom Lane) The SQL standard specifies that IS NULL should return TRUE for a row of all null values (thus ROW(NULL,NULL) IS NULL yields TRUE), but this is not meant to apply recursively (thus ROW(NULL, ROW(NULL,NULL)) IS NULL yields FALSE). The core executor got this right, but certain planner optimizations treated the test as recursive (thus producing TRUE in both cases), and contrib/postgres_fdw could produce remote queries that misbehaved similarly.
•
Make the inet and cidr data types properly reject IPv6 addresses with too many colon-separated fields (Tom Lane)
•
Prevent crash in close_ps() (the point ## lseg operator) for NaN input coordinates (Tom Lane) Make it return NULL instead of crashing.
2081
Appendix E. Release Notes •
Fix several one-byte buffer over-reads in to_number() (Peter Eisentraut) In several cases the to_number() function would read one more character than it should from the input string. There is a small chance of a crash, if the input happens to be adjacent to the end of memory.
•
Avoid unsafe intermediate state during expensive paths through heap_update() (Masahiko Sawada, Andres Freund) Previously, these cases locked the target tuple (by setting its XMAX) but did not WAL-log that action, thus risking data integrity problems if the page were spilled to disk and then a database crash occurred before the tuple update could be completed.
•
Avoid crash in postgres -C when the specified variable has a null string value (Michael Paquier)
•
Avoid consuming a transaction ID during VACUUM (Alexander Korotkov) Some cases in VACUUM unnecessarily caused an XID to be assigned to the current transaction. Normally this is negligible, but if one is up against the XID wraparound limit, consuming more XIDs during anti-wraparound vacuums is a very bad thing.
•
Avoid canceling hot-standby queries during VACUUM FREEZE (Simon Riggs, Álvaro Herrera) VACUUM FREEZE on an otherwise-idle master server could result in unnecessary cancellations of
queries on its standby servers. •
When
a manual ANALYZE specifies a changes_since_analyze counter (Tom Lane)
column
list,
don’t
reset
the
table’s
If we’re only analyzing some columns, we should not prevent routine auto-analyze from happening for the other columns. •
Fix ANALYZE’s overestimation of n_distinct for a unique or nearly-unique column with many null entries (Tom Lane) The nulls could get counted as though they were themselves distinct values, leading to serious planner misestimates in some types of queries.
•
Prevent autovacuum from starting multiple workers for the same shared catalog (Álvaro Herrera) Normally this isn’t much of a problem because the vacuum doesn’t take long anyway; but in the case of a severely bloated catalog, it could result in all but one worker uselessly waiting instead of doing useful work on other tables.
•
Prevent infinite loop in GiST index build for geometric columns containing NaN component values (Tom Lane)
•
Fix contrib/btree_gin to handle the smallest possible bigint value correctly (Peter Eisentraut)
•
Teach libpq to correctly decode server version from future servers (Peter Eisentraut) It’s planned to switch to two-part instead of three-part server version numbers for releases after 9.6. Make sure that PQserverVersion() returns the correct value for such cases.
•
Fix ecpg’s code for unsigned long long array elements (Michael Meskes)
•
In pg_dump with both -c and -C options, avoid emitting an unwanted CREATE SCHEMA public command (David Johnston, Tom Lane)
•
Make pg_basebackup accept -Z 0 as specifying no compression (Fujii Masao)
•
Fix makefiles’ rule for building AIX shared libraries to be safe for parallel make (Noah Misch)
•
Fix TAP tests and MSVC scripts to work when build directory’s path name contains spaces (Michael Paquier, Kyotaro Horiguchi)
2082
Appendix E. Release Notes •
Make regression tests safe for Danish and Welsh locales (Jeff Janes, Tom Lane) Change some test data that triggered the unusual sorting rules of these locales.
•
Update our copy of the timezone code to match IANA’s tzcode release 2016c (Tom Lane) This is needed to cope with anticipated future changes in the time zone data files. It also fixes some corner-case bugs in coping with unusual time zones.
•
Update time zone data files to tzdata release 2016f for DST law changes in Kemerovo and Novosibirsk, plus historical corrections for Azerbaijan, Belarus, and Morocco.
E.27. Release 9.2.17 Release date: 2016-05-12 This release contains a variety of fixes from 9.2.16. For information about new features in the 9.2 major release, see Section E.44.
E.27.1. Migration to Version 9.2.17 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.27.2. Changes •
Clear the OpenSSL error queue before OpenSSL calls, rather than assuming it’s clear already; and make sure we leave it clear afterwards (Peter Geoghegan, Dave Vitek, Peter Eisentraut) This change prevents problems when there are multiple connections using OpenSSL within a single process and not all the code involved follows the same rules for when to clear the error queue. Failures have been reported specifically when a client application uses SSL connections in libpq concurrently with SSL connections using the PHP, Python, or Ruby wrappers for OpenSSL. It’s possible for similar problems to arise within the server as well, if an extension module establishes an outgoing SSL connection.
•
Fix “failed to build any N -way joins” planner error with a full join enclosed in the right-hand side of a left join (Tom Lane)
•
Fix incorrect handling of equivalence-class tests in multilevel nestloop plans (Tom Lane) Given a three-or-more-way equivalence class of variables, such as X.X = Y.Y = Z.Z, it was possible for the planner to omit some of the tests needed to enforce that all the variables are actually equal, leading to join rows being output that didn’t satisfy the WHERE clauses. For various reasons, erroneous plans were seldom selected in practice, so that this bug has gone undetected for a long time.
•
Fix possible misbehavior of TH, th, and Y,YYY format codes in to_timestamp() (Tom Lane) These could advance off the end of the input string, causing subsequent format codes to read garbage.
•
Fix dumping of rules and views in which the array argument of a value operator ANY (array ) construct is a sub-SELECT (Tom Lane)
2083
Appendix E. Release Notes •
Make pg_regress use a startup timeout from the PGCTLTIMEOUT environment variable, if that’s set (Tom Lane) This is for consistency with a behavior recently added to pg_ctl; it eases automated testing on slow machines.
•
Fix pg_upgrade to correctly restore extension membership for operator families containing only one operator class (Tom Lane) In such a case, the operator family was restored into the new database, but it was no longer marked as part of the extension. This had no immediate ill effects, but would cause later pg_dump runs to emit output that would cause (harmless) errors on restore.
•
Back-port 9.4-era memory-barrier code changes into 9.2 and 9.3 (Tom Lane) These changes were not originally needed in pre-9.4 branches, but we recently back-patched a fix that expected the barrier code to work properly. Only IA64 (when using icc), HPPA, and Alpha platforms are affected.
•
Reduce the number of SysV semaphores used by a build configured with --disable-spinlocks (Tom Lane)
•
Rename internal function strtoi() to strtoint() to avoid conflict with a NetBSD library function (Thomas Munro)
•
Fix reporting of errors from bind() and listen() system calls on Windows (Tom Lane)
•
Reduce verbosity of compiler output when building with Microsoft Visual Studio (Christian Ullrich)
•
Avoid possibly-unsafe use of Windows’ FormatMessage() function (Christian Ullrich) Use the FORMAT_MESSAGE_IGNORE_INSERTS flag where appropriate. No live bug is known to exist here, but it seems like a good idea to be careful.
•
Update time zone data files to tzdata release 2016d for DST law changes in Russia and Venezuela. There are new zone names Europe/Kirov and Asia/Tomsk to reflect the fact that these regions now have different time zone histories from adjacent regions.
E.28. Release 9.2.16 Release date: 2016-03-31 This release contains a variety of fixes from 9.2.15. For information about new features in the 9.2 major release, see Section E.44.
E.28.1. Migration to Version 9.2.16 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.28.2. Changes •
Fix incorrect handling of NULL index entries in indexed ROW() comparisons (Tom Lane)
2084
Appendix E. Release Notes An index search using a row comparison such as ROW(a, b) > ROW(’x’, ’y’) would stop upon reaching a NULL entry in the b column, ignoring the fact that there might be non-NULL b values associated with later values of a. •
Avoid unlikely data-loss scenarios due to renaming files without adequate fsync() calls before and after (Michael Paquier, Tomas Vondra, Andres Freund)
•
Correctly handle cases where pg_subtrans is close to XID wraparound during server startup (Jeff Janes)
•
Fix corner-case crash due to trying to free localeconv() output strings more than once (Tom Lane)
•
Fix parsing of affix files for ispell dictionaries (Tom Lane) The code could go wrong if the affix file contained any characters whose byte length changes during case-folding, for example I in Turkish UTF8 locales.
•
Avoid use of sscanf() to parse ispell dictionary files (Artur Zakirov) This dodges a portability problem on FreeBSD-derived platforms (including macOS).
•
Avoid a crash on old Windows versions (before 7SP1/2008R2SP1) with an AVX2-capable CPU and a Postgres build done with Visual Studio 2013 (Christian Ullrich) This is a workaround for a bug in Visual Studio 2013’s runtime library, which Microsoft have stated they will not fix in that version.
•
Fix psql’s tab completion logic to handle multibyte characters properly (Kyotaro Horiguchi, Robert Haas)
•
Fix psql’s tab completion for SECURITY LABEL (Tom Lane) Pressing TAB after SECURITY LABEL might cause a crash or offering of inappropriate keywords.
•
Make pg_ctl accept a wait timeout from the PGCTLTIMEOUT environment variable, if none is specified on the command line (Noah Misch) This eases testing of slower buildfarm members by allowing them to globally specify a longer-thannormal timeout for postmaster startup and shutdown.
•
Fix incorrect test for Windows service status in pg_ctl (Manuel Mathar) The previous set of minor releases attempted to fix pg_ctl to properly determine whether to send log messages to Window’s Event Log, but got the test backwards.
•
Fix pgbench to correctly handle the combination of -C and -M prepared options (Tom Lane)
•
In PL/Perl, properly translate empty Postgres arrays into empty Perl arrays (Alex Hunsaker)
•
Make PL/Python cope with function names that aren’t valid Python identifiers (Jim Nasby)
•
Fix multiple mistakes in the statistics returned by contrib/pgstattuple’s pgstatindex() function (Tom Lane)
•
Remove dependency on psed in MSVC builds, since it’s no longer provided by core Perl (Michael Paquier, Andrew Dunstan)
•
Update time zone data files to tzdata release 2016c for DST law changes in Azerbaijan, Chile, Haiti, Palestine, and Russia (Altai, Astrakhan, Kirov, Sakhalin, Ulyanovsk regions), plus historical corrections for Lithuania, Moldova, and Russia (Kaliningrad, Samara, Volgograd).
2085
Appendix E. Release Notes
E.29. Release 9.2.15 Release date: 2016-02-11 This release contains a variety of fixes from 9.2.14. For information about new features in the 9.2 major release, see Section E.44.
E.29.1. Migration to Version 9.2.15 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.29.2. Changes •
Fix infinite loops and buffer-overrun problems in regular expressions (Tom Lane) Very large character ranges in bracket expressions could cause infinite loops in some cases, and memory overwrites in other cases. (CVE-2016-0773)
•
Perform an immediate shutdown if the postmaster.pid file is removed (Tom Lane) The postmaster now checks every minute or so that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The main motivation for this change is to ensure that failed buildfarm runs will get cleaned up without manual intervention; but it also serves to limit the bad effects if a DBA forcibly removes postmaster.pid and then starts a new postmaster.
•
In SERIALIZABLE transaction isolation mode, serialization anomalies could be missed due to race conditions during insertions (Kevin Grittner, Thomas Munro)
•
Fix failure to emit appropriate WAL records when doing ALTER TABLE ... SET TABLESPACE for unlogged relations (Michael Paquier, Andres Freund) Even though the relation’s data is unlogged, the move must be logged or the relation will be inaccessible after a standby is promoted to master.
•
Fix possible misinitialization of unlogged relations at the end of crash recovery (Andres Freund, Michael Paquier)
•
Fix ALTER COLUMN TYPE to reconstruct inherited check constraints properly (Tom Lane)
•
Fix REASSIGN OWNED to change ownership of composite types properly (Álvaro Herrera)
•
Fix REASSIGN OWNED and ALTER OWNER to correctly update granted-permissions lists when changing owners of data types, foreign data wrappers, or foreign servers (Bruce Momjian, Álvaro Herrera)
•
Fix REASSIGN OWNED to ignore foreign user mappings, rather than fail (Álvaro Herrera)
•
Add more defenses against bad planner cost estimates for GIN index scans when the index’s internal statistics are very out-of-date (Tom Lane)
•
Make planner cope with hypothetical GIN indexes suggested by an index advisor plug-in (Julien Rouhaud)
•
Fix dumping of whole-row Vars in ROW() and VALUES() lists (Tom Lane)
•
Fix possible internal overflow in numeric division (Dean Rasheed)
2086
Appendix E. Release Notes •
Fix enforcement of restrictions inside parentheses within regular expression lookahead constraints (Tom Lane) Lookahead constraints aren’t allowed to contain backrefs, and parentheses within them are always considered non-capturing, according to the manual. However, the code failed to handle these cases properly inside a parenthesized subexpression, and would give unexpected results.
•
Conversion of regular expressions to indexscan bounds could produce incorrect bounds from regexps containing lookahead constraints (Tom Lane)
•
Fix regular-expression compiler to handle loops of constraint arcs (Tom Lane) The code added for CVE-2007-4772 was both incomplete, in that it didn’t handle loops involving more than one state, and incorrect, in that it could cause assertion failures (though there seem to be no bad consequences of that in a non-assert build). Multi-state loops would cause the compiler to run until the query was canceled or it reached the too-many-states error condition.
•
Improve memory-usage accounting in regular-expression compiler (Tom Lane) This causes the code to emit “regular expression is too complex” errors in some cases that previously used unreasonable amounts of time and memory.
•
Improve performance of regular-expression compiler (Tom Lane)
•
Make %h and %r escapes in log_line_prefix work for messages emitted due to log_connections (Tom Lane) Previously, %h/%r started to work just after a new session had emitted the “connection received” log message; now they work for that message too.
•
On Windows, ensure the shared-memory mapping handle gets closed in child processes that don’t need it (Tom Lane, Amit Kapila) This oversight resulted in failure to recover from crashes whenever logging_collector is turned on.
•
Fix possible failure to detect socket EOF in non-blocking mode on Windows (Tom Lane) It’s not entirely clear whether this problem can happen in pre-9.5 branches, but if it did, the symptom would be that a walsender process would wait indefinitely rather than noticing a loss of connection.
•
Avoid leaking a token handle during SSPI authentication (Christian Ullrich)
•
In psql, ensure that libreadline’s idea of the screen size is updated when the terminal window size changes (Merlin Moncure) Previously, libreadline did not notice if the window was resized during query output, leading to strange behavior during later input of multiline queries.
•
Fix psql’s \det command to interpret its pattern argument the same way as other \d commands with potentially schema-qualified patterns do (Reece Hart)
•
Avoid possible crash in psql’s \c command when previous connection was via Unix socket and command specifies a new hostname and same username (Tom Lane)
•
In pg_ctl start -w, test child process status directly rather than relying on heuristics (Tom Lane, Michael Paquier) Previously, pg_ctl relied on an assumption that the new postmaster would always create postmaster.pid within five seconds. But that can fail on heavily-loaded systems, causing pg_ctl to report incorrectly that the postmaster failed to start.
2087
Appendix E. Release Notes Except on Windows, this change also means that a pg_ctl start -w done immediately after another such command will now reliably fail, whereas previously it would report success if done within two seconds of the first command. •
In pg_ctl start -w, don’t attempt to use a wildcard listen address to connect to the postmaster (Kondo Yuta) On Windows, pg_ctl would fail to detect postmaster startup if listen_addresses is set to 0.0.0.0 or ::, because it would try to use that value verbatim as the address to connect to, which doesn’t work. Instead assume that 127.0.0.1 or ::1, respectively, is the right thing to use.
•
In pg_ctl on Windows, check service status to decide where to send output, rather than checking if standard output is a terminal (Michael Paquier)
•
In pg_dump and pg_basebackup, adopt the GNU convention for handling tar-archive members exceeding 8GB (Tom Lane) The POSIX standard for tar file format does not allow archive member files to exceed 8GB, but most modern implementations of tar support an extension that fixes that. Adopt this extension so that pg_dump with -Ft no longer fails on tables with more than 8GB of data, and so that pg_basebackup can handle files larger than 8GB. In addition, fix some portability issues that could cause failures for members between 4GB and 8GB on some platforms. Potentially these problems could cause unrecoverable data loss due to unreadable backup files.
•
Fix assorted corner-case bugs in pg_dump’s processing of extension member objects (Tom Lane)
•
Make pg_dump mark a view’s triggers as needing to be processed after its rule, to prevent possible failure during parallel pg_restore (Tom Lane)
•
Ensure that relation option values are properly quoted in pg_dump (Kouhei Sutou, Tom Lane) A reloption value that isn’t a simple identifier or number could lead to dump/reload failures due to syntax errors in CREATE statements issued by pg_dump. This is not an issue with any reloption currently supported by core PostgreSQL, but extensions could allow reloptions that cause the problem.
•
Fix pg_upgrade’s file-copying code to handle errors properly on Windows (Bruce Momjian)
•
Install guards in pgbench against corner-case overflow conditions during evaluation of scriptspecified division or modulo operators (Fabien Coelho, Michael Paquier)
•
Fix failure to localize messages emitted by pg_receivexlog and pg_recvlogical (Ioseph Kim)
•
Avoid dump/reload problems when using both plpython2 and plpython3 (Tom Lane) In principle, both versions of PL/Python can be used in the same database, though not in the same session (because the two versions of libpython cannot safely be used concurrently). However, pg_restore and pg_upgrade both do things that can fall foul of the same-session restriction. Work around that by changing the timing of the check.
•
Fix PL/Python regression tests to pass with Python 3.5 (Peter Eisentraut)
•
Prevent certain PL/Java parameters from being set by non-superusers (Noah Misch) This change mitigates a PL/Java security bug (CVE-2016-0766), which was fixed in PL/Java by marking these parameters as superuser-only. To fix the security hazard for sites that update PostgreSQL more frequently than PL/Java, make the core code aware of them also.
•
Improve libpq’s handling of out-of-memory situations (Michael Paquier, Amit Kapila, Heikki Linnakangas)
•
Fix order of arguments in ecpg-generated typedef statements (Michael Meskes)
•
Use %g not %f format in ecpg’s PGTYPESnumeric_from_double() (Tom Lane)
2088
Appendix E. Release Notes •
Fix ecpg-supplied header files to not contain comments continued from a preprocessor directive line onto the next line (Michael Meskes) Such a comment is rejected by ecpg. It’s not yet clear whether ecpg itself should be changed.
•
Ensure that contrib/pgcrypto’s crypt() function can be interrupted by query cancel (Andreas Karlsson)
•
Accept flex versions later than 2.5.x (Tom Lane, Michael Paquier) Now that flex 2.6.0 has been released, the version checks in our build scripts needed to be adjusted.
•
Install our missing script where PGXS builds can find it (Jim Nasby) This allows sane behavior in a PGXS build done on a machine where build tools such as bison are missing.
•
Ensure that dynloader.h is included in the installed header files in MSVC builds (Bruce Momjian, Michael Paquier)
•
Add variant regression test expected-output file to match behavior of current libxml2 (Tom Lane) The fix for libxml2’s CVE-2015-7499 causes it not to output error context reports in some cases where it used to do so. This seems to be a bug, but we’ll probably have to live with it for some time, so work around it.
•
Update time zone data files to tzdata release 2016a for DST law changes in Cayman Islands, Metlakatla, and Trans-Baikal Territory (Zabaykalsky Krai), plus historical corrections for Pakistan.
E.30. Release 9.2.14 Release date: 2015-10-08 This release contains a variety of fixes from 9.2.13. For information about new features in the 9.2 major release, see Section E.44.
E.30.1. Migration to Version 9.2.14 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.30.2. Changes •
Fix contrib/pgcrypto to detect and report too-short crypt() salts (Josh Kupershmidt) Certain invalid salt arguments crashed the server or disclosed a few bytes of server memory. We have not ruled out the viability of attacks that arrange for presence of confidential information in the disclosed bytes, but they seem unlikely. (CVE-2015-5288)
•
Fix subtransaction cleanup after a portal (cursor) belonging to an outer subtransaction fails (Tom Lane, Michael Paquier) A function executed in an outer-subtransaction cursor could cause an assertion failure or crash by referencing a relation created within an inner subtransaction.
2089
Appendix E. Release Notes •
Fix insertion of relations into the relation cache “init file” (Tom Lane) An
oversight
in
a
patch
in
the
most
recent
minor
releases
caused
pg_trigger_tgrelid_tgname_index to be omitted from the init file. Subsequent sessions
detected this, then deemed the init file to be broken and silently ignored it, resulting in a significant degradation in session startup time. In addition to fixing the bug, install some guards so that any similar future mistake will be more obvious. •
Avoid O(N^2) behavior when inserting many tuples into a SPI query result (Neil Conway)
•
Improve LISTEN startup time when there are many unread notifications (Matt Newell)
•
Back-patch 9.3-era addition of per-resource-owner lock caches (Jeff Janes) This substantially improves performance when pg_dump tries to dump a large number of tables.
•
Disable SSL renegotiation by default (Michael Paquier, Andres Freund) While use of SSL renegotiation is a good idea in theory, we have seen too many bugs in practice, both in the underlying OpenSSL library and in our usage of it. Renegotiation will be removed entirely in 9.5 and later. In the older branches, just change the default value of ssl_renegotiation_limit to zero (disabled).
•
Lower the minimum values of the *_freeze_max_age parameters (Andres Freund) This is mainly to make tests of related behavior less time-consuming, but it may also be of value for installations with limited disk space.
•
Limit the maximum value of wal_buffers to 2GB to avoid server crashes (Josh Berkus)
•
Fix rare internal overflow in multiplication of numeric values (Dean Rasheed)
•
Guard against hard-to-reach stack overflows involving record types, range types, json, jsonb, tsquery, ltxtquery and query_int (Noah Misch)
•
Fix handling of DOW and DOY in datetime input (Greg Stark) These tokens aren’t meant to be used in datetime values, but previously they resulted in opaque internal error messages rather than “invalid input syntax”.
•
Add more query-cancel checks to regular expression matching (Tom Lane)
•
Add recursion depth protections to regular expression, SIMILAR TO, and LIKE matching (Tom Lane) Suitable search patterns and a low stack depth limit could lead to stack-overrun crashes.
•
Fix potential infinite loop in regular expression execution (Tom Lane) A search pattern that can apparently match a zero-length string, but actually doesn’t match because of a back reference, could lead to an infinite loop.
•
In regular expression execution, correctly record match data for capturing parentheses within a quantifier even when the match is zero-length (Tom Lane)
•
Fix low-memory failures in regular expression compilation (Andreas Seltenreich)
•
Fix low-probability memory leak during regular expression execution (Tom Lane)
•
Fix rare low-memory failure in lock cleanup during transaction abort (Tom Lane)
•
Fix “unexpected out-of-memory situation during sort” errors when using tuplestores with small work_mem settings (Tom Lane)
•
Fix very-low-probability stack overrun in qsort (Tom Lane)
•
Fix “invalid memory alloc request size” failure in hash joins with large work_mem settings (Tomas Vondra, Tom Lane)
2090
Appendix E. Release Notes •
Fix assorted planner bugs (Tom Lane) These mistakes could lead to incorrect query plans that would give wrong answers, or to assertion failures in assert-enabled builds, or to odd planner errors such as “could not devise a query plan for the given query”, “could not find pathkey item to sort”, “plan should not reference subplan’s variable”, or “failed to assign all NestLoopParams to plan nodes”. Thanks are due to Andreas Seltenreich and Piotr Stefaniak for fuzz testing that exposed these problems.
•
Improve planner’s performance for UPDATE/DELETE on large inheritance sets (Tom Lane, Dean Rasheed)
•
Ensure standby promotion trigger files are removed at postmaster startup (Michael Paquier, Fujii Masao) This prevents unwanted promotion from occurring if these files appear in a database backup that is used to initialize a new standby server.
•
During postmaster shutdown, ensure that per-socket lock files are removed and listen sockets are closed before we remove the postmaster.pid file (Tom Lane) This avoids race-condition failures if an external script attempts to start a new postmaster as soon as pg_ctl stop returns.
•
Fix postmaster’s handling of a startup-process crash during crash recovery (Tom Lane) If, during a crash recovery cycle, the startup process crashes without having restored database consistency, we’d try to launch a new startup process, which typically would just crash again, leading to an infinite loop.
•
Do not print a WARNING when an autovacuum worker is already gone when we attempt to signal it, and reduce log verbosity for such signals (Tom Lane)
•
Prevent autovacuum launcher from sleeping unduly long if the server clock is moved backwards a large amount (Álvaro Herrera)
•
Ensure that cleanup of a GIN index’s pending-insertions list is interruptable by cancel requests (Jeff Janes)
•
Allow all-zeroes pages in GIN indexes to be reused (Heikki Linnakangas) Such a page might be left behind after a crash.
•
Fix handling of all-zeroes pages in SP-GiST indexes (Heikki Linnakangas) VACUUM attempted to recycle such pages, but did so in a way that wasn’t crash-safe.
•
Fix off-by-one error that led to otherwise-harmless warnings about “apparent wraparound” in subtrans/multixact truncation (Thomas Munro)
•
Fix misreporting of CONTINUE and MOVE statement types in PL/pgSQL’s error context messages (Pavel Stehule, Tom Lane)
Fix PL/Python crash when returning the string representation of a record result (Tom Lane)
•
Fix some places in PL/Tcl that neglected to check for failure of malloc() calls (Michael Paquier, Álvaro Herrera)
•
In contrib/isn, fix output of ISBN-13 numbers that begin with 979 (Fabien Coelho) EANs beginning with 979 (but not 9790) are considered ISBNs, but they must be printed in the new 13-digit format, not the 10-digit format.
•
Fix contrib/sepgsql’s handling of SELECT INTO statements (Kohei KaiGai)
2091
Appendix E. Release Notes •
Improve libpq’s handling of out-of-memory conditions (Michael Paquier, Heikki Linnakangas)
•
Fix memory leaks and missing out-of-memory checks in ecpg (Michael Paquier)
•
Fix psql’s code for locale-aware formatting of numeric output (Tom Lane) The formatting code invoked by \pset numericlocale on did the wrong thing for some uncommon cases such as numbers with an exponent but no decimal point. It could also mangle already-localized output from the money data type.
•
Prevent crash in psql’s \c command when there is no current connection (Noah Misch)
•
Make pg_dump handle inherited NOT VALID check constraints correctly (Tom Lane)
•
Fix selection of default zlib compression level in pg_dump’s directory output format (Andrew Dunstan)
•
Ensure that temporary files created during a pg_dump run with tar-format output are not worldreadable (Michael Paquier)
•
Fix pg_dump and pg_upgrade to support cases where the postgres or template1 database is in a non-default tablespace (Marti Raudsepp, Bruce Momjian)
•
Fix pg_dump to handle object privileges sanely when dumping from a server too old to have a particular privilege type (Tom Lane) When dumping data types from pre-9.2 servers, and when dumping functions or procedural languages from pre-7.3 servers, pg_dump would produce GRANT/REVOKE commands that revoked the owner’s grantable privileges and instead granted all privileges to PUBLIC. Since the privileges involved are just USAGE and EXECUTE, this isn’t a security problem, but it’s certainly a surprising representation of the older systems’ behavior. Fix it to leave the default privilege state alone in these cases.
•
Fix pg_dump to dump shell types (Tom Lane) Shell types (that is, not-yet-fully-defined types) aren’t useful for much, but nonetheless pg_dump should dump them.
•
Fix assorted minor memory leaks in pg_dump and other client-side programs (Michael Paquier)
•
Fix spinlock assembly code for PPC hardware to be compatible with AIX’s native assembler (Tom Lane) Building with gcc didn’t work if gcc had been configured to use the native assembler, which is becoming more common.
•
On AIX, test the -qlonglong compiler option rather than just assuming it’s safe to use (Noah Misch)
•
On AIX, use -Wl,-brtllib link option to allow symbols to be resolved at runtime (Noah Misch) Perl relies on this ability in 5.8.0 and later.
•
Avoid use of inline functions when compiling with 32-bit xlc, due to compiler bugs (Noah Misch)
•
Use librt for sched_yield() when necessary, which it is on some Solaris versions (Oskari Saarenmaa)
•
Fix Windows install.bat script to handle target directory names that contain spaces (Heikki Linnakangas)
•
Make the numeric form of the PostgreSQL version number (e.g., 90405) readily available to extension Makefiles, as a variable named VERSION_NUM (Michael Paquier)
2092
Appendix E. Release Notes •
Update time zone data files to tzdata release 2015g for DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, and Uruguay. There is a new zone name America/Fort_Nelson for the Canadian Northern Rockies.
E.31. Release 9.2.13 Release date: 2015-06-12 This release contains a small number of fixes from 9.2.12. For information about new features in the 9.2 major release, see Section E.44.
E.31.1. Migration to Version 9.2.13 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
E.31.2. Changes •
Fix rare failure to invalidate relation cache init file (Tom Lane) With just the wrong timing of concurrent activity, a VACUUM FULL on a system catalog might fail to update the “init file” that’s used to avoid cache-loading work for new sessions. This would result in later sessions being unable to access that catalog at all. This is a very ancient bug, but it’s so hard to trigger that no reproducible case had been seen until recently.
•
Avoid deadlock between incoming sessions and CREATE/DROP DATABASE (Tom Lane) A new session starting in a database that is the target of a DROP DATABASE command, or is the template for a CREATE DATABASE command, could cause the command to wait for five seconds and then fail, even if the new session would have exited before that.
E.32. Release 9.2.12 Release date: 2015-06-04 This release contains a small number of fixes from 9.2.11. For information about new features in the 9.2 major release, see Section E.44.
E.32.1. Migration to Version 9.2.12 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.11, see Section E.33.
2093
Appendix E. Release Notes
E.32.2. Changes •
Avoid failures while fsync’ing data directory during crash restart (Abhijit Menon-Sen, Tom Lane) In the previous minor releases we added a patch to fsync everything in the data directory after a crash. Unfortunately its response to any error condition was to fail, thereby preventing the server from starting up, even when the problem was quite harmless. An example is that an unwritable file in the data directory would prevent restart on some platforms; but it is common to make SSL certificate files unwritable by the server. Revise this behavior so that permissions failures are ignored altogether, and other types of failures are logged but do not prevent continuing.
•
Fix pg_get_functiondef() to show functions’ LEAKPROOF property, if set (Jeevan Chalke)
•
Remove configure’s check prohibiting linking to a threaded libpython on OpenBSD (Tom Lane) The failure this restriction was meant to prevent seems to not be a problem anymore on current OpenBSD versions.
•
Allow libpq to use TLS protocol versions beyond v1 (Noah Misch) For a long time, libpq was coded so that the only SSL protocol it would allow was TLS v1. Now that newer TLS versions are becoming popular, allow it to negotiate the highest commonly-supported TLS version with the server. (PostgreSQL servers were already capable of such negotiation, so no change is needed on the server side.) This is a back-patch of a change already released in 9.4.0.
E.33. Release 9.2.11 Release date: 2015-05-22 This release contains a variety of fixes from 9.2.10. For information about new features in the 9.2 major release, see Section E.44.
E.33.1. Migration to Version 9.2.11 A dump/restore is not required for those running 9.2.X. However, if you use contrib/citext’s regexp_matches() functions, see the changelog entry below about that. Also, if you are upgrading from a version earlier than 9.2.10, see Section E.34.
E.33.2. Changes •
Avoid possible crash when client disconnects just before the authentication timeout expires (Benkocs Norbert Attila) If the timeout interrupt fired partway through the session shutdown sequence, SSL-related state would be freed twice, typically causing a crash and hence denial of service to other sessions. Experimentation shows that an unauthenticated remote attacker could trigger the bug somewhat consistently, hence treat as security issue. (CVE-2015-3165)
•
Improve detection of system-call failures (Noah Misch)
2094
Appendix E. Release Notes Our replacement implementation of snprintf() failed to check for errors reported by the underlying system library calls; the main case that might be missed is out-of-memory situations. In the worst case this might lead to information exposure, due to our code assuming that a buffer had been overwritten when it hadn’t been. Also, there were a few places in which security-relevant calls of other system library functions did not check for failure. It remains possible that some calls of the *printf() family of functions are vulnerable to information disclosure if an out-of-memory error occurs at just the wrong time. We judge the risk to not be large, but will continue analysis in this area. (CVE-2015-3166) •
In contrib/pgcrypto, uniformly report decryption failures as “Wrong key or corrupt data” (Noah Misch) Previously, some cases of decryption with an incorrect key could report other error message texts. It has been shown that such variance in error reports can aid attackers in recovering keys from other systems. While it’s unknown whether pgcrypto’s specific behaviors are likewise exploitable, it seems better to avoid the risk by using a one-size-fits-all message. (CVE-2015-3167)
•
Fix incorrect declaration of contrib/citext’s regexp_matches() functions (Tom Lane) These functions should return setof text[], like the core functions they are wrappers for; but they were incorrectly declared as returning just text[]. This mistake had two results: first, if there was no match you got a scalar null result, whereas what you should get is an empty set (zero rows). Second, the g flag was effectively ignored, since you would get only one result array even if there were multiple matches. While the latter behavior is clearly a bug, there might be applications depending on the former behavior; therefore the function declarations will not be changed by default until PostgreSQL 9.5. In pre-9.5 branches, the old behavior exists in version 1.0 of the citext extension, while we have provided corrected declarations in version 1.1 (which is not installed by default). To adopt the fix in pre-9.5 branches, execute ALTER EXTENSION citext UPDATE TO ’1.1’ in each database in which citext is installed. (You can also “update” back to 1.0 if you need to undo that.) Be aware that either update direction will require dropping and recreating any views or rules that use citext’s regexp_matches() functions.
•
Fix incorrect checking of deferred exclusion constraints after a HOT update (Tom Lane) If a new row that potentially violates a deferred exclusion constraint is HOT-updated (that is, no indexed columns change and the row can be stored back onto the same table page) later in the same transaction, the exclusion constraint would be reported as violated when the check finally occurred, even if the row(s) the new row originally conflicted with had been deleted.
•
Fix planning of star-schema-style queries (Tom Lane) Sometimes, efficient scanning of a large table requires that index parameters be provided from more than one other table (commonly, dimension tables whose keys are needed to index a large fact table). The planner should be able to find such plans, but an overly restrictive search heuristic prevented it.
•
Prevent improper reordering of antijoins (NOT EXISTS joins) versus other outer joins (Tom Lane) This oversight in the planner has been observed to cause “could not find RelOptInfo for given relids” errors, but it seems possible that sometimes an incorrect query plan might get past that consistency check and result in silently-wrong query output.
•
Fix incorrect matching of subexpressions in outer-join plan nodes (Tom Lane) Previously, if textually identical non-strict subexpressions were used both above and below an outer join, the planner might try to re-use the value computed below the join, which would be incorrect because the executor would force the value to NULL in case of an unmatched outer row.
2095
Appendix E. Release Notes •
Fix GEQO planner to cope with failure of its join order heuristic (Tom Lane) This oversight has been seen to lead to “failed to join all relations together” errors in queries involving LATERAL, and that might happen in other cases as well.
•
Fix possible deadlock at startup when max_prepared_transactions is too small (Heikki Linnakangas)
•
Don’t archive useless preallocated WAL files after a timeline switch (Heikki Linnakangas)
•
Avoid “cannot GetMultiXactIdMembers() during recovery” error (Álvaro Herrera)
•
Recursively fsync() the data directory after a crash (Abhijit Menon-Sen, Robert Haas) This ensures consistency if another crash occurs shortly later. (The second crash would have to be a system-level crash, not just a database crash, for there to be a problem.)
•
Fix autovacuum launcher’s possible failure to shut down, if an error occurs after it receives SIGTERM (Álvaro Herrera)
•
Cope with unexpected signals in LockBufferForCleanup() (Andres Freund) This oversight could result in spurious errors about “multiple backends attempting to wait for pincount 1”.
•
Fix crash when doing COPY IN to a table with check constraints that contain whole-row references (Tom Lane) The known failure case only crashes in 9.4 and up, but there is very similar code in 9.3 and 9.2, so back-patch those branches as well.
•
Avoid waiting for WAL flush or synchronous replication during commit of a transaction that was read-only so far as the user is concerned (Andres Freund) Previously, a delay could occur at commit in transactions that had written WAL due to HOT page pruning, leading to undesirable effects such as sessions getting stuck at startup if all synchronous replicas are down. Sessions have also been observed to get stuck in catchup interrupt processing when using synchronous replication; this will fix that problem as well.
•
Fix crash when manipulating hash indexes on temporary tables (Heikki Linnakangas)
•
Fix possible failure during hash index bucket split, if other processes are modifying the index concurrently (Tom Lane)
•
Check for interrupts while analyzing index expressions (Jeff Janes) ANALYZE executes index expressions many times; if there are slow functions in such an expression, it’s desirable to be able to cancel the ANALYZE before that loop finishes.
•
Ensure tableoid of a foreign table is reported correctly when a READ COMMITTED recheck occurs after locking rows in SELECT FOR UPDATE, UPDATE, or DELETE (Etsuro Fujita)
•
Add the name of the target server to object description strings for foreign-server user mappings (Álvaro Herrera)
•
Recommend setting include_realm to 1 when using Kerberos/GSSAPI/SSPI authentication (Stephen Frost) Without this, identically-named users from different realms cannot be distinguished. For the moment this is only a documentation change, but it will become the default setting in PostgreSQL 9.5.
•
Remove code for matching IPv4 pg_hba.conf entries to IPv4-in-IPv6 addresses (Tom Lane) This hack was added in 2003 in response to a report that some Linux kernels of the time would report IPv4 connections as having IPv4-in-IPv6 addresses. However, the logic was accidentally
2096
Appendix E. Release Notes broken in 9.0. The lack of any field complaints since then shows that it’s not needed anymore. Now we have reports that the broken code causes crashes on some systems, so let’s just remove it rather than fix it. (Had we chosen to fix it, that would make for a subtle and potentially security-sensitive change in the effective meaning of IPv4 pg_hba.conf entries, which does not seem like a good thing to do in minor releases.) •
Report WAL flush, not insert, position in IDENTIFY_SYSTEM replication command (Heikki Linnakangas) This avoids a possible startup failure in pg_receivexlog.
•
While shutting down service on Windows, periodically send status updates to the Service Control Manager to prevent it from killing the service too soon; and ensure that pg_ctl will wait for shutdown (Krystian Bigaj)
•
Reduce risk of network deadlock when using libpq’s non-blocking mode (Heikki Linnakangas) When sending large volumes of data, it’s important to drain the input buffer every so often, in case the server has sent enough response data to cause it to block on output. (A typical scenario is that the server is sending a stream of NOTICE messages during COPY FROM STDIN.) This worked properly in the normal blocking mode, but not so much in non-blocking mode. We’ve modified libpq to opportunistically drain input when it can, but a full defense against this problem requires application cooperation: the application should watch for socket read-ready as well as write-ready conditions, and be sure to call PQconsumeInput() upon read-ready.
•
In libpq, fix misparsing of empty values in URI connection strings (Thomas Fanghaenel)
•
Fix array handling in ecpg (Michael Meskes)
•
Fix psql to sanely handle URIs and conninfo strings as the first parameter to \connect (David Fetter, Andrew Dunstan, Álvaro Herrera) This syntax has been accepted (but undocumented) for a long time, but previously some parameters might be taken from the old connection instead of the given string, which was agreed to be undesirable.
•
Suppress incorrect complaints from psql on some platforms that it failed to write ~/.psql_history at exit (Tom Lane) This misbehavior was caused by a workaround for a bug in very old (pre-2006) versions of libedit. We fixed it by removing the workaround, which will cause a similar failure to appear for anyone still using such versions of libedit. Recommendation: upgrade that library, or use libreadline.
•
Fix pg_dump’s rule for deciding which casts are system-provided casts that should not be dumped (Tom Lane)
•
In pg_dump, fix failure to honor -Z compression level option together with -Fd (Michael Paquier)
•
Make pg_dump consider foreign key relationships between extension configuration tables while choosing dump order (Gilles Darold, Michael Paquier, Stephen Frost) This oversight could result in producing dumps that fail to reload because foreign key constraints are transiently violated.
•
Fix dumping of views that are just VALUES(...) but have column aliases (Tom Lane)
•
In pg_upgrade, force timeline 1 in the new cluster (Bruce Momjian) This change prevents upgrade failures caused by bogus complaints about missing WAL history files.
•
In pg_upgrade, check for improperly non-connectable databases before proceeding (Bruce Momjian)
2097
Appendix E. Release Notes •
In pg_upgrade, quote directory paths properly in the generated delete_old_cluster script (Bruce Momjian)
•
In pg_upgrade, preserve database-level freezing info properly (Bruce Momjian) This oversight could cause missing-clog-file errors for tables within the postgres and template1 databases.
•
Run pg_upgrade and pg_resetxlog with restricted privileges on Windows, so that they don’t fail when run by an administrator (Muhammad Asif Naeem)
•
Improve handling of readdir() failures when scanning directories in initdb and pg_basebackup (Marco Nenciarini)
•
Fix failure in pg_receivexlog (Andres Freund) A patch merge mistake in 9.2.10 led to “could not create archive status file” errors.
•
Fix slow sorting algorithm in contrib/intarray (Tom Lane)
•
Fix compile failure on Sparc V8 machines (Rob Rowan)
•
Update time zone data files to tzdata release 2015d for DST law changes in Egypt, Mongolia, and Palestine, plus historical changes in Canada and Chile. Also adopt revised zone abbreviations for the America/Adak zone (HST/HDT not HAST/HADT).
E.34. Release 9.2.10 Release date: 2015-02-05 This release contains a variety of fixes from 9.2.9. For information about new features in the 9.2 major release, see Section E.44.
E.34.1. Migration to Version 9.2.10 A dump/restore is not required for those running 9.2.X. However, if you are a Windows user and are using the “Norwegian (Bokmål)” locale, manual action is needed after the upgrade to replace any “Norwegian (Bokmål)_Norway” locale names stored in PostgreSQL system catalogs with the plain-ASCII alias “Norwegian_Norway”. For details see http://wiki.postgresql.org/wiki/Changes_To_Norwegian_Locale Also, if you are upgrading from a version earlier than 9.2.9, see Section E.35.
E.34.2. Changes •
Fix buffer overruns in to_char() (Bruce Momjian) When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241)
•
Fix buffer overrun in replacement *printf() functions (Tom Lane)
2098
Appendix E. Release Notes PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well. This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242) •
Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch) Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243)
•
Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas) If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message’s data as a new protocol message. An attacker able to submit crafted binary data within a command parameter might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244)
•
Fix information leak via constraint-violation error messages (Stephen Frost) Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161)
•
Lock down regression testing’s temporary installations on Windows (Noah Misch) Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067)
•
Cope with the Windows locale named “Norwegian (Bokmål)” (Heikki Linnakangas) Non-ASCII locale names are problematic since it’s not clear what encoding they should be represented in. Map the troublesome locale name to a plain-ASCII alias, “Norwegian_Norway”.
•
Avoid possible data corruption if ALTER DATABASE SET TABLESPACE is used to move a database to a new tablespace and then shortly later move it back to its original tablespace (Tom Lane)
•
Avoid corrupting tables when ANALYZE inside a transaction is rolled back (Andres Freund, Tom Lane, Michael Paquier) If the failing transaction had earlier removed the last index, rule, or trigger from the table, the table would be left in a corrupted state with the relevant pg_class flags not set though they should be.
•
Ensure that unlogged tables are copied correctly during CREATE DATABASE or ALTER DATABASE SET TABLESPACE (Pavan Deolasee, Andres Freund)
•
Fix DROP’s dependency searching to correctly handle the case where a table column is recursively visited before its table (Petr Jelinek, Tom Lane)
2099
Appendix E. Release Notes This case is only known to arise when an extension creates both a datatype and a table using that datatype. The faulty code might refuse a DROP EXTENSION unless CASCADE is specified, which should not be required. •
Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane) In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug.
•
Fix planning of SELECT FOR UPDATE when using a partial index on a child table (Kyotaro Horiguchi) In READ COMMITTED mode, SELECT FOR UPDATE must also recheck the partial index’s WHERE condition when rechecking a recently-updated row to see if it still satisfies the query’s WHERE condition. This requirement was missed if the index belonged to an inheritance child table, so that it was possible to incorrectly return rows that no longer satisfy the query condition.
•
Fix corner case wherein SELECT FOR UPDATE could return a row twice, and possibly miss returning other rows (Tom Lane) In READ COMMITTED mode, a SELECT FOR UPDATE that is scanning an inheritance tree could incorrectly return a row from a prior child table instead of the one it should return from a later child table.
•
Reject duplicate column names in the referenced-columns list of a FOREIGN KEY declaration (David Rowley) This restriction is per SQL standard. Previously we did not reject the case explicitly, but later on the code would fail with bizarre-looking errors.
•
Restore previous behavior of conversion of domains to JSON (Tom Lane) This change causes domains over numeric and boolean to be treated like their base types for purposes of conversion to JSON. It worked like that before 9.3.5 and 9.2.9, but was unintentionally changed while fixing a related problem.
•
Fix bugs in raising a numeric value to a large integral power (Tom Lane) The previous code could get a wrong answer, or consume excessive amounts of time and memory before realizing that the answer must overflow.
•
In numeric_recv(), truncate away any fractional digits that would be hidden according to the value’s dscale field (Tom Lane) A numeric value’s display scale (dscale) should never be less than the number of nonzero fractional digits; but apparently there’s at least one broken client application that transmits binary numeric values in which that’s true. This leads to strange behavior since the extra digits are taken into account by arithmetic operations even though they aren’t printed. The least risky fix seems to be to truncate away such “hidden” digits on receipt, so that the value is indeed what it prints as.
•
Fix incorrect search for shortest-first regular expression matches (Tom Lane) Matching would often fail when the number of allowed iterations is limited by a ? quantifier or a bound expression.
•
Reject out-of-range numeric timezone specifications (Tom Lane) Simple numeric timezone specifications exceeding +/- 168 hours (one week) would be accepted, but could then cause null-pointer dereference crashes in certain operations. There’s no use-case for such large UTC offsets, so reject them.
•
Fix bugs in tsquery @> tsquery operator (Heikki Linnakangas)
2100
Appendix E. Release Notes Two different terms would be considered to match if they had the same CRC. Also, if the second operand had more terms than the first, it would be assumed not to be contained in the first; which is wrong since it might contain duplicate terms. •
Improve ispell dictionary’s defenses against bad affix files (Tom Lane)
•
Allow more than 64K phrases in a thesaurus dictionary (David Boutin) The previous coding could crash on an oversize dictionary, so this was deemed a back-patchable bug fix rather than a feature addition.
•
Fix namespace handling in xpath() (Ali Akbar) Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation.
•
Ensure that whole-row variables expose nonempty column names to functions that pay attention to column names within composite arguments (Tom Lane) In some contexts, constructs like row_to_json(tab.*) may not produce the expected column names. This is fixed properly as of 9.4; in older branches, just ensure that we produce some nonempty name. (In some cases this will be the underlying table’s column name rather than the query-assigned alias that should theoretically be visible.)
•
Fix mishandling of system columns, particularly tableoid, in FDW queries (Etsuro Fujita)
•
Avoid doing indexed_column = ANY (array ) as an index qualifier if that leads to an inferior plan (Andrew Gierth) In some cases, = ANY conditions applied to non-first index columns would be done as index conditions even though it would be better to use them as simple filter conditions.
•
Fix planner problems with nested append relations, such as inherited tables within UNION ALL subqueries (Tom Lane)
•
Fail cleanly when a GiST index tuple doesn’t fit on a page, rather than going into infinite recursion (Andrew Gierth)
•
Exempt tables that have per-table cost_limit and/or cost_delay settings from autovacuum’s global cost balancing rules (Álvaro Herrera) The previous behavior resulted in basically ignoring these per-table settings, which was unintended. Now, a table having such settings will be vacuumed using those settings, independently of what is going on in other autovacuum workers. This may result in heavier total I/O load than before, so such settings should be re-examined for sanity.
•
Avoid wholesale autovacuuming when autovacuum is nominally off (Tom Lane) Even when autovacuum is nominally off, we will still launch autovacuum worker processes to vacuum tables that are at risk of XID wraparound. However, such a worker process then proceeded to vacuum all tables in the target database, if they met the usual thresholds for autovacuuming. This is at best pretty unexpected; at worst it delays response to the wraparound threat. Fix it so that if autovacuum is turned off, workers only do anti-wraparound vacuums and not any other work.
•
During crash recovery, ensure that unlogged relations are rewritten as empty and are synced to disk before recovery is considered complete (Abhijit Menon-Sen, Andres Freund) This prevents scenarios in which unlogged relations might contain garbage data following database crash recovery.
2101
Appendix E. Release Notes •
Fix race condition between hot standby queries and replaying a full-page image (Heikki Linnakangas) This mistake could result in transient errors in queries being executed in hot standby.
•
Fix several cases where recovery logic improperly ignored WAL records for COMMIT/ABORT PREPARED (Heikki Linnakangas) The most notable oversight was that recovery_target_xid could not be used to stop at a twophase commit.
•
Prevent latest WAL file from being archived a second time at completion of crash recovery (Fujii Masao)
•
Avoid creating unnecessary .ready marker files for timeline history files (Fujii Masao)
•
Fix possible null pointer dereference when an empty prepared statement is used and the log_statement setting is mod or ddl (Fujii Masao)
•
Change “pgstat wait timeout” warning message to be LOG level, and rephrase it to be more understandable (Tom Lane) This message was originally thought to be essentially a can’t-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads “using stale statistics instead of current ones because stats collector is not responding”.
•
Fix SPARC spinlock implementation to ensure correctness if the CPU is being run in a non-TSO coherency mode, as some non-Solaris kernels do (Andres Freund)
•
Warn if macOS’s setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch)
•
Fix processing of repeated dbname parameters in PQconnectdbParams() (Alex Shulgin) Unexpected behavior ensued if the first occurrence of dbname contained a connection string or URI to be expanded.
•
Ensure that libpq reports a suitable error message on unexpected socket EOF (Marko Tiikkaja, Tom Lane) Depending on kernel behavior, libpq might return an empty error string rather than something useful when the server unexpectedly closed the socket.
•
Clear any old error message during PQreset() (Heikki Linnakangas) If PQreset() is called repeatedly, and the connection cannot be re-established, error messages from the failed connection attempts kept accumulating in the PGconn’s error string.
•
Properly handle out-of-memory conditions while parsing connection options in libpq (Alex Shulgin, Heikki Linnakangas)
•
Fix array overrun in ecpg’s version of ParseDateTime() (Michael Paquier)
•
In initdb, give a clearer error message if a password file is specified but is empty (Mats Erik Andersson)
•
Fix psql’s \s command to work nicely with libedit, and add pager support (Stepan Rutz, Tom Lane) When using libedit rather than readline, \s printed the command history in a fairly unreadable encoded format, and on recent libedit versions might fail altogether. Fix that by printing the history ourselves rather than having the library do it. A pleasant side-effect is that the pager is used if appropriate.
2102
Appendix E. Release Notes This patch also fixes a bug that caused newline encoding to be applied inconsistently when saving the command history with libedit. Multiline history entries written by older psql versions will be read cleanly with this patch, but perhaps not vice versa, depending on the exact libedit versions involved. •
Improve consistency of parsing of psql’s special variables (Tom Lane) Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors.
•
Fix psql’s expanded-mode display to work consistently when using border = 3 and linestyle = ascii or unicode (Stephen Frost)
•
Improve performance of pg_dump when the database contains many instances of multiple dependency paths between the same two objects (Tom Lane)
•
Fix pg_dumpall to restore its ability to dump from pre-8.1 servers (Gilles Darold)
•
Fix possible deadlock during parallel restore of a schema-only dump (Robert Haas, Tom Lane)
•
Fix core dump in pg_dump --binary-upgrade on zero-column composite type (Rushabh Lathia)
•
Prevent WAL files created by pg_basebackup -x/-X from being archived again when the standby is promoted (Andres Freund)
•
Fix failure of contrib/auto_explain to print per-node timing information when doing EXPLAIN ANALYZE (Tom Lane)
•
Fix upgrade-from-unpackaged script for contrib/citext (Tom Lane)
•
Fix block number checking in contrib/pageinspect’s get_raw_page() (Tom Lane) The incorrect checking logic could prevent access to some pages in non-main relation forks.
•
Fix contrib/pgcrypto’s pgp_sym_decrypt() to not fail on messages whose length is 6 less than a power of 2 (Marko Tiikkaja)
•
Fix file descriptor leak in contrib/pg_test_fsync (Jeff Janes) This could cause failure to remove temporary files on Windows.
•
Handle unexpected query results, especially NULLs, safely in contrib/tablefunc’s connectby() (Michael Paquier) connectby() previously crashed if it encountered a NULL key value. It now prints that row but
doesn’t recurse further. •
Avoid a possible crash in contrib/xml2’s xslt_process() (Mark Simonetti) libxslt seems to have an undocumented dependency on the order in which resources are freed; reorder our calls to avoid a crash.
•
Mark some contrib I/O functions with correct volatility properties (Tom Lane) The previous over-conservative marking was immaterial in normal use, but could cause optimization problems or rejection of valid index expression definitions. Since the consequences are not large, we’ve just adjusted the function definitions in the extension modules’ scripts, without changing version numbers.
•
Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier)
2103
Appendix E. Release Notes These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues. •
Detect incompatible OpenLDAP versions during build (Noah Misch) With OpenLDAP versions 2.4.24 through 2.4.31, inclusive, PostgreSQL backends can crash at exit. Raise a warning during configure based on the compile-time OpenLDAP version number, and test the crashing scenario in the contrib/dblink regression test.
•
In non-MSVC Windows builds, ensure libpq.dll is installed with execute permissions (Noah Misch)
•
Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane) This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations.
•
Support time zone abbreviations that change UTC offset from time to time (Tom Lane) Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date.
•
Update time zone abbreviations lists (Tom Lane) Add CST (China Standard Time) to our lists. Remove references to ADT as “Arabia Daylight Time”, an abbreviation that’s been out of use since 2007; therefore, claiming there is a conflict with “Atlantic Daylight Time” doesn’t seem especially helpful. Fix entirely incorrect GMT offsets for CKT (Cook Islands), FJT, and FJST (Fiji); we didn’t even have them on the proper side of the date line.
•
Update time zone data files to tzdata release 2015a. The IANA timezone database has adopted abbreviations of the form Ax ST/Ax DT for all Australian time zones, reflecting what they believe to be current majority practice Down Under. These names do not conflict with usage elsewhere (other than ACST for Acre Summer Time, which has been in disuse since 1994). Accordingly, adopt these names into our “Default” timezone abbreviation set. The “Australia” abbreviation set now contains only CST, EAST, EST, SAST, SAT, and WST, all of which are thought to be mostly historical usage. Note that SAST has also been changed to be South Africa Standard Time in the “Default” abbreviation set. Also, add zone abbreviations SRET (Asia/Srednekolymsk) and XJT (Asia/Urumqi), and use WSST/WSDT for western Samoa. Also, there were DST law changes in Chile, Mexico, the Turks & Caicos Islands (America/Grand_Turk), and Fiji. There is a new zone Pacific/Bougainville for portions of Papua New Guinea. Also, numerous corrections for historical (pre-1970) time zone data.
E.35. Release 9.2.9 Release date: 2014-07-24
2104
Appendix E. Release Notes This release contains a variety of fixes from 9.2.8. For information about new features in the 9.2 major release, see Section E.44.
E.35.1. Migration to Version 9.2.9 A dump/restore is not required for those running 9.2.X. However, this release corrects an index corruption problem in some GiST indexes. See the first changelog entry below to find out whether your installation has been affected and what steps you should take if so. Also, if you are upgrading from a version earlier than 9.2.6, see Section E.38.
E.35.2. Changes •
Correctly initialize padding bytes in contrib/btree_gist indexes on bit columns (Heikki Linnakangas) This error could result in incorrect query results due to values that should compare equal not being seen as equal. Users with GiST indexes on bit or bit varying columns should REINDEX those indexes after installing this update.
•
Protect against torn pages when deleting GIN list pages (Heikki Linnakangas) This fix prevents possible index corruption if a system crash occurs while the page update is being written to disk.
•
Don’t clear the right-link of a GiST index page while replaying updates from WAL (Heikki Linnakangas) This error could lead to transiently wrong answers from GiST index scans performed in Hot Standby.
•
Fix corner-case infinite loop during insertion into an SP-GiST text index (Tom Lane)
•
Fix feedback status when hot_standby_feedback is turned off on-the-fly (Simon Riggs)
•
Fix
possibly-incorrect
cache
invalidation
during
nested
calls
to
ReceiveSharedInvalidMessages (Andres Freund) •
Fix planner’s mishandling of nested PlaceHolderVars generated in nested-nestloop plans (Tom Lane) This oversight could result in “variable not found in subplan target lists” errors, or in silently wrong query results.
•
Fix “could not find pathkey item to sort” planner failures with UNION ALL over subqueries reading from tables with inheritance children (Tom Lane)
•
Don’t assume a subquery’s output is unique if there’s a set-returning function in its targetlist (David Rowley) This oversight could lead to misoptimization of constructs like WHERE x IN (SELECT y, generate_series(1,10) FROM t GROUP BY y).
•
Improve planner to drop constant-NULL inputs of AND/OR when possible (Tom Lane) This change fixes some cases where the more aggressive parameter substitution done by 9.2 and later can lead to a worse plan than older versions produced.
2105
Appendix E. Release Notes •
Fix identification of input type category in to_json() and friends (Tom Lane) This is known to have led to inadequate quoting of money fields in the JSON result, and there may have been wrong results for other data types as well.
•
Fix failure to detoast fields in composite elements of structured types (Tom Lane) This corrects cases where TOAST pointers could be copied into other tables without being dereferenced. If the original data is later deleted, it would lead to errors like “missing chunk number 0 for toast value ...” when the now-dangling pointer is used.
•
Fix “record type has not been registered” failures with whole-row references to the output of Append plan nodes (Tom Lane)
•
Fix possible crash when invoking a user-defined function while rewinding a cursor (Tom Lane)
•
Fix query-lifespan memory leak while evaluating the arguments for a function in FROM (Tom Lane)
•
Fix session-lifespan memory leaks in regular-expression processing (Tom Lane, Arthur O’Dwyer, Greg Stark)
•
Fix data encoding error in hungarian.stop (Tom Lane)
•
Prevent foreign tables from being created with OIDS when default_with_oids is true (Etsuro Fujita)
•
Fix liveness checks for rows that were inserted in the current transaction and then deleted by a now-rolled-back subtransaction (Andres Freund) This could cause problems (at least spurious warnings, and at worst an infinite loop) if CREATE INDEX or CLUSTER were done later in the same transaction.
•
Clear pg_stat_activity.xact_start during PREPARE TRANSACTION (Andres Freund) After the PREPARE, the originating session is no longer in a transaction, so it should not continue to display a transaction start time.
•
Fix REASSIGN OWNED to not fail for text search objects (Álvaro Herrera)
•
Block signals during postmaster startup (Tom Lane) This ensures that the postmaster will properly clean up after itself if, for example, it receives SIGINT while still starting up.
•
Fix client host name lookup when processing pg_hba.conf entries that specify host names instead of IP addresses (Tom Lane) Ensure that reverse-DNS lookup failures are reported, instead of just silently not matching such entries. Also ensure that we make only one reverse-DNS lookup attempt per connection, not one per host name entry, which is what previously happened if the lookup attempts failed.
•
Allow the root user to use postgres -C variable and postgres --describe-config (MauMau) The prohibition on starting the server as root does not need to extend to these operations, and relaxing it prevents failure of pg_ctl in some scenarios.
•
Secure Unix-domain sockets of temporary postmasters started during make check (Noah Misch) Any local user able to access the socket file could connect as the server’s bootstrap superuser, then proceed to execute arbitrary code as the operating-system user running the test, as we previously noted in CVE-2014-0067. This change defends against that risk by placing the server’s socket in a temporary, mode 0700 subdirectory of /tmp. The hazard remains however on platforms where Unix sockets are not supported, notably Windows, because then the temporary postmaster must accept local TCP connections.
2106
Appendix E. Release Notes A useful side effect of this change is to simplify make check testing in builds that override DEFAULT_PGSOCKET_DIR. Popular non-default values like /var/run/postgresql are often not writable by the build user, requiring workarounds that will no longer be necessary. •
Fix tablespace creation WAL replay to work on Windows (MauMau)
•
Fix detection of socket creation failures on Windows (Bruce Momjian)
•
On Windows, allow new sessions to absorb values of PGC_BACKEND parameters (such as log_connections) from the configuration file (Amit Kapila) Previously, if such a parameter were changed in the file post-startup, the change would have no effect.
•
Properly quote executable path names on Windows (Nikhil Deshpande) This oversight could cause initdb and pg_upgrade to fail on Windows, if the installation path contained both spaces and @ signs.
•
Fix linking of libpython on macOS (Tom Lane) The method we previously used can fail with the Python library supplied by Xcode 5.0 and later.
•
Avoid buffer bloat in libpq when the server consistently sends data faster than the client can absorb it (Shin-ichi Morita, Tom Lane) libpq could be coerced into enlarging its input buffer until it runs out of memory (which would be reported misleadingly as “lost synchronization with server”). Under ordinary circumstances it’s quite far-fetched that data could be continuously transmitted more quickly than the recv() loop can absorb it, but this has been observed when the client is artificially slowed by scheduler constraints.
•
Ensure that LDAP lookup attempts in libpq time out as intended (Laurenz Albe)
•
Fix ecpg to do the right thing when an array of char * is the target for a FETCH statement returning more than one row, as well as some other array-handling fixes (Ashutosh Bapat)
•
Fix pg_restore’s processing of old-style large object comments (Tom Lane) A direct-to-database restore from an archive file generated by a pre-9.0 version of pg_dump would usually fail if the archive contained more than a few comments for large objects.
•
Fix pg_upgrade for cases where the new server creates a TOAST table but the old version did not (Bruce Momjian) This rare situation would manifest as “relation OID mismatch” errors.
•
Prevent contrib/auto_explain from changing the output of a user’s EXPLAIN (Tom Lane) If auto_explain is active, it could cause an EXPLAIN (ANALYZE, TIMING OFF) command to nonetheless print timing information.
•
Fix query-lifespan memory leak in contrib/dblink (MauMau, Joe Conway)
•
In contrib/pgcrypto functions, ensure sensitive information is cleared from stack variables before returning (Marko Kreen)
•
Prevent use of already-freed memory in contrib/pgstattuple’s pgstat_heap() (Noah Misch)
•
In contrib/uuid-ossp, cache the state of the OSSP UUID library across calls (Tom Lane) This improves the efficiency of UUID generation and reduces the amount of entropy drawn from /dev/urandom, on platforms that have that.
2107
Appendix E. Release Notes •
Update time zone data files to tzdata release 2014e for DST law changes in Crimea, Egypt, and Morocco.
E.36. Release 9.2.8 Release date: 2014-03-20 This release contains a variety of fixes from 9.2.7. For information about new features in the 9.2 major release, see Section E.44.
E.36.1. Migration to Version 9.2.8 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.6, see Section E.38.
E.36.2. Changes •
Restore GIN metapages unconditionally to avoid torn-page risk (Heikki Linnakangas) Although this oversight could theoretically result in a corrupted index, it is unlikely to have caused any problems in practice, since the active part of a GIN metapage is smaller than a standard 512byte disk sector.
•
Avoid race condition in checking transaction commit status during receipt of a NOTIFY message (Marko Tiikkaja) This prevents a scenario wherein a sufficiently fast client might respond to a notification before database updates made by the notifier have become visible to the recipient.
•
Allow regular-expression operators to be terminated early by query cancel requests (Tom Lane) This prevents scenarios wherein a pathological regular expression could lock up a server process uninterruptibly for a long time.
•
Remove incorrect code that tried to allow OVERLAPS with single-element row arguments (Joshua Yanovski) This code never worked correctly, and since the case is neither specified by the SQL standard nor documented, it seemed better to remove it than fix it.
•
Avoid getting more than AccessShareLock when de-parsing a rule or view (Dean Rasheed) This oversight resulted in pg_dump unexpectedly acquiring RowExclusiveLock locks on tables mentioned as the targets of INSERT/UPDATE/DELETE commands in rules. While usually harmless, that could interfere with concurrent transactions that tried to acquire, for example, ShareLock on those tables.
•
Improve performance of index endpoint probes during planning (Tom Lane) This change fixes a significant performance problem that occurred when there were many not-yetcommitted rows at the end of the index, which is a common situation for indexes on sequentiallyassigned values such as timestamps or sequence-generated identifiers.
•
Fix walsender’s failure to shut down cleanly when client is pg_receivexlog (Fujii Masao)
2108
Appendix E. Release Notes •
Check WAL level and hot standby parameters correctly when doing crash recovery that will be followed by archive recovery (Heikki Linnakangas)
•
Fix test to see if hot standby connections can be allowed immediately after a crash (Heikki Linnakangas)
•
Prevent interrupts while reporting non-ERROR messages (Tom Lane) This guards against rare server-process freezeups due to recursive entry to syslog(), and perhaps other related problems.
•
Fix memory leak in PL/Perl when returning a composite result, including multiple-OUT-parameter cases (Alex Hunsaker)
•
Fix tracking of psql script line numbers during \copy from out-of-line data (Kumar Rajeev Rastogi, Amit Khandekar) \copy ... from incremented the script file line number for each data line, even if the data was
not coming from the script file. This mistake resulted in wrong line numbers being reported for any errors occurring later in the same script file. •
Prevent intermittent “could not reserve shared memory region” failures on recent Windows versions (MauMau)
•
Update time zone data files to tzdata release 2014a for DST law changes in Fiji and Turkey, plus historical changes in Israel and Ukraine.
E.37. Release 9.2.7 Release date: 2014-02-20 This release contains a variety of fixes from 9.2.6. For information about new features in the 9.2 major release, see Section E.44.
E.37.1. Migration to Version 9.2.7 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.6, see Section E.38.
E.37.2. Changes •
Shore up GRANT ... WITH ADMIN OPTION restrictions (Noah Misch) Granting a role without ADMIN OPTION is supposed to prevent the grantee from adding or removing members from the granted role, but this restriction was easily bypassed by doing SET ROLE first. The security impact is mostly that a role member can revoke the access of others, contrary to the wishes of his grantor. Unapproved role member additions are a lesser concern, since an uncooperative role member could provide most of his rights to others anyway by creating views or SECURITY DEFINER functions. (CVE-2014-0060)
•
Prevent privilege escalation via manual calls to PL validator functions (Andres Freund) The primary role of PL validator functions is to be called implicitly during CREATE FUNCTION, but they are also normal SQL functions that a user can call explicitly. Calling a validator on a
2109
Appendix E. Release Notes function actually written in some other language was not checked for and could be exploited for privilege-escalation purposes. The fix involves adding a call to a privilege-checking function in each validator function. Non-core procedural languages will also need to make this change to their own validator functions, if any. (CVE-2014-0061) •
Avoid multiple name lookups during table and index DDL (Robert Haas, Andres Freund) If the name lookups come to different conclusions due to concurrent activity, we might perform some parts of the DDL on a different table than other parts. At least in the case of CREATE INDEX, this can be used to cause the permissions checks to be performed against a different table than the index creation, allowing for a privilege escalation attack. (CVE-2014-0062)
•
Prevent buffer overrun with long datetime strings (Noah Misch) The MAXDATELEN constant was too small for the longest possible value of type interval, allowing a buffer overrun in interval_out(). Although the datetime input functions were more careful about avoiding buffer overrun, the limit was short enough to cause them to reject some valid inputs, such as input containing a very long timezone name. The ecpg library contained these vulnerabilities along with some of its own. (CVE-2014-0063)
•
Prevent buffer overrun due to integer overflow in size calculations (Noah Misch, Heikki Linnakangas) Several functions, mostly type input functions, calculated an allocation size without checking for overflow. If overflow did occur, a too-small buffer would be allocated and then written past. (CVE2014-0064)
•
Prevent overruns of fixed-size buffers (Peter Eisentraut, Jozef Mlich) Use strlcpy() and related functions to provide a clear guarantee that fixed-size buffers are not overrun. Unlike the preceding items, it is unclear whether these cases really represent live issues, since in most cases there appear to be previous constraints on the size of the input string. Nonetheless it seems prudent to silence all Coverity warnings of this type. (CVE-2014-0065)
•
Avoid crashing if crypt() returns NULL (Honza Horak, Bruce Momjian) There are relatively few scenarios in which crypt() could return NULL, but contrib/chkpass would crash if it did. One practical case in which this could be an issue is if libc is configured to refuse to execute unapproved hashing algorithms (e.g., “FIPS mode”). (CVE-2014-0066)
•
Document risks of make check in the regression testing instructions (Noah Misch, Tom Lane) Since the temporary server started by make check uses “trust” authentication, another user on the same machine could connect to it as database superuser, and then potentially exploit the privileges of the operating-system user who started the tests. A future release will probably incorporate changes in the testing procedure to prevent this risk, but some public discussion is needed first. So for the moment, just warn people against using make check when there are untrusted users on the same machine. (CVE-2014-0067)
•
Fix possible mis-replay of WAL records when some segments of a relation aren’t full size (Greg Stark, Tom Lane) The WAL update could be applied to the wrong page, potentially many pages past where it should have been. Aside from corrupting data, this error has been observed to result in significant “bloat” of standby servers compared to their masters, due to updates being applied far beyond where the end-of-file should have been. This failure mode does not appear to be a significant risk during crash recovery, only when initially synchronizing a standby created from a base backup taken from a quickly-changing master.
•
Fix bug in determining when recovery has reached consistency (Tomonari Katsumata, Heikki Linnakangas)
2110
Appendix E. Release Notes In some cases WAL replay would mistakenly conclude that the database was already consistent at the start of replay, thus possibly allowing hot-standby queries before the database was really consistent. Other symptoms such as “PANIC: WAL contains references to invalid pages” were also possible. •
Fix improper locking of btree index pages while replaying a VACUUM operation in hot-standby mode (Andres Freund, Heikki Linnakangas, Tom Lane) This error could result in “PANIC: WAL contains references to invalid pages” failures.
•
Ensure that insertions into non-leaf GIN index pages write a full-page WAL record when appropriate (Heikki Linnakangas) The previous coding risked index corruption in the event of a partial-page write during a system crash.
•
When pause_at_recovery_target and recovery_target_inclusive are both set, ensure the target record is applied before pausing, not after (Heikki Linnakangas)
•
Fix race conditions during server process exit (Robert Haas) Ensure that signal handlers don’t attempt to use the process’s MyProc pointer after it’s no longer valid.
•
Fix race conditions in walsender shutdown logic and walreceiver SIGHUP signal handler (Tom Lane)
•
Fix unsafe references to errno within error reporting logic (Christian Kruse) This would typically lead to odd behaviors such as missing or inappropriate HINT fields.
•
Fix possible crashes from using ereport() too early during server startup (Tom Lane) The principal case we’ve seen in the field is a crash if the server is started in a directory it doesn’t have permission to read.
•
Clear retry flags properly in OpenSSL socket write function (Alexander Kukushkin) This omission could result in a server lockup after unexpected loss of an SSL-encrypted connection.
•
Fix length checking for Unicode identifiers (U&"..." syntax) containing escapes (Tom Lane) A spurious truncation warning would be printed for such identifiers if the escaped form of the identifier was too long, but the identifier actually didn’t need truncation after de-escaping.
•
Allow keywords that are type names to be used in lists of roles (Stephen Frost) A previous patch allowed such keywords to be used without quoting in places such as role identifiers; but it missed cases where a list of role identifiers was permitted, such as DROP ROLE.
•
Fix parser crash for EXISTS(SELECT * FROM zero_column_table) (Tom Lane)
•
Fix possible crash due to invalid plan for nested sub-selects, such as WHERE (... x IN (SELECT ...) ...) IN (SELECT ...) (Tom Lane)
•
Fix UPDATE/DELETE of an inherited target table that has UNION ALL subqueries (Tom Lane) Without this fix, UNION ALL subqueries aren’t correctly inserted into the update plans for inheritance child tables after the first one, typically resulting in no update happening for those child table(s).
•
Ensure that ANALYZE creates statistics for a table column even when all the values in it are “too wide” (Tom Lane)
2111
Appendix E. Release Notes ANALYZE intentionally omits very wide values from its histogram and most-common-values cal-
culations, but it neglected to do something sane in the case that all the sampled entries are too wide. •
In ALTER TABLE ... SET TABLESPACE, allow the database’s default tablespace to be used without a permissions check (Stephen Frost) CREATE TABLE has always allowed such usage, but ALTER TABLE didn’t get the memo.
•
Fix “cannot accept a set” error when some arms of a CASE return a set and others don’t (Tom Lane)
•
Properly distinguish numbers from non-numbers when generating JSON output (Andrew Dunstan)
•
Fix checks for all-zero client addresses in pgstat functions (Kevin Grittner)
•
Fix possible misclassification of multibyte characters by the text search parser (Tom Lane) Non-ASCII characters could be misclassified when using C locale with a multibyte encoding. On Cygwin, non-C locales could fail as well.
•
Fix possible misbehavior in plainto_tsquery() (Heikki Linnakangas) Use memmove() not memcpy() for copying overlapping memory regions. There have been no field reports of this actually causing trouble, but it’s certainly risky.
•
Fix placement of permissions checks in pg_start_backup() and pg_stop_backup() (Andres Freund, Magnus Hagander) The previous coding might attempt to do catalog access when it shouldn’t.
•
Accept SHIFT_JIS as an encoding name for locale checking purposes (Tatsuo Ishii)
•
Fix *-qualification of named parameters in SQL-language functions (Tom Lane) Given a composite-type parameter named foo, $1.* worked fine, but foo.* not so much.
•
Fix misbehavior of PQhost() on Windows (Fujii Masao) It should return localhost if no host has been specified.
•
Improve error handling in libpq and psql for failures during COPY TO STDOUT/FROM STDIN (Tom Lane) In particular this fixes an infinite loop that could occur in 9.2 and up if the server connection was lost during COPY FROM STDIN. Variants of that scenario might be possible in older versions, or with other client applications.
•
Fix incorrect translation handling in some psql \d commands (Peter Eisentraut, Tom Lane)
•
Ensure pg_basebackup’s background process is killed when exiting its foreground process (Magnus Hagander)
•
Fix possible incorrect printing of filenames in pg_basebackup’s verbose mode (Magnus Hagander)
•
Avoid including tablespaces inside PGDATA twice in base backups (Dimitri Fontaine, Magnus Hagander)
•
Fix misaligned descriptors in ecpg (MauMau)
•
In ecpg, handle lack of a hostname in the connection parameters properly (Michael Meskes)
•
Fix performance regression in contrib/dblink connection startup (Joe Conway) Avoid an unnecessary round trip when client and server encodings match.
•
In contrib/isn, fix incorrect calculation of the check digit for ISMN values (Fabien Coelho)
•
Fix contrib/pg_stat_statement’s handling of CURRENT_DATE and related constructs (Kyotaro Horiguchi)
2112
Appendix E. Release Notes •
Ensure client-code-only installation procedure works as documented (Peter Eisentraut)
•
In Mingw and Cygwin builds, install the libpq DLL in the bin directory (Andrew Dunstan) This duplicates what the MSVC build has long done. It should fix problems with programs like psql failing to start because they can’t find the DLL.
•
Avoid using the deprecated dllwrap tool in Cygwin builds (Marco Atzeri)
•
Don’t generate plain-text HISTORY and src/test/regress/README files anymore (Tom Lane) These text files duplicated the main HTML and PDF documentation formats. The trouble involved in maintaining them greatly outweighs the likely audience for plain-text format. Distribution tarballs will still contain files by these names, but they’ll just be stubs directing the reader to consult the main documentation. The plain-text INSTALL file will still be maintained, as there is arguably a use-case for that.
•
Update time zone data files to tzdata release 2013i for DST law changes in Jordan and historical changes in Cuba. In addition, the zones Asia/Riyadh87, Asia/Riyadh88, and Asia/Riyadh89 have been removed, as they are no longer maintained by IANA, and never represented actual civil timekeeping practice.
E.38. Release 9.2.6 Release date: 2013-12-05 This release contains a variety of fixes from 9.2.5. For information about new features in the 9.2 major release, see Section E.44.
E.38.1. Migration to Version 9.2.6 A dump/restore is not required for those running 9.2.X. However, this release corrects a number of potential data corruption issues. See the first two changelog entries below to find out whether your installation has been affected and what steps you can take if so. Also, if you are upgrading from a version earlier than 9.2.4, see Section E.40.
E.38.2. Changes •
Fix VACUUM’s tests to see whether it can update relfrozenxid (Andres Freund) In some cases VACUUM (either manual or autovacuum) could incorrectly advance a table’s relfrozenxid value, allowing tuples to escape freezing, causing those rows to become invisible once 2^31 transactions have elapsed. The probability of data loss is fairly low since multiple incorrect advancements would need to happen before actual loss occurs, but it’s not zero. In 9.2.0 and later, the probability of loss is higher, and it’s also possible to get “could not access status of transaction” errors as a consequence of this bug. Users upgrading from releases 9.0.4 or 8.4.8 or earlier are not affected, but all later versions contain the bug. The issue can be ameliorated by, after upgrading, vacuuming all tables in all databases while having vacuum_freeze_table_age set to zero. This will fix any latent corruption but will not be able to
2113
Appendix E. Release Notes fix all pre-existing data errors. However, an installation can be presumed safe after performing this vacuuming if it has executed fewer than 2^31 update transactions in its lifetime (check this with SELECT txid_current() < 2^31). •
Fix initialization of pg_clog and pg_subtrans during hot standby startup (Andres Freund, Heikki Linnakangas) This bug can cause data loss on standby servers at the moment they start to accept hot-standby queries, by marking committed transactions as uncommitted. The likelihood of such corruption is small unless, at the time of standby startup, the primary server has executed many updating transactions since its last checkpoint. Symptoms include missing rows, rows that should have been deleted being still visible, and obsolete versions of updated rows being still visible alongside their newer versions. This bug was introduced in versions 9.3.0, 9.2.5, 9.1.10, and 9.0.14. Standby servers that have only been running earlier releases are not at risk. It’s recommended that standby servers that have ever run any of the buggy releases be re-cloned from the primary (e.g., with a new base backup) after upgrading.
•
Fix dangling-pointer problem in fast-path locking (Tom Lane) This could lead to corruption of the lock data structures in shared memory, causing “lock already held” and other odd errors.
•
Truncate pg_multixact contents during WAL replay (Andres Freund) This avoids ever-increasing disk space consumption in standby servers.
•
Ensure an anti-wraparound VACUUM counts a page as scanned when it’s only verified that no tuples need freezing (Sergey Burladyan, Jeff Janes) This bug could result in failing to advance relfrozenxid, so that the table would still be thought to need another anti-wraparound vacuum. In the worst case the database might even shut down to prevent wraparound.
•
Fix race condition in GIN index posting tree page deletion (Heikki Linnakangas) This could lead to transient wrong answers or query failures.
•
Fix “unexpected spgdoinsert() failure” error during SP-GiST index creation (Teodor Sigaev)
•
Avoid flattening a subquery whose SELECT list contains a volatile function wrapped inside a subSELECT (Tom Lane) This avoids unexpected results due to extra evaluations of the volatile function.
•
Fix planner’s processing of non-simple-variable subquery outputs nested within outer joins (Tom Lane) This error could lead to incorrect plans for queries involving multiple levels of subqueries within JOIN syntax.
•
Fix incorrect planning in cases where the same non-strict expression appears in multiple WHERE and outer JOIN equality clauses (Tom Lane)
•
Fix planner crash with whole-row reference to a subquery (Tom Lane)
•
Fix incorrect generation of optimized MIN()/MAX() plans for inheritance trees (Tom Lane) The planner could fail in cases where the MIN()/MAX() argument was an expression rather than a simple variable.
•
Fix premature deletion of temporary files (Andres Freund)
•
Prevent intra-transaction memory leak when printing range values (Tom Lane)
2114
Appendix E. Release Notes This fix actually cures transient memory leaks in any datatype output function, but range types are the only ones known to have had a significant problem. •
Prevent incorrect display of dropped columns in NOT NULL and CHECK constraint violation messages (Michael Paquier and Tom Lane)
•
Allow default arguments and named-argument notation for window functions (Tom Lane) Previously, these cases were likely to crash.
•
Fix possible read past end of memory in rule printing (Peter Eisentraut)
•
Fix array slicing of int2vector and oidvector values (Tom Lane) Expressions of this kind are now implicitly promoted to regular int2 or oid arrays.
•
Fix incorrect behaviors when using a SQL-standard, simple GMT offset timezone (Tom Lane) In some cases, the system would use the simple GMT offset value when it should have used the regular timezone setting that had prevailed before the simple offset was selected. This change also causes the timeofday function to honor the simple GMT offset zone.
•
Prevent possible misbehavior when logging translations of Windows error codes (Tom Lane)
•
Properly quote generated command lines in pg_ctl (Naoya Anzai and Tom Lane) This fix applies only to Windows.
•
Fix pg_dumpall to work when a source database sets default_transaction_read_only via ALTER DATABASE SET (Kevin Grittner) Previously, the generated script would fail during restore.
•
Make ecpg search for quoted cursor names case-sensitively (Zoltán Böszörményi)
•
Fix ecpg’s processing of lists of variables declared varchar (Zoltán Böszörményi)
•
Make contrib/lo defend against incorrect trigger definitions (Marc Cousin)
•
Update time zone data files to tzdata release 2013h for DST law changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco, and Palestine. Also, new timezone abbreviations WIB, WIT, WITA for Indonesia.
E.39. Release 9.2.5 Release date: 2013-10-10 This release contains a variety of fixes from 9.2.4. For information about new features in the 9.2 major release, see Section E.44.
E.39.1. Migration to Version 9.2.5 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.4, see Section E.40.
2115
Appendix E. Release Notes
E.39.2. Changes •
Prevent corruption of multi-byte characters when attempting to case-fold identifiers (Andrew Dunstan) PostgreSQL case-folds non-ASCII characters only when using a single-byte server encoding.
•
Fix memory leak when creating B-tree indexes on range columns (Heikki Linnakangas)
•
Fix checkpoint memory leak in background writer when wal_level = hot_standby (Naoya Anzai)
•
Fix memory leak caused by lo_open() failure (Heikki Linnakangas)
•
Fix memory overcommit bug when work_mem is using more than 24GB of memory (Stephen Frost)
•
Serializable snapshot fixes (Kevin Grittner, Heikki Linnakangas)
•
Fix deadlock bug in libpq when using SSL (Stephen Frost)
•
Fix possible SSL state corruption in threaded libpq applications (Nick Phillips, Stephen Frost)
•
Improve estimate of planner cost when choosing between generic and custom plans (Tom Lane) This change will favor generic plans when planning cost is high.
•
Properly compute row estimates for boolean columns containing many NULL values (Andrew Gierth) Previously tests like col IS NOT TRUE and col IS NOT FALSE did not properly factor in NULL values when estimating plan costs.
•
Fix accounting for qualifier evaluation costs in UNION ALL and inheritance queries (Tom Lane) This fixes cases where suboptimal query plans could be chosen if some WHERE clauses are expensive to calculate.
•
Prevent pushing down WHERE clauses into unsafe UNION/INTERSECT subqueries (Tom Lane) Subqueries of a UNION or INTERSECT that contain set-returning functions or volatile functions in their SELECT lists could be improperly optimized, leading to run-time errors or incorrect query results.
•
Fix rare case of “failed to locate grouping columns” planner failure (Tom Lane)
•
Fix pg_dump of foreign tables with dropped columns (Andrew Dunstan) Previously such cases could cause a pg_upgrade error.
•
Reorder pg_dump processing of extension-related rules and event triggers (Joe Conway)
•
Force dumping of extension tables if specified by pg_dump -t or -n (Joe Conway)
•
Improve view dumping code’s handling of dropped columns in referenced tables (Tom Lane)
•
Fix pg_restore -l with the directory archive to display the correct format name (Fujii Masao)
•
Properly record index comments created using UNIQUE and PRIMARY KEY syntax (Andres Freund) This fixes a parallel pg_restore failure.
•
Cause pg_basebackup -x with an empty xlog directory to throw an error rather than crashing (Magnus Hagander, Haruka Takatsuka)
•
Properly guarantee transmission of WAL files before clean switchover (Fujii Masao) Previously, the streaming replication connection might close before all WAL files had been replayed on the standby.
2116
Appendix E. Release Notes •
Fix WAL segment timeline handling during recovery (Mitsumasa Kondo, Heikki Linnakangas) WAL file recycling during standby recovery could lead to premature recovery completion, resulting in data loss.
•
Prevent errors in WAL replay due to references to uninitialized empty pages (Andres Freund)
•
Fix REINDEX TABLE and REINDEX DATABASE to properly revalidate constraints and mark invalidated indexes as valid (Noah Misch) REINDEX INDEX has always worked properly.
•
Avoid deadlocks during insertion into SP-GiST indexes (Teodor Sigaev)
•
Fix possible deadlock during concurrent CREATE INDEX CONCURRENTLY operations (Tom Lane)
•
Fix GiST index lookup crash (Tom Lane)
•
Fix regexp_matches() handling of zero-length matches (Jeevan Chalke) Previously, zero-length matches like ’^’ could return too many matches.
•
Fix crash for overly-complex regular expressions (Heikki Linnakangas)
•
Fix regular expression match failures for back references combined with non-greedy quantifiers (Jeevan Chalke)
•
Prevent CREATE FUNCTION from checking SET variables unless function body checking is enabled (Tom Lane)
•
Allow ALTER DEFAULT PRIVILEGES to operate on schemas without requiring CREATE permission (Tom Lane)
•
Loosen restriction on keywords used in queries (Tom Lane) Specifically, lessen keyword restrictions for role names, language names, EXPLAIN and COPY options, and SET values. This allows COPY ... (FORMAT BINARY) to work as expected; previously BINARY needed to be quoted.
•
Print proper line number during COPY failure (Heikki Linnakangas)
•
Fix pgp_pub_decrypt() so it works for secret keys with passwords (Marko Kreen)
•
Make pg_upgrade use pg_dump --quote-all-identifiers to avoid problems with keyword changes between releases (Tom Lane)
•
Remove rare inaccurate warning during vacuum of index-less tables (Heikki Linnakangas)
•
Ensure that VACUUM ANALYZE still runs the ANALYZE phase if its attempt to truncate the file is cancelled due to lock conflicts (Kevin Grittner)
•
Avoid possible failure when performing transaction control commands (e.g ROLLBACK) in prepared queries (Tom Lane)
•
Ensure that floating-point data input accepts standard spellings of “infinity” on all platforms (Tom Lane) The C99 standard says that allowable spellings are inf, +inf, -inf, infinity, +infinity, and -infinity. Make sure we recognize these even if the platform’s strtod function doesn’t.
•
Avoid unnecessary reporting when track_activities is off (Tom Lane)
•
Expand ability to compare rows to records and arrays (Rafal Rzepecki, Tom Lane)
•
Prevent crash when psql’s PSQLRC variable contains a tilde (Bruce Momjian)
•
Add spinlock support for ARM64 (Mark Salter)
2117
Appendix E. Release Notes •
Update time zone data files to tzdata release 2013d for DST law changes in Israel, Morocco, Palestine, and Paraguay. Also, historical zone data corrections for Macquarie Island.
E.40. Release 9.2.4 Release date: 2013-04-04 This release contains a variety of fixes from 9.2.3. For information about new features in the 9.2 major release, see Section E.44.
E.40.1. Migration to Version 9.2.4 A dump/restore is not required for those running 9.2.X. However, this release corrects several errors in management of GiST indexes. After installing this update, it is advisable to REINDEX any GiST indexes that meet one or more of the conditions described below. Also, if you are upgrading from a version earlier than 9.2.2, see Section E.42.
E.40.2. Changes •
Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi) A connection request containing a database name that begins with “-” could be crafted to damage or destroy files within the server’s data directory, even if the request is eventually rejected. (CVE2013-1899)
•
Reset OpenSSL randomness state in each postmaster child process (Marko Kreen) This avoids a scenario wherein random numbers generated by contrib/pgcrypto functions might be relatively easy for another database user to guess. The risk is only significant when the postmaster is configured with ssl = on but most connections don’t use SSL encryption. (CVE2013-1900)
•
Make REPLICATION privilege checks test current user not authenticated user (Noah Misch) An unprivileged database user could exploit this mistake to call pg_start_backup() or pg_stop_backup(), thus possibly interfering with creation of routine backups. (CVE-2013-1901)
•
Fix GiST indexes to not use “fuzzy” geometric comparisons when it’s not appropriate to do so (Alexander Korotkov) The core geometric types perform comparisons using “fuzzy” equality, but gist_box_same must do exact comparisons, else GiST indexes using it might become inconsistent. After installing this update, users should REINDEX any GiST indexes on box, polygon, circle, or point columns, since all of these use gist_box_same.
•
Fix erroneous range-union and penalty logic in GiST indexes that use contrib/btree_gist for variable-width data types, that is text, bytea, bit, and numeric columns (Tom Lane)
2118
Appendix E. Release Notes These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in useless index bloat. Users are advised to REINDEX such indexes after installing this update. •
Fix bugs in GiST page splitting code for multi-column indexes (Tom Lane) These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in indexes that are unnecessarily inefficient to search. Users are advised to REINDEX multi-column GiST indexes after installing this update.
•
Fix gist_point_consistent to handle fuzziness consistently (Alexander Korotkov) Index scans on GiST indexes on point columns would sometimes yield results different from a sequential scan, because gist_point_consistent disagreed with the underlying operator code about whether to do comparisons exactly or fuzzily.
•
Fix buffer leak in WAL replay (Heikki Linnakangas) This bug could result in “incorrect local pin count” errors during replay, making recovery impossible.
•
Ensure we do crash recovery before entering archive recovery, if the database was not stopped cleanly and a recovery.conf file is present (Heikki Linnakangas, Kyotaro Horiguchi, Mitsumasa Kondo) This is needed to ensure that the database is consistent in certain scenarios, such as initializing a standby server with a filesystem snapshot from a running server.
Fix race condition in DELETE RETURNING (Tom Lane) Under the right circumstances, DELETE RETURNING could attempt to fetch data from a shared buffer that the current process no longer has any pin on. If some other process changed the buffer meanwhile, this would lead to garbage RETURNING output, or even a crash.
•
Fix infinite-loop risk in regular expression compilation (Tom Lane, Don Porter)
•
Fix potential null-pointer dereference in regular expression compilation (Tom Lane)
•
Fix to_char() to use ASCII-only case-folding rules where appropriate (Tom Lane) This fixes misbehavior of some template patterns that should be locale-independent, but mishandled “I” and “i” in Turkish locales.
•
Fix unwanted rejection of timestamp 1999-12-31 24:00:00 (Tom Lane)
•
Fix SQL-language functions to be safely usable as support functions for range types (Tom Lane)
•
Fix logic error when a single transaction does UNLISTEN then LISTEN (Tom Lane) The session wound up not listening for notify events at all, though it surely should listen in this case.
•
Fix possible planner crash after columns have been added to a view that’s depended on by another view (Tom Lane)
•
Fix performance issue in EXPLAIN (ANALYZE, TIMING OFF) (Pavel Stehule)
•
Remove useless “picksplit doesn’t support secondary split” log messages (Josh Hansen, Tom Lane) This message seems to have been added in expectation of code that was never written, and probably never will be, since GiST’s default handling of secondary splits is actually pretty good. So stop nagging end users about it.
2119
Appendix E. Release Notes •
Remove vestigial secondary-split support in gist_box_picksplit() (Tom Lane) Not only was this implementation of secondary-split not better than the default implementation, it’s actually worse. So remove it and let the default code path handle the case.
•
Fix possible failure to send a session’s last few transaction commit/abort counts to the statistics collector (Tom Lane)
•
Eliminate memory leaks in PL/Perl’s spi_prepare() function (Alex Hunsaker, Tom Lane)
Avoid crash in pg_dump when an incorrect connection string is given (Heikki Linnakangas)
•
Ignore invalid indexes in pg_dump and pg_upgrade (Michael Paquier, Bruce Momjian) Dumping invalid indexes can cause problems at restore time, for example if the reason the index creation failed was because it tried to enforce a uniqueness condition not satisfied by the table’s data. Also, if the index creation is in fact still in progress, it seems reasonable to consider it to be an uncommitted DDL change, which pg_dump wouldn’t be expected to dump anyway. pg_upgrade now also skips invalid indexes rather than failing.
•
In pg_basebackup, include only the current server version’s subdirectory when backing up a tablespace (Heikki Linnakangas)
•
Add a server version check in pg_basebackup and pg_receivexlog, so they fail cleanly with version combinations that won’t work (Heikki Linnakangas)
•
Fix contrib/dblink to handle inconsistent settings of DateStyle or IntervalStyle safely (Daniel Farina, Tom Lane) Previously, if the remote server had different settings of these parameters, ambiguous dates might be read incorrectly. This fix ensures that datetime and interval columns fetched by a dblink query will be interpreted correctly. Note however that inconsistent settings are still risky, since literal values appearing in SQL commands sent to the remote server might be interpreted differently than they would be locally.
•
Fix contrib/pg_trgm’s similarity() function to return zero for trigram-less strings (Tom Lane) Previously it returned NaN due to internal division by zero.
•
Enable building PostgreSQL with Microsoft Visual Studio 2012 (Brar Piening, Noah Misch)
•
Update time zone data files to tzdata release 2013b for DST law changes in Chile, Haiti, Morocco, Paraguay, and some Russian areas. Also, historical zone data corrections for numerous places. Also, update the time zone abbreviation files for recent changes in Russia and elsewhere: CHOT, GET, IRKT, KGT, KRAT, MAGT, MAWT, MSK, NOVT, OMST, TKT, VLAT, WST, YAKT, YEKT now follow their current meanings, and VOLT (Europe/Volgograd) and MIST (Antarctica/Macquarie) are added to the default abbreviations list.
E.41. Release 9.2.3 Release date: 2013-02-07 This release contains a variety of fixes from 9.2.2. For information about new features in the 9.2 major release, see Section E.44.
2120
Appendix E. Release Notes
E.41.1. Migration to Version 9.2.3 A dump/restore is not required for those running 9.2.X. However, if you are upgrading from a version earlier than 9.2.2, see Section E.42.
E.41.2. Changes •
Prevent execution of enum_recv from SQL (Tom Lane) The function was misdeclared, allowing a simple SQL command to crash the server. In principle an attacker might be able to use it to examine the contents of server memory. Our thanks to Sumit Soni (via Secunia SVCRP) for reporting this issue. (CVE-2013-0255)
•
Fix multiple problems in detection of when a consistent database state has been reached during WAL replay (Fujii Masao, Heikki Linnakangas, Simon Riggs, Andres Freund)
•
Fix detection of end-of-backup point when no actual redo work is required (Heikki Linnakangas) This mistake could result in incorrect “WAL ends before end of online backup” errors.
•
Update minimum recovery point when truncating a relation file (Heikki Linnakangas) Once data has been discarded, it’s no longer safe to stop recovery at an earlier point in the timeline.
•
Fix recycling of WAL segments after changing recovery target timeline (Heikki Linnakangas)
•
Properly restore timeline history files from archive on cascading standby servers (Heikki Linnakangas)
•
Fix lock conflict detection on hot-standby servers (Andres Freund, Robert Haas)
•
Fix missing cancellations in hot standby mode (Noah Misch, Simon Riggs) The need to cancel conflicting hot-standby queries would sometimes be missed, allowing those queries to see inconsistent data.
•
Prevent recovery pause feature from pausing before users can connect (Tom Lane)
•
Fix SQL grammar to allow subscripting or field selection from a sub-SELECT result (Tom Lane)
•
Fix performance problems with autovacuum truncation in busy workloads (Jan Wieck) Truncation of empty pages at the end of a table requires exclusive lock, but autovacuum was coded to fail (and release the table lock) when there are conflicting lock requests. Under load, it is easily possible that truncation would never occur, resulting in table bloat. Fix by performing a partial truncation, releasing the lock, then attempting to re-acquire the lock and continue. This fix also greatly reduces the average time before autovacuum releases the lock after a conflicting request arrives.
•
Improve performance of SPI_execute and related functions, thereby improving PL/pgSQL’s EXECUTE (Heikki Linnakangas, Tom Lane) Remove some data-copying overhead that was added in 9.2 as a consequence of revisions in the plan caching mechanism. This eliminates a performance regression compared to 9.1, and also saves memory, especially when the query string to be executed contains many SQL statements. A side benefit is that multi-statement query strings are now processed fully serially, that is we complete execution of earlier statements before running parse analysis and planning on the following ones. This eliminates a long-standing issue, in that DDL that should affect the behavior of a later statement will now behave as expected.
2121
Appendix E. Release Notes •
Restore pre-9.2 cost estimates for index usage (Tom Lane) An ill-considered change of a fudge factor led to undesirably high cost estimates for use of very large indexes.
•
Fix intermittent crash in DROP INDEX CONCURRENTLY (Tom Lane)
•
Fix potential corruption of shared-memory lock table during CREATE/DROP INDEX CONCURRENTLY (Tom Lane)
•
Fix COPY’s multiple-tuple-insertion code for the case of a tuple larger than page size minus fillfactor (Heikki Linnakangas) The previous coding could get into an infinite loop.
•
Protect against race conditions when scanning pg_tablespace (Stephen Frost, Tom Lane) CREATE DATABASE and DROP DATABASE could misbehave if there were concurrent updates of pg_tablespace entries.
•
Prevent DROP OWNED from trying to drop whole databases or tablespaces (Álvaro Herrera) For safety, ownership of these objects must be reassigned, not dropped.
•
Fix error in vacuum_freeze_table_age implementation (Andres Freund) In installations that have existed for more than vacuum_freeze_min_age transactions, this mistake prevented autovacuum from using partial-table scans, so that a full-table scan would always happen instead.
•
Prevent misbehavior when a RowExpr or XmlExpr is parse-analyzed twice (Andres Freund, Tom Lane) This mistake could be user-visible in contexts such as CREATE TABLE LIKE INCLUDING INDEXES.
•
Improve defenses against integer overflow in hashtable sizing calculations (Jeff Davis)
•
Fix some bugs associated with privileges on datatypes (Tom Lane) There were some issues with default privileges for types, and pg_dump failed to dump such privileges at all.
•
Fix failure to ignore leftover temporary tables after a server crash (Tom Lane)
•
Fix failure to rotate postmaster log files for size reasons on Windows (Jeff Janes, Heikki Linnakangas)
•
Reject out-of-range dates in to_date() (Hitoshi Harada)
•
Fix pg_extension_config_dump() to handle extension-update cases properly (Tom Lane) This function will now replace any existing entry for the target table, making it usable in extension update scripts.
•
Fix PL/pgSQL’s reporting of plan-time errors in possibly-simple expressions (Tom Lane) The previous coding resulted in sometimes omitting the first line in the CONTEXT traceback for the error.
•
Fix PL/Python’s handling of functions used as triggers on multiple tables (Andres Freund)
•
Ensure that non-ASCII prompt strings are translated to the correct code page on Windows (Alexander Law, Noah Misch) This bug affected psql and some other client programs.
•
Fix possible crash in psql’s \? command when not connected to a database (Meng Qingzhong)
2122
Appendix E. Release Notes •
Fix possible error if a relation file is removed while pg_basebackup is running (Heikki Linnakangas)
•
Tolerate timeline switches while pg_basebackup -X fetch is backing up a standby server (Heikki Linnakangas)
•
Make pg_dump exclude data of unlogged tables when running on a hot-standby server (Magnus Hagander) This would fail anyway because the data is not available on the standby server, so it seems most convenient to assume --no-unlogged-table-data automatically.
•
Fix pg_upgrade to deal with invalid indexes safely (Bruce Momjian)
•
Fix pg_upgrade’s -O/-o options (Marti Raudsepp)
•
Fix one-byte buffer overrun in libpq’s PQprintTuples (Xi Wang) This ancient function is not used anywhere by PostgreSQL itself, but it might still be used by some client code.
•
Make ecpglib use translated messages properly (Chen Huajun)
•
Properly install ecpg_compat and pgtypes libraries on MSVC (Jiang Guiqing)
•
Include our version of isinf() in libecpg if it’s not provided by the system (Jiang Guiqing)
•
Rearrange configure’s tests for supplied functions so it is not fooled by bogus exports from libedit/libreadline (Christoph Berg)
•
Ensure Windows build number increases over time (Magnus Hagander)
•
Make pgxs build executables with the right .exe suffix when cross-compiling for Windows (Zoltan Boszormenyi)
•
Add new timezone abbreviation FET (Tom Lane) This is now used in some eastern-European time zones.
E.42. Release 9.2.2 Release date: 2012-12-06 This release contains a variety of fixes from 9.2.1. For information about new features in the 9.2 major release, see Section E.44.
E.42.1. Migration to Version 9.2.2 A dump/restore is not required for those running 9.2.X. However, you may need to perform REINDEX operations to correct problems in concurrently-built indexes, as described in the first changelog item below. Also, if you are upgrading from version 9.2.0, see Section E.43.
2123
Appendix E. Release Notes
E.42.2. Changes •
Fix multiple bugs associated with CREATE/DROP INDEX CONCURRENTLY (Andres Freund, Tom Lane, Simon Riggs, Pavan Deolasee) An error introduced while adding DROP INDEX CONCURRENTLY allowed incorrect indexing decisions to be made during the initial phase of CREATE INDEX CONCURRENTLY; so that indexes built by that command could be corrupt. It is recommended that indexes built in 9.2.X with CREATE INDEX CONCURRENTLY be rebuilt after applying this update. In addition, fix CREATE/DROP INDEX CONCURRENTLY to use in-place updates when changing the state of an index’s pg_index row. This prevents race conditions that could cause concurrent sessions to miss updating the target index, thus again resulting in corrupt concurrently-created indexes. Also, fix various other operations to ensure that they ignore invalid indexes resulting from a failed CREATE INDEX CONCURRENTLY command. The most important of these is VACUUM, because an auto-vacuum could easily be launched on the table before corrective action can be taken to fix or remove the invalid index. Also fix DROP INDEX CONCURRENTLY to not disable insertions into the target index until all queries using it are done. Also fix misbehavior if DROP INDEX CONCURRENTLY is canceled: the previous coding could leave an un-droppable index behind.
•
Correct predicate locking for DROP INDEX CONCURRENTLY (Kevin Grittner) Previously, SSI predicate locks were processed at the wrong time, possibly leading to incorrect behavior of serializable transactions executing in parallel with the DROP.
•
Fix buffer locking during WAL replay (Tom Lane) The WAL replay code was insufficiently careful about locking buffers when replaying WAL records that affect more than one page. This could result in hot standby queries transiently seeing inconsistent states, resulting in wrong answers or unexpected failures.
•
Fix an error in WAL generation logic for GIN indexes (Tom Lane) This could result in index corruption, if a torn-page failure occurred.
•
Fix an error in WAL replay logic for SP-GiST indexes (Tom Lane) This could result in index corruption after a crash, or on a standby server.
•
Fix incorrect detection of end-of-base-backup location during WAL recovery (Heikki Linnakangas) This mistake allowed hot standby mode to start up before the database reaches a consistent state.
•
Properly remove startup process’s virtual XID lock when promoting a hot standby server to normal running (Simon Riggs) This oversight could prevent subsequent execution of certain operations such as CREATE INDEX CONCURRENTLY.
Prevent the postmaster from launching new child processes after it’s received a shutdown signal (Tom Lane) This mistake could result in shutdown taking longer than it should, or even never completing at all without additional user action.
2124
Appendix E. Release Notes •
Fix the syslogger process to not fail when log_rotation_age exceeds 2^31 milliseconds (about 25 days) (Tom Lane)
•
Fix WaitLatch() to return promptly when the requested timeout expires (Jeff Janes, Tom Lane) With the previous coding, a steady stream of non-wait-terminating interrupts could delay return from WaitLatch() indefinitely. This has been shown to be a problem for the autovacuum launcher process, and might cause trouble elsewhere as well.
•
Avoid corruption of internal hash tables when out of memory (Hitoshi Harada)
•
Prevent file descriptors for dropped tables from being held open past transaction end (Tom Lane) This should reduce problems with long-since-dropped tables continuing to occupy disk space.
•
Prevent database-wide crash and restart when a new child process is unable to create a pipe for its latch (Tom Lane) Although the new process must fail, there is no good reason to force a database-wide restart, so avoid that. This improves robustness when the kernel is nearly out of file descriptors.
•
Avoid planner crash with joins to unflattened subqueries (Tom Lane)
•
Fix planning of non-strict equivalence clauses above outer joins (Tom Lane) The planner could derive incorrect constraints from a clause equating a non-strict construct to something else, for example WHERE COALESCE(foo, 0) = 0 when foo is coming from the nullable side of an outer join. 9.2 showed this type of error in more cases than previous releases, but the basic bug has been there for a long time.
•
Fix SELECT DISTINCT with index-optimized MIN/MAX on an inheritance tree (Tom Lane) The planner would fail with “failed to re-find MinMaxAggInfo record” given this combination of factors.
•
Make sure the planner sees implicit and explicit casts as equivalent for all purposes, except in the minority of cases where there’s actually a semantic difference (Tom Lane)
•
Include join clauses when considering whether partial indexes can be used for a query (Tom Lane) A strict join clause can be sufficient to establish an x IS NOT NULL predicate, for example. This fixes a planner regression in 9.2, since previous versions could make comparable deductions.
•
Limit growth of planning time when there are many indexable join clauses for the same index (Tom Lane)
•
Improve planner’s ability to prove exclusion constraints from equivalence classes (Tom Lane)
•
Fix partial-row matching in hashed subplans to handle cross-type cases correctly (Tom Lane) This affects multicolumn NOT IN subplans, such as WHERE (a, b) NOT IN (SELECT x, y FROM ...) when for instance b and y are int4 and int8 respectively. This mistake led to wrong answers or crashes depending on the specific datatypes involved.
•
Fix btree mark/restore functions to handle array keys (Tom Lane) This oversight could result in wrong answers from merge joins whose inner side is an index scan using an indexed_column = ANY(array ) condition.
•
Revert patch for taking fewer snapshots (Tom Lane) The 9.2 change to reduce the number of snapshots taken during query execution led to some anomalous behaviors not seen in previous releases, because execution would proceed with a snapshot acquired before locking the tables used by the query. Thus, for example, a query would not be guaranteed to see updates committed by a preceding transaction even if that transaction had exclu-
2125
Appendix E. Release Notes sive lock. We’ll probably revisit this in future releases, but meanwhile put it back the way it was before 9.2. •
Acquire buffer lock when re-fetching the old tuple for an AFTER ROW UPDATE/DELETE trigger (Andres Freund) In very unusual circumstances, this oversight could result in passing incorrect data to a trigger WHEN condition, or to the precheck logic for a foreign-key enforcement trigger. That could result in a crash, or in an incorrect decision about whether to fire the trigger.
•
Fix ALTER COLUMN TYPE to handle inherited check constraints properly (Pavan Deolasee) This worked correctly in pre-8.4 releases, and now works correctly in 8.4 and later.
•
Fix ALTER EXTENSION SET SCHEMA’s failure to move some subsidiary objects into the new schema (Álvaro Herrera, Dimitri Fontaine)
•
Handle CREATE TABLE AS EXECUTE correctly in extended query protocol (Tom Lane)
•
Don’t modify the input parse tree in DROP RULE IF NOT EXISTS and DROP TRIGGER IF NOT EXISTS (Tom Lane)
This mistake would cause errors if a cached statement of one of these types was re-executed. •
Fix REASSIGN OWNED to handle grants on tablespaces (Álvaro Herrera)
•
Ignore incorrect pg_attribute entries for system columns for views (Tom Lane) Views do not have any system columns. However, we forgot to remove such entries when converting a table to a view. That’s fixed properly for 9.3 and later, but in previous branches we need to defend against existing mis-converted views.
•
Fix rule printing to dump INSERT INTO table DEFAULT VALUES correctly (Tom Lane)
•
Guard against stack overflow when there are too many UNION/INTERSECT/EXCEPT clauses in a query (Tom Lane)
•
Prevent platform-dependent failures when dividing the minimum possible integer value by -1 (Xi Wang, Tom Lane)
•
Fix possible access past end of string in date parsing (Hitoshi Harada)
•
Fix failure to advance XID epoch if XID wraparound happens during a checkpoint and wal_level is hot_standby (Tom Lane, Andres Freund) While this mistake had no particular impact on PostgreSQL itself, it was bad for applications that rely on txid_current() and related functions: the TXID value would appear to go backwards.
•
Fix pg_terminate_backend() and pg_cancel_backend() to not throw error for a non-existent target process (Josh Kupershmidt) This case already worked as intended when called by a superuser, but not so much when called by ordinary users.
•
Fix display of pg_stat_replication.sync_state at a page boundary (Kyotaro Horiguchi)
•
Produce an understandable error message if the length of the path name for a Unix-domain socket exceeds the platform-specific limit (Tom Lane, Andrew Dunstan) Formerly, this would result in something quite unhelpful, such as “Non-recoverable failure in name resolution”.
•
Fix memory leaks when sending composite column values to the client (Tom Lane)
•
Save some cycles by not searching for subtransaction locks at commit (Simon Riggs) In a transaction holding many exclusive locks, this useless activity could be quite costly.
2126
Appendix E. Release Notes •
Make pg_ctl more robust about reading the postmaster.pid file (Heikki Linnakangas) This fixes race conditions and possible file descriptor leakage.
•
Fix possible crash in psql if incorrectly-encoded data is presented and the client_encoding setting is a client-only encoding, such as SJIS (Jiang Guiqing)
•
Make pg_dump dump SEQUENCE SET items in the data not pre-data section of the archive (Tom Lane) This fixes an undesirable inconsistency between the meanings of --data-only and --section=data, and also fixes dumping of sequences that are marked as extension configuration tables.
•
Fix pg_dump’s handling of DROP DATABASE commands in --clean mode (Guillaume Lelarge) Beginning in 9.2.0, pg_dump --clean would issue a DROP DATABASE command, which was either useless or dangerous depending on the usage scenario. It no longer does that. This change also fixes the combination of --clean and --create to work sensibly, i.e., emit DROP DATABASE then CREATE DATABASE before reconnecting to the target database.
•
Fix pg_dump for views with circular dependencies and no relation options (Tom Lane) The previous fix to dump relation options when a view is involved in a circular dependency didn’t work right for the case that the view has no options; it emitted ALTER VIEW foo SET () which is invalid syntax.
•
Fix bugs in the restore.sql script emitted by pg_dump in tar output format (Tom Lane) The script would fail outright on tables whose names include upper-case characters. Also, make the script capable of restoring data in --inserts mode as well as the regular COPY mode.
•
Fix pg_restore to accept POSIX-conformant tar files (Brian Weaver, Tom Lane) The original coding of pg_dump’s tar output mode produced files that are not fully conformant with the POSIX standard. This has been corrected for version 9.3. This patch updates previous branches so that they will accept both the incorrect and the corrected formats, in hopes of avoiding compatibility problems when 9.3 comes out.
•
Fix tar files emitted by pg_basebackup to be POSIX conformant (Brian Weaver, Tom Lane)
•
Fix pg_resetxlog to locate postmaster.pid correctly when given a relative path to the data directory (Tom Lane) This mistake could lead to pg_resetxlog not noticing that there is an active postmaster using the data directory.
•
Fix libpq’s lo_import() and lo_export() functions to report file I/O errors properly (Tom Lane)
•
Fix ecpg’s processing of nested structure pointer variables (Muhammad Usama)
•
Fix ecpg’s ecpg_get_data function to handle arrays properly (Michael Meskes)
•
Prevent pg_upgrade from trying to process TOAST tables for system catalogs (Bruce Momjian) This fixes an error seen when the information_schema has been dropped and recreated. Other failures were also possible.
•
Improve pg_upgrade performance by setting synchronous_commit to off in the new cluster (Bruce Momjian)
•
Make contrib/pageinspect’s btree page inspection functions take buffer locks while examining pages (Tom Lane)
•
Work around unportable behavior of malloc(0) and realloc(NULL, 0) (Tom Lane)
2127
Appendix E. Release Notes On platforms where these calls return NULL, some code mistakenly thought that meant out-ofmemory. This is known to have broken pg_dump for databases containing no user-defined aggregates. There might be other cases as well. •
Ensure that make install for an extension creates the extension installation directory (Cédric Villemain) Previously, this step was missed if MODULEDIR was set in the extension’s Makefile.
•
Fix pgxs support for building loadable modules on AIX (Tom Lane) Building modules outside the original source tree didn’t work on AIX.
•
Update time zone data files to tzdata release 2012j for DST law changes in Cuba, Israel, Jordan, Libya, Palestine, Western Samoa, and portions of Brazil.
E.43. Release 9.2.1 Release date: 2012-09-24 This release contains a variety of fixes from 9.2.0. For information about new features in the 9.2 major release, see Section E.44.
E.43.1. Migration to Version 9.2.1 A dump/restore is not required for those running 9.2.X. However, you may need to perform REINDEX and/or VACUUM operations to recover from the effects of the data corruption bug described in the first changelog item below.
E.43.2. Changes •
Fix persistence marking of shared buffers during WAL replay (Jeff Davis) This mistake can result in buffers not being written out during checkpoints, resulting in data corruption if the server later crashes without ever having written those buffers. Corruption can occur on any server following crash recovery, but it is significantly more likely to occur on standby slave servers since those perform much more WAL replay. There is a low probability of corruption of btree and GIN indexes. There is a much higher probability of corruption of table “visibility maps”, which might lead to wrong answers from index-only scans. Table data proper cannot be corrupted by this bug. While no index corruption due to this bug is known to have occurred in the field, as a precautionary measure it is recommended that production installations REINDEX all btree and GIN indexes at a convenient time after upgrading to 9.2.1. Also,
it is recommended to perform a VACUUM of all tables while having vacuum_freeze_table_age set to zero. This will fix any incorrect visibility map data. vacuum_cost_delay can be adjusted to reduce the performance impact of vacuuming, while causing it to take longer to finish.
•
Fix possible incorrect sorting of output from queries involving WHERE indexed_column IN (list_of_values) (Tom Lane)
2128
Appendix E. Release Notes •
Fix planner failure for queries involving GROUP BY expressions along with window functions and aggregates (Tom Lane)
•
Fix planner’s assignment of executor parameters (Tom Lane) This error could result in wrong answers from queries that scan the same WITH subquery multiple times.
•
Improve planner’s handling of join conditions in index scans (Tom Lane)
•
Improve selectivity estimation for text search queries involving prefixes, i.e. word:* patterns (Tom Lane)
•
Fix delayed recognition of permissions changes (Tom Lane) A command that needed no locks other than ones its transaction already had might fail to notice a concurrent GRANT or REVOKE that committed since the start of its transaction.
•
Fix ANALYZE to not fail when a column is a domain over an array type (Tom Lane)
•
Prevent PL/Perl from crashing if a recursive PL/Perl function is redefined while being executed (Tom Lane)
•
Work around possible misoptimization in PL/Perl (Tom Lane) Some Linux distributions contain an incorrect version of pthread.h that results in incorrect compiled code in PL/Perl, leading to crashes if a PL/Perl function calls another one that throws an error.
•
Remove unnecessary dependency on pg_config from pg_upgrade (Peter Eisentraut)
•
Update time zone data files to tzdata release 2012f for DST law changes in Fiji
E.44. Release 9.2 Release date: 2012-09-10
E.44.1. Overview This release has been largely focused on performance improvements, though new SQL features are not lacking. Work also continues in the area of replication support. Major enhancements include: •
Allow queries to retrieve data only from indexes, avoiding heap access (index-only scans)
•
Allow the planner to generate custom plans for specific parameter values even when using prepared statements
•
Improve the planner’s ability to use nested loops with inner index scans
•
Allow streaming replication slaves to forward data to other slaves (cascading replication)
•
Allow pg_basebackup to make base backups from standby servers
•
Add a pg_receivexlog tool to archive WAL file changes as they are written
•
Add the SP-GiST (Space-Partitioned GiST) index access method
•
Add support for range data types
•
Add a JSON data type
2129
Appendix E. Release Notes •
Add a security_barrier option for views
•
Allow libpq connection strings to have the format of a URI
•
Add a single-row processing mode to libpq for better handling of large result sets
The above items are explained in more detail in the sections below.
E.44.2. Migration to Version 9.2 A dump/restore using pg_dump, or use of pg_upgrade, is required for those wishing to migrate data from any previous release. Version 9.2 contains a number of changes that may affect compatibility with previous releases. Observe the following incompatibilities:
E.44.2.1. System Catalogs •
Remove the spclocation field from pg_tablespace (Magnus Hagander) This field was duplicative of the symbolic links that actually define tablespace locations, and thus risked errors of omission when moving a tablespace. This change allows tablespace directories to be moved while the server is down, by manually adjusting the symbolic links. To replace this field, we have added pg_tablespace_location() to allow querying of the symbolic links.
•
Move tsvector most-common-element statistics to new pg_stats columns (Alexander Korotkov) Consult most_common_elems and most_common_elem_freqs for the data formerly available in most_common_vals and most_common_freqs for a tsvector column.
E.44.2.2. Functions •
Remove hstore’s => operator (Robert Haas) Users should now use hstore(text, text). Since PostgreSQL 9.0, a warning message has been emitted when an operator named => is created because the SQL standard reserves that token for another use.
•
Ensure that xpath() escapes special characters in string values (Florian Pflug) Without this it is possible for the result not to be valid XML.
•
Make pg_relation_size() and friends return NULL if the object does not exist (Phil Sorber) This prevents queries that call these functions from returning errors immediately after a concurrent DROP.
•
Make EXTRACT(EPOCH FROM timestamp without time zone) measure the epoch from local midnight, not UTC midnight (Tom Lane) This change reverts an ill-considered change made in release 7.3. Measuring from UTC midnight was inconsistent because it made the result dependent on the timezone setting, which computations for timestamp without time zone should not be. The previous behavior remains available by casting the input value to timestamp with time zone.
•
Properly parse time strings with trailing yesterday, today, and tomorrow (Dean Rasheed)
2130
Appendix E. Release Notes Previously, SELECT ’04:00:00 yesterday’::timestamp returned yesterday’s date at midnight. •
Fix to_date() and to_timestamp() to wrap incomplete dates toward 2020 (Bruce Momjian) Previously, supplied years and year masks of less than four digits wrapped inconsistently.
E.44.2.3. Object Modification •
Prevent ALTER DOMAIN from working on non-domain types (Peter Eisentraut) Owner and schema changes were previously possible on non-domain types.
•
No longer forcibly lowercase procedural language names in CREATE FUNCTION (Robert Haas) While unquoted language identifiers are still lowercased, strings and quoted identifiers are no longer forcibly down-cased. Thus for example CREATE FUNCTION ... LANGUAGE ’C’ will no longer work; it must be spelled ’c’, or better omit the quotes.
•
Change system-generated names of foreign key enforcement triggers (Tom Lane) This change ensures that the triggers fire in the correct order in some corner cases involving selfreferential foreign key constraints.
E.44.2.4. Command-Line Tools •
Provide consistent backquote, variable expansion, and quoted substring behavior in psql metacommand arguments (Tom Lane) Previously, such references were treated oddly when not separated by whitespace from adjacent text. For example ’FOO’BAR was output as FOO BAR (unexpected insertion of a space) and FOO’BAR’BAZ was output unchanged (not removing the quotes as most would expect).
•
No longer treat clusterdb table names as double-quoted; no longer treat reindexdb table and index names as double-quoted (Bruce Momjian) Users must now include double-quotes in the command arguments if quoting is wanted.
•
createuser no longer prompts for option settings by default (Peter Eisentraut) Use --interactive to obtain the old behavior.
•
Disable prompting for the user name in dropuser unless --interactive is specified (Peter Eisentraut)
E.44.2.5. Server Settings •
Add server parameters for specifying the locations of server-side SSL files (Peter Eisentraut) This allows changing the names and locations of the files that were previously hard-coded as server.crt, server.key, root.crt, and root.crl in the data directory. The server will no longer examine root.crt or root.crl by default; to load these files, the associated parameters must be set to non-default values.
•
Remove the silent_mode parameter (Heikki Linnakangas) Similar behavior can be obtained with pg_ctl start -l postmaster.log.
2131
Appendix E. Release Notes •
Remove the wal_sender_delay parameter, as it is no longer needed (Tom Lane)
•
Remove the custom_variable_classes parameter (Tom Lane) The checking provided by this setting was dubious. Now any setting can be prefixed by any class name.
E.44.2.6. Monitoring •
Rename pg_stat_activity.procpid to pid, to match other system tables (Magnus Hagander)
•
Create a separate pg_stat_activity column to report process state (Scott Mead, Magnus Hagander) The previous query and query_start values now remain available for an idle session, allowing enhanced analysis.
•
Rename pg_stat_activity.current_query to query because it is not cleared when the query completes (Magnus Hagander)
•
Change all SQL-level statistics timing values to be float8 columns measured in milliseconds (Tom Lane) This change eliminates the designed-in assumption that the values are accurate to microseconds and no more (since the float8 values can be fractional). The columns affected are pg_stat_user_functions.total_time, pg_stat_user_functions.self_time, pg_stat_xact_user_functions.total_time, and pg_stat_xact_user_functions.self_time. The statistics functions underlying these columns now also return float8 milliseconds, rather than bigint microseconds. contrib/pg_stat_statements’ total_time column is now also measured in milliseconds.
E.44.3. Changes Below you will find a detailed account of the changes between PostgreSQL 9.2 and the previous major release.
E.44.3.1. Server E.44.3.1.1. Performance •
Allow queries to retrieve data only from indexes, avoiding heap access (Robert Haas, Ibrar Ahmed, Heikki Linnakangas, Tom Lane) This feature is often called index-only scans. Heap access can be skipped for heap pages containing only tuples that are visible to all sessions, as reported by the visibility map; so the benefit applies mainly to mostly-static data. The visibility map was made crash-safe as a necessary part of implementing this feature.
•
Add the SP-GiST (Space-Partitioned GiST) index access method (Teodor Sigaev, Oleg Bartunov, Tom Lane)
2132
Appendix E. Release Notes SP-GiST is comparable to GiST in flexibility, but supports unbalanced partitioned search structures rather than balanced trees. For suitable problems, SP-GiST can be faster than GiST in both index build time and search time. •
Allow group commit to work effectively under heavy load (Peter Geoghegan, Simon Riggs, Heikki Linnakangas) Previously, batching of commits became ineffective as the write workload increased, because of internal lock contention.
•
Allow uncontended locks to be managed using a new fast-path lock mechanism (Robert Haas)
•
Reduce overhead of creating virtual transaction ID locks (Robert Haas)
•
Reduce the overhead of serializable isolation level locks (Dan Ports)
•
Improve PowerPC and Itanium spinlock performance (Manabu Ori, Robert Haas, Tom Lane)
•
Reduce overhead for shared invalidation cache messages (Robert Haas)
•
Move the frequently accessed members of the PGPROC shared memory array to a separate array (Pavan Deolasee, Heikki Linnakangas, Robert Haas)
•
Improve COPY performance by adding tuples to the heap in batches (Heikki Linnakangas)
•
Improve GiST index performance for geometric data types by producing better trees with less memory allocation overhead (Alexander Korotkov)
•
Improve GiST index build times (Alexander Korotkov, Heikki Linnakangas)
•
Allow hint bits to be set sooner for temporary and unlogged tables (Robert Haas)
•
Allow sorting to be performed by inlined, non-SQL-callable comparison functions (Peter Geoghegan, Robert Haas, Tom Lane)
•
Make the number of CLOG buffers scale based on shared_buffers (Robert Haas, Simon Riggs, Tom Lane)
•
Improve performance of buffer pool scans that occur when tables or databases are dropped (Jeff Janes, Simon Riggs)
•
Improve performance of checkpointer’s fsync-request queue when many tables are being dropped or truncated (Tom Lane)
•
Pass the safe number of file descriptors to child processes on Windows (Heikki Linnakangas) This allows Windows sessions to use more open file descriptors than before.
E.44.3.1.2. Process Management •
Create a dedicated background process to perform checkpoints (Simon Riggs) Formerly the background writer did both dirty-page writing and checkpointing. Separating this into two processes allows each goal to be accomplished more predictably.
•
Improve asynchronous commit behavior by waking the walwriter sooner (Simon Riggs) Previously, only wal_writer_delay triggered WAL flushing to disk; now filling a WAL buffer also triggers WAL writes.
•
Allow the bgwriter, walwriter, checkpointer, statistics collector, log collector, and archiver background processes to sleep more efficiently during periods of inactivity (Peter Geoghegan, Tom Lane)
2133
Appendix E. Release Notes This series of changes reduces the frequency of process wake-ups when there is nothing to do, dramatically reducing power consumption on idle servers.
E.44.3.1.3. Optimizer •
Allow the planner to generate custom plans for specific parameter values even when using prepared statements (Tom Lane) In the past, a prepared statement always had a single “generic” plan that was used for all parameter values, which was frequently much inferior to the plans used for non-prepared statements containing explicit constant values. Now, the planner attempts to generate custom plans for specific parameter values. A generic plan will only be used after custom plans have repeatedly proven to provide no benefit. This change should eliminate the performance penalties formerly seen from use of prepared statements (including non-dynamic statements in PL/pgSQL).
•
Improve the planner’s ability to use nested loops with inner index scans (Tom Lane) The new “parameterized path” mechanism allows inner index scans to use values from relations that are more than one join level up from the scan. This can greatly improve performance in situations where semantic restrictions (such as outer joins) limit the allowed join orderings.
•
Improve the planning API for foreign data wrappers (Etsuro Fujita, Shigeru Hanada, Tom Lane) Wrappers can now provide multiple access “paths” for their tables, allowing more flexibility in join planning.
•
Recognize self-contradictory restriction clauses for non-table relations (Tom Lane) This check is only performed when constraint_exclusion is on.
•
Allow indexed_col op ANY(ARRAY[...]) conditions to be used in plain index scans and index-only scans (Tom Lane) Formerly such conditions could only be used in bitmap index scans.
•
Support MIN/MAX index optimizations on boolean columns (Marti Raudsepp)
•
Account for set-returning functions in SELECT target lists when setting row count estimates (Tom Lane)
•
Fix planner to handle indexes with duplicated columns more reliably (Tom Lane)
•
Collect and use element-frequency statistics for arrays (Alexander Korotkov, Tom Lane) This change improves selectivity estimation for the array operators (array containment and overlaps).
•
Allow statistics to be collected for foreign tables (Etsuro Fujita)
•
Improve cost estimates for use of partial indexes (Tom Lane)
•
Improve the planner’s ability to use statistics for columns referenced in subqueries (Tom Lane)
•
Improve statistical estimates for subqueries using DISTINCT (Tom Lane)
E.44.3.1.4. Authentication •
Do not treat role names and samerole specified in pg_hba.conf as automatically including superusers (Andrew Dunstan) This makes it easier to use reject lines with group roles.
2134
Appendix E. Release Notes •
Adjust pg_hba.conf processing to handle token parsing more consistently (Brendan Jurd, Álvaro Herrera)
•
Disallow empty pg_hba.conf files (Tom Lane) This was done to more quickly detect misconfiguration.
•
Make superuser privilege imply replication privilege (Noah Misch) This avoids the need to explicitly assign such privileges.
E.44.3.1.5. Monitoring •
Attempt to log the current query string during a backend crash (Marti Raudsepp)
•
Make logging of autovacuum I/O activity more verbose (Greg Smith, Noah Misch) This logging is triggered by log_autovacuum_min_duration.
•
Make WAL replay report failures sooner (Fujii Masao) There were some cases where failures were only reported once the server went into master mode.
•
Add pg_xlog_location_diff() to simplify WAL location comparisons (Euler Taveira de Oliveira) This is useful for computing replication lag.
•
Support configurable event log application names on Windows (MauMau, Magnus Hagander) This allows different instances to use the event log with different identifiers, by setting the event_source server parameter, which is similar to how syslog_ident works.
•
Change “unexpected EOF” messages to DEBUG1 level, except when there is an open transaction (Magnus Hagander) This change reduces log chatter caused by applications that close database connections ungracefully.
E.44.3.1.6. Statistical Views •
Track temporary file sizes and file counts in the pg_stat_database system view (Tomas Vondra)
•
Add a deadlock counter to the pg_stat_database system view (Magnus Hagander)
•
Add a server parameter track_io_timing to track I/O timings (Ants Aasma, Robert Haas)
•
Report checkpoint timing information in pg_stat_bgwriter (Greg Smith, Peter Geoghegan)
E.44.3.1.7. Server Settings •
Silently ignore nonexistent schemas specified in search_path (Tom Lane) This makes it more convenient to use generic path settings, which might include some schemas that don’t exist in all databases.
•
Allow superusers to set deadlock_timeout per-session, not just per-cluster (Noah Misch) This allows deadlock_timeout to be reduced for transactions that are likely to be involved in a deadlock, thus detecting the failure more quickly. Alternatively, increasing the value can be used to reduce the chances of a session being chosen for cancellation due to a deadlock.
2135
Appendix E. Release Notes •
Add a server parameter temp_file_limit to constrain temporary file space usage per session (Mark Kirkwood)
•
Allow a superuser to SET an extension’s superuser-only custom variable before loading the associated extension (Tom Lane) The system now remembers whether a SET was performed by a superuser, so that proper privilege checking can be done when the extension is loaded.
•
Add postmaster -C option to query configuration parameters (Bruce Momjian) This allows pg_ctl to better handle cases where PGDATA or -D points to a configuration-only directory.
•
Replace an empty locale name with the implied value in CREATE DATABASE (Tom Lane) This prevents cases where pg_database.datcollate or datctype could be interpreted differently after a server restart.
E.44.3.1.7.1. postgresql.conf •
Allow multiple errors in postgresql.conf to be reported, rather than just the first one (Alexey Klyukin, Tom Lane)
•
Allow a reload of postgresql.conf to be processed by all sessions, even if there are some settings that are invalid for particular sessions (Alexey Klyukin) Previously, such not-valid-within-session values would cause all setting changes to be ignored by that session.
•
Add an include_if_exists facility for configuration files (Greg Smith) This works the same as include, except that an error is not thrown if the file is missing.
•
Identify the server time zone during initdb, and set postgresql.conf entries timezone and log_timezone accordingly (Tom Lane) This avoids expensive time zone probes during server start.
•
Fix pg_settings to report postgresql.conf line numbers on Windows (Tom Lane)
E.44.3.2. Replication and Recovery •
Allow streaming replication slaves to forward data to other slaves (cascading replication) (Fujii Masao) Previously, only the master server could supply streaming replication log files to standby servers.
•
Add new synchronous_commit mode remote_write (Fujii Masao, Simon Riggs) This mode waits for the standby server to write transaction data to its own operating system, but does not wait for the data to be flushed to the standby’s disk.
•
Add a pg_receivexlog tool to archive WAL file changes as they are written, rather than waiting for completed WAL files (Magnus Hagander)
•
Allow pg_basebackup to make base backups from standby servers (Jun Ishizuka, Fujii Masao) This feature lets the work of making new base backups be off-loaded from the primary server.
2136
Appendix E. Release Notes •
Allow streaming of WAL files while pg_basebackup is performing a backup (Magnus Hagander) This allows passing of WAL files to the standby before they are discarded on the primary.
E.44.3.3. Queries •
Cancel the running query if the client gets disconnected (Florian Pflug) If the backend detects loss of client connection during a query, it will now cancel the query rather than attempting to finish it.
•
Retain column names at run time for row expressions (Andrew Dunstan, Tom Lane) This change allows better results when a row value is converted to hstore or json type: the fields of the resulting value will now have the expected names.
•
Improve column labels used for sub-SELECT results (Marti Raudsepp) Previously, the generic label ?column? was used.
•
Improve heuristics for determining the types of unknown values (Tom Lane) The longstanding rule that an unknown constant might have the same type as the value on the other side of the operator using it is now applied when considering polymorphic operators, not only for simple operator matches.
•
Warn about creating casts to or from domain types (Robert Haas) Such casts have no effect.
•
When a row fails a CHECK or NOT NULL constraint, show the row’s contents as error detail (Jan Kundrát) This should make it easier to identify which row is problematic when an insert or update is processing many rows.
E.44.3.4. Object Manipulation •
Provide more reliable operation during concurrent DDL (Robert Haas, Noah Misch) This change adds locking that should eliminate “cache lookup failed” errors in many scenarios. Also, it is no longer possible to add relations to a schema that is being concurrently dropped, a scenario that formerly led to inconsistent system catalog contents.
•
Add CONCURRENTLY option to DROP INDEX (Simon Riggs) This allows index removal without blocking other sessions.
•
Allow foreign data wrappers to have per-column options (Shigeru Hanada)
•
Improve pretty-printing of view definitions (Andrew Dunstan)
E.44.3.4.1. Constraints •
Allow CHECK constraints to be declared NOT VALID (Álvaro Herrera) Adding a NOT VALID constraint does not cause the table to be scanned to verify that existing rows meet the constraint. Subsequently, newly added or updated rows are checked. Such constraints are
2137
Appendix E. Release Notes ignored by the planner when considering constraint_exclusion, since it is not certain that all rows meet the constraint. The new ALTER TABLE VALIDATE command allows NOT VALID constraints to be checked for existing rows, after which they are converted into ordinary constraints. •
Allow CHECK constraints to be declared NO INHERIT (Nikhil Sontakke, Alex Hunsaker, Álvaro Herrera) This makes them enforceable only on the parent table, not on child tables.
•
Add the ability to rename constraints (Peter Eisentraut)
E.44.3.4.2. ALTER •
Reduce need to rebuild tables and indexes for certain ALTER TABLE ... ALTER COLUMN TYPE operations (Noah Misch) Increasing the length limit for a varchar or varbit column, or removing the limit altogether, no longer requires a table rewrite. Similarly, increasing the allowable precision of a numeric column, or changing a column from constrained numeric to unconstrained numeric, no longer requires a table rewrite. Table rewrites are also avoided in similar cases involving the interval, timestamp, and timestamptz types.
•
Avoid having ALTER TABLE revalidate foreign key constraints in some cases where it is not necessary (Noah Misch)
•
Add IF EXISTS options to some ALTER commands (Pavel Stehule) For example, ALTER FOREIGN TABLE IF EXISTS foo RENAME TO bar.
•
Add ALTER FOREIGN DATA WRAPPER ... RENAME and ALTER SERVER ... RENAME (Peter Eisentraut)
•
Add ALTER DOMAIN ... RENAME (Peter Eisentraut) You could already rename domains using ALTER TYPE.
•
Throw an error for ALTER DOMAIN ... DROP CONSTRAINT on a nonexistent constraint (Peter Eisentraut) An IF EXISTS option has been added to provide the previous behavior.
E.44.3.4.3. CREATE TABLE •
Allow CREATE TABLE (LIKE ...) from foreign tables, views, and composite types (Peter Eisentraut) For example, this allows a table to be created whose schema matches a view.
•
Fix CREATE TABLE (LIKE ...) to avoid index name conflicts when copying index comments (Tom Lane)
•
Fix CREATE TABLE ... AS EXECUTE to handle WITH NO DATA and column name specifications (Tom Lane)
2138
Appendix E. Release Notes E.44.3.4.4. Object Permissions •
Add a security_barrier option for views (KaiGai Kohei, Robert Haas) This option prevents optimizations that might allow view-protected data to be exposed to users, for example pushing a clause involving an insecure function into the WHERE clause of the view. Such views can be expected to perform more poorly than ordinary views.
•
Add a new LEAKPROOF function attribute to mark functions that can safely be pushed down into security_barrier views (KaiGai Kohei)
•
Add support for privileges on data types (Peter Eisentraut) This adds support for the SQL-conforming USAGE privilege on types and domains. The intent is to be able to restrict which users can create dependencies on types, since such dependencies limit the owner’s ability to alter the type.
•
Check for INSERT privileges in SELECT INTO / CREATE TABLE AS (KaiGai Kohei) Because the object is being created by SELECT INTO or CREATE TABLE AS, the creator would ordinarily have insert permissions; but there are corner cases where this is not true, such as when ALTER DEFAULT PRIVILEGES has removed such permissions.
E.44.3.5. Utility Operations •
Allow VACUUM to more easily skip pages that cannot be locked (Simon Riggs, Robert Haas) This change should greatly reduce the incidence of VACUUM getting “stuck” waiting for other sessions.
•
Make EXPLAIN (BUFFERS) count blocks dirtied and written (Robert Haas)
•
Make EXPLAIN ANALYZE report the number of rows rejected by filter steps (Marko Tiikkaja)
•
Allow EXPLAIN ANALYZE to avoid timing overhead when time values are not wanted (Tomas Vondra) This is accomplished by setting the new TIMING option to FALSE.
E.44.3.6. Data Types •
Add support for range data types (Jeff Davis, Tom Lane, Alexander Korotkov) A range data type stores a lower and upper bound belonging to its base data type. It supports operations like contains, overlaps, and intersection.
•
Add a JSON data type (Robert Haas) This type stores JSON (JavaScript Object Notation) data with proper validation.
•
Add array_to_json() and row_to_json() (Andrew Dunstan)
•
Add a SMALLSERIAL data type (Mike Pultz) This is like SERIAL, except it stores the sequence in a two-byte integer column (int2).
•
Allow domains to be declared NOT VALID (Álvaro Herrera)
2139
Appendix E. Release Notes This option can be set at domain creation time, or via ALTER DOMAIN ... ADD CONSTRAINT ... NOT VALID. ALTER DOMAIN ... VALIDATE CONSTRAINT fully validates the constraint. •
Support more locale-specific formatting options for the money data type (Tom Lane) Specifically, honor all the POSIX options for ordering of the value, sign, and currency symbol in monetary output. Also, make sure that the thousands separator is only inserted to the left of the decimal point, as required by POSIX.
•
Add bitwise “and”, “or”, and “not” operators for the macaddr data type (Brendan Jurd)
•
Allow xpath() to return a single-element XML array when supplied a scalar value (Florian Pflug) Previously, it returned an empty array. This change will also cause xpath_exists() to return true, not false, for such expressions.
•
Improve XML error handling to be more robust (Florian Pflug)
E.44.3.7. Functions •
Allow non-superusers to use pg_cancel_backend() and pg_terminate_backend() on other sessions belonging to the same user (Magnus Hagander, Josh Kupershmidt, Dan Farina) Previously only superusers were allowed to use these functions.
•
Allow importing and exporting of transaction snapshots (Joachim Wieland, Tom Lane) This allows multiple transactions to share identical views of the database state. Snapshots are exported via pg_export_snapshot() and imported via SET TRANSACTION SNAPSHOT. Only snapshots from currently-running transactions can be imported.
•
Support COLLATION FOR on expressions (Peter Eisentraut) This returns a string representing the collation of the expression.
•
Add pg_opfamily_is_visible() (Josh Kupershmidt)
•
Add a numeric variant of pg_size_pretty() for use with pg_xlog_location_diff() (Fujii Masao)
•
Add a pg_trigger_depth() function (Kevin Grittner) This reports the current trigger call depth.
•
Allow string_agg() to process bytea values (Pavel Stehule)
•
Fix regular expressions in which a back-reference occurs within a larger quantified subexpression (Tom Lane) For example, ^(\w+)( \1)+$. Previous releases did not check that the back-reference actually matched the first occurrence.
Add composite-type attributes to the information schema element_types view (Peter Eisentraut)
•
Implement interval_type columns in the information schema (Peter Eisentraut)
2140
Appendix E. Release Notes Formerly these columns read as nulls. •
Implement collation-related columns in the information schema attributes, columns, domains, and element_types views (Peter Eisentraut)
•
Implement the with_hierarchy column in the information schema table_privileges view (Peter Eisentraut)
•
Add display of sequence USAGE privileges to information schema (Peter Eisentraut)
•
Make the information schema show default privileges (Peter Eisentraut) Previously, non-empty default permissions were not represented in the views.
E.44.3.9. Server-Side Languages E.44.3.9.1. PL/pgSQL Server-Side Language •
Allow the PL/pgSQL OPEN cursor command to supply parameters by name (Yeb Havinga)
•
Add a GET STACKED DIAGNOSTICS PL/pgSQL command to retrieve exception info (Pavel Stehule)
•
Speed up PL/pgSQL array assignment by caching type information (Pavel Stehule)
•
Improve performance and memory consumption for long chains of ELSIF clauses (Tom Lane)
•
Output the function signature, not just the name, in PL/pgSQL error messages (Pavel Stehule)
E.44.3.9.2. PL/Python Server-Side Language •
Add PL/Python SPI cursor support (Jan Urbanski) This allows PL/Python to read partial result sets.
•
Add result metadata functions to PL/Python (Peter Eisentraut) Specifically, this adds result object functions .colnames, .coltypes, and .coltypmods.
•
Remove support for Python 2.2 (Peter Eisentraut)
E.44.3.9.3. SQL Server-Side Language •
Allow SQL-language functions to reference parameters by name (Matthew Draper) To use this, simply name the function arguments and then reference the argument names in the SQL function body.
E.44.3.10. Client Applications •
Add initdb options --auth-local and --auth-host (Peter Eisentraut) This allows separate control of local and host pg_hba.conf authentication settings. --auth still controls both.
2141
Appendix E. Release Notes •
Add --replication/--no-replication flags to createuser to control replication permission (Fujii Masao)
•
Add the --if-exists option to dropdb and dropuser (Josh Kupershmidt)
•
Give command-line tools the ability to specify the name of the database to connect to, and fall back to template1 if a postgres database connection fails (Robert Haas)
E.44.3.10.1. psql •
Add a display mode to auto-expand output based on the display width (Peter Eisentraut) This adds the auto option to the \x command, which switches to the expanded mode when the normal output would be wider than the screen.
•
Allow inclusion of a script file that is named relative to the directory of the file from which it was invoked (Gurjeet Singh) This is done with a new command \ir.
•
Add support for non-ASCII characters in psql variable names (Tom Lane)
•
Add support for major-version-specific .psqlrc files (Bruce Momjian) psql already supported minor-version-specific .psqlrc files.
•
Provide environment variable overrides for psql history and startup file locations (Andrew Dunstan) PSQL_HISTORY and PSQLRC now determine these file names if set.
•
Add a \setenv command to modify the environment variables passed to child processes (Andrew Dunstan)
•
Name psql’s temporary editor files with a .sql extension (Peter Eisentraut) This allows extension-sensitive editors to select the right mode.
•
Allow psql to use zero-byte field and record separators (Peter Eisentraut) Various shell tools use zero-byte (NUL) separators, e.g. find.
•
Make the \timing option report times for failed queries (Magnus Hagander) Previously times were reported only for successful queries.
•
Unify and tighten psql’s treatment of \copy and SQL COPY (Noah Misch) This fix makes failure behavior more predictable and honors \set ON_ERROR_ROLLBACK.
E.44.3.10.2. Informational Commands •
Make \d on a sequence show the table/column name owning it (Magnus Hagander)
•
Show statistics target for columns in \d+ (Magnus Hagander)
•
Show role password expiration dates in \du (Fabrízio de Royes Mello)
•
Display comments for casts, conversions, domains, and languages (Josh Kupershmidt) These are included in the output of \dC+, \dc+, \dD+, and \dL respectively.
•
Display comments for SQL/MED objects (Josh Kupershmidt) These are included in the output of \des+, \det+, and \dew+ for foreign servers, foreign tables, and foreign data wrappers respectively.
2142
Appendix E. Release Notes •
Change \dd to display comments only for object types without their own backslash command (Josh Kupershmidt)
E.44.3.10.3. Tab Completion •
In psql tab completion, complete SQL keywords in either upper or lower case according to the new COMP_KEYWORD_CASE setting (Peter Eisentraut)
•
Add tab completion support for EXECUTE (Andreas Karlsson)
•
Allow tab completion of role references in GRANT/REVOKE (Peter Eisentraut)
•
Allow tab completion of file names to supply quotes, when necessary (Noah Misch)
•
Change tab completion support for TABLE to also include views (Magnus Hagander)
E.44.3.10.4. pg_dump •
Add an --exclude-table-data option to pg_dump (Andrew Dunstan) This allows dumping of a table’s definition but not its data, on a per-table basis.
•
Add a --section option to pg_dump and pg_restore (Andrew Dunstan) Valid values are pre-data, data, and post-data. The option can be given more than once to select two or more sections.
•
Make pg_dumpall dump all roles first, then all configuration settings on roles (Phil Sorber) This allows a role’s configuration settings to mention other roles without generating an error.
•
Allow pg_dumpall to avoid errors if the postgres database is missing in the new cluster (Robert Haas)
•
Dump foreign server user mappings in user name order (Peter Eisentraut) This helps produce deterministic dump files.
•
Dump operators in a predictable order (Peter Eisentraut)
•
Tighten rules for when extension configuration tables are dumped by pg_dump (Tom Lane)
•
Make pg_dump emit more useful dependency information (Tom Lane) The dependency links included in archive-format dumps were formerly of very limited use, because they frequently referenced objects that appeared nowhere in the dump. Now they represent actual dependencies (possibly indirect) among the dumped objects.
•
Improve pg_dump’s performance when dumping many database objects (Tom Lane)
E.44.3.11. libpq •
Allow libpq connection strings to have the format of a URI (Alexander Shulgin) The syntax begins with postgres://. This can allow applications to avoid implementing their own parser for URIs representing database connections.
•
Add a connection option to disable SSL compression (Laurenz Albe)
2143
Appendix E. Release Notes This can be used to remove the overhead of SSL compression on fast networks. •
Add a single-row processing mode for better handling of large result sets (Kyotaro Horiguchi, Marko Kreen) Previously, libpq always collected the entire query result in memory before passing it back to the application.
•
Add
const qualifiers to the declarations of the functions PQconnectStartParams, and PQpingParams (Lionel Elie Mamane)
PQconnectdbParams,
•
Allow the .pgpass file to include escaped characters in the password field (Robert Haas)
•
Make library functions use abort() instead of exit() when it is necessary to terminate the process (Peter Eisentraut) This choice does not interfere with the normal exit codes used by the program, and generates a signal that can be caught by the caller.
E.44.3.12. Source Code •
Remove dead ports (Peter Eisentraut) The following platforms are no longer supported: dgux, nextstep, sunos4, svr4, ultrix4, univel, bsdi.
•
Add support for building with MS Visual Studio 2010 (Brar Piening)
•
Enable compiling with the MinGW-w64 32-bit compiler (Lars Kanis)
•
Install plpgsql.h into include/server during installation (Heikki Linnakangas)
•
Improve the latch facility to include detection of postmaster death (Peter Geoghegan, Heikki Linnakangas, Tom Lane) This eliminates one of the main reasons that background processes formerly had to wake up to poll for events.
•
Use C flexible array members, where supported (Peter Eisentraut)
•
Improve the concurrent transaction regression tests (isolationtester) (Noah Misch)
•
Modify thread_test to create its test files in the current directory, rather than /tmp (Bruce Momjian)
•
Improve flex and bison warning and error reporting (Tom Lane)
•
Add memory barrier support (Robert Haas) This is currently unused.
•
Modify pgindent to use a typedef file (Bruce Momjian)
•
Add a hook for processing messages due to be sent to the server log (Martin Pihlak)
•
Add object access hooks for DROP commands (KaiGai Kohei)
•
Centralize DROP handling for some object types (KaiGai Kohei)
•
Add a pg_upgrade test suite (Peter Eisentraut)
•
Sync regular expression code with TCL 8.5.11 and improve internal processing (Tom Lane)
•
Move CRC tables to libpgport, and provide them in a separate include file (Daniel Farina)
•
Add options to git_changelog for use in major release note creation (Bruce Momjian)
•
Support Linux’s /proc/self/oom_score_adj API (Tom Lane)
2144
Appendix E. Release Notes
E.44.3.13. Additional Modules •
Improve efficiency of dblink by using libpq’s new single-row processing mode (Kyotaro Horiguchi, Marko Kreen) This improvement does not apply to dblink_send_query()/dblink_get_result().
•
Support force_not_null option in file_fdw (Shigeru Hanada)
•
Implement dry-run mode for pg_archivecleanup (Gabriele Bartolini) This only outputs the names of files to be deleted.
•
Add new pgbench switches --unlogged-tables, --tablespace, and --index-tablespace (Robert Haas)
•
Change pg_test_fsync to test for a fixed amount of time, rather than a fixed number of cycles (Bruce Momjian) The -o/cycles option was removed, and -s/seconds added.
•
Add a pg_test_timing utility to measure clock monotonicity and timing overhead (Ants Aasma, Greg Smith)
•
Add a tcn (triggered change notification) module to generate NOTIFY events on table changes (Kevin Grittner)
E.44.3.13.1. pg_upgrade •
Adjust pg_upgrade environment variables (Bruce Momjian) Rename data, bin, and port environment variables to begin with PG, and support PGPORTOLD/PGPORTNEW, to replace PGPORT.
•
Overhaul pg_upgrade logging and failure reporting (Bruce Momjian) Create four append-only log files, and delete them on success. Add -r/--retain option to unconditionally retain these files. Also remove pg_upgrade options -g/-G/-l options as unnecessary, and tighten log file permissions.
•
Make pg_upgrade create a script to incrementally generate more accurate optimizer statistics (Bruce Momjian) This reduces the time needed to generate minimal cluster statistics after an upgrade.
•
Allow pg_upgrade to upgrade an old cluster that does not have a postgres database (Bruce Momjian)
•
Allow pg_upgrade to handle cases where some old or new databases are missing, as long as they are empty (Bruce Momjian)
•
Allow pg_upgrade to handle configuration-only directory installations (Bruce Momjian)
•
In pg_upgrade, add -o/-O options to pass parameters to the servers (Bruce Momjian) This is useful for configuration-only directory installs.
•
Change pg_upgrade to use port 50432 by default (Bruce Momjian) This helps avoid unintended client connections during the upgrade.
•
Reduce cluster locking in pg_upgrade (Bruce Momjian) Specifically, only lock the old cluster if link mode is used, and do it right after the schema is restored.
2145
Appendix E. Release Notes E.44.3.13.2. pg_stat_statements •
Allow pg_stat_statements to aggregate similar queries via SQL text normalization (Peter Geoghegan, Tom Lane) Users with applications that use non-parameterized SQL will now be able to monitor query performance without detailed log analysis.
•
Add dirtied and written block counts and read/write times to pg_stat_statements (Robert Haas, Ants Aasma)
•
Prevent pg_stat_statements from double-counting PREPARE and EXECUTE commands (Tom Lane)
E.44.3.13.3. sepgsql •
Support SECURITY LABEL on global objects (KaiGai Kohei, Robert Haas) Specifically, add security labels to databases, tablespaces, and roles.
•
Allow sepgsql to honor database labels (KaiGai Kohei)
•
Perform sepgsql permission checks during the creation of various objects (KaiGai Kohei)
•
Add sepgsql_setcon() and related functions to control the sepgsql security domain (KaiGai Kohei)
•
Add a user space access cache to sepgsql to improve performance (KaiGai Kohei)
E.44.3.14. Documentation •
Add a rule to optionally build HTML documentation using the stylesheet from the website (Magnus Hagander) Use gmake STYLE=website draft.
•
Improve EXPLAIN documentation (Tom Lane)
•
Document that user/database names are preserved with double-quoting by command-line tools like vacuumdb (Bruce Momjian)
•
Document the actual string returned by the client for MD5 authentication (Cyan Ogilvie)
•
Deprecate use of GLOBAL and LOCAL in CREATE TEMP TABLE (Noah Misch) PostgreSQL has long treated these keyword as no-ops, and continues to do so; but in future they might mean what the SQL standard says they mean, so applications should avoid using them.
E.45. Release 9.1.24 Release date: 2016-10-27 This release contains a variety of fixes from 9.1.23. For information about new features in the 9.1 major release, see Section E.69.
2146
Appendix E. Release Notes This is expected to be the last PostgreSQL release in the 9.1.X series. Users are encouraged to update to a newer release branch soon.
E.45.1. Migration to Version 9.1.24 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.45.2. Changes •
Fix EvalPlanQual rechecks involving CTE scans (Tom Lane) The recheck would always see the CTE as returning no rows, typically leading to failure to update rows that were recently updated.
•
Fix improper repetition of previous results from hashed aggregation in a subquery (Andrew Gierth) The test to see if we can reuse a previously-computed hash table of the aggregate state values neglected the possibility of an outer query reference appearing in an aggregate argument expression. A change in the value of such a reference should lead to recalculating the hash table, but did not.
•
Fix timeout length when VACUUM is waiting for exclusive table lock so that it can truncate the table (Simon Riggs) The timeout was meant to be 50 milliseconds, but it was actually only 50 microseconds, causing VACUUM to give up on truncation much more easily than intended. Set it to the intended value.
•
Remove artificial restrictions on the values accepted by numeric_in() and numeric_recv() (Tom Lane) We allow numeric values up to the limit of the storage format (more than 1e100000), so it seems fairly pointless that numeric_in() rejected scientific-notation exponents above 1000. Likewise, it was silly for numeric_recv() to reject more than 1000 digits in an input value.
•
Avoid very-low-probability data corruption due to testing tuple visibility without holding buffer lock (Thomas Munro, Peter Geoghegan, Tom Lane)
•
Fix file descriptor leakage when truncating a temporary relation of more than 1GB (Andres Freund)
•
Disallow starting a standalone backend with standby_mode turned on (Michael Paquier) This can’t do anything useful, since there will be no WAL receiver process to fetch more WAL data; and it could result in misbehavior in code that wasn’t designed with this situation in mind.
•
Don’t try to share SSL contexts across multiple connections in libpq (Heikki Linnakangas) This led to assorted corner-case bugs, particularly when trying to use different SSL parameters for different connections.
•
Avoid corner-case memory leak in libpq (Tom Lane) The reported problem involved leaking an error report during PQreset(), but there might be related cases.
•
Make ecpg’s --help and --version options work consistently with our other executables (Haribabu Kommi)
•
Fix contrib/intarray/bench/bench.pl to print the results of the EXPLAIN it does when given the -e option (Daniel Gustafsson)
2147
Appendix E. Release Notes •
Prevent failure of obsolete dynamic time zone abbreviations (Tom Lane) If a dynamic time zone abbreviation does not match any entry in the referenced time zone, treat it as equivalent to the time zone name. This avoids unexpected failures when IANA removes abbreviations from their time zone database, as they did in tzdata release 2016f and seem likely to do again in the future. The consequences were not limited to not recognizing the individual abbreviation; any mismatch caused the pg_timezone_abbrevs view to fail altogether.
•
Update time zone data files to tzdata release 2016h for DST law changes in Palestine and Turkey, plus historical corrections for Turkey and some regions of Russia. Switch to numeric abbreviations for some time zones in Antarctica, the former Soviet Union, and Sri Lanka. The IANA time zone database previously provided textual abbreviations for all time zones, sometimes making up abbreviations that have little or no currency among the local population. They are in process of reversing that policy in favor of using numeric UTC offsets in zones where there is no evidence of real-world use of an English abbreviation. At least for the time being, PostgreSQL will continue to accept such removed abbreviations for timestamp input. But they will not be shown in the pg_timezone_names view nor used for output. In this update, AMT is no longer shown as being in use to mean Armenia Time. Therefore, we have changed the Default abbreviation set to interpret it as Amazon Time, thus UTC-4 not UTC+4.
E.46. Release 9.1.23 Release date: 2016-08-11 This release contains a variety of fixes from 9.1.22. For information about new features in the 9.1 major release, see Section E.69. The PostgreSQL community will stop releasing updates for the 9.1.X release series in September 2016. Users are encouraged to update to a newer release branch soon.
E.46.1. Migration to Version 9.1.23 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.46.2. Changes •
Fix possible mis-evaluation of nested CASE-WHEN expressions (Heikki Linnakangas, Michael Paquier, Tom Lane) A CASE expression appearing within the test value subexpression of another CASE could become confused about whether its own test value was null or not. Also, inlining of a SQL function implementing the equality operator used by a CASE expression could result in passing the wrong test value to functions called within a CASE expression in the SQL function’s body. If the test values were of different data types, a crash might result; moreover such situations could be abused to allow disclosure of portions of server memory. (CVE-2016-5423)
•
Fix client programs’ handling of special characters in database and role names (Noah Misch, Nathan Bossart, Michael Paquier)
2148
Appendix E. Release Notes Numerous places in vacuumdb and other client programs could become confused by database and role names containing double quotes or backslashes. Tighten up quoting rules to make that safe. Also, ensure that when a conninfo string is used as a database name parameter to these programs, it is correctly treated as such throughout. Fix handling of paired double quotes in psql’s \connect and \password commands to match the documentation. Introduce a new -reuse-previous option in psql’s \connect command to allow explicit control of whether to re-use connection parameters from a previous connection. (Without this, the choice is based on whether the database name looks like a conninfo string, as before.) This allows secure handling of database names containing special characters in pg_dumpall scripts. pg_dumpall now refuses to deal with database and role names containing carriage returns or newlines, as it seems impractical to quote those characters safely on Windows. In future we may reject such names on the server side, but that step has not been taken yet. These are considered security fixes because crafted object names containing special characters could have been used to execute commands with superuser privileges the next time a superuser executes pg_dumpall or other routine maintenance operations. (CVE-2016-5424) •
Fix corner-case misbehaviors for IS NULL/IS NOT NULL applied to nested composite values (Andrew Gierth, Tom Lane) The SQL standard specifies that IS NULL should return TRUE for a row of all null values (thus ROW(NULL,NULL) IS NULL yields TRUE), but this is not meant to apply recursively (thus ROW(NULL, ROW(NULL,NULL)) IS NULL yields FALSE). The core executor got this right, but certain planner optimizations treated the test as recursive (thus producing TRUE in both cases), and contrib/postgres_fdw could produce remote queries that misbehaved similarly.
•
Make the inet and cidr data types properly reject IPv6 addresses with too many colon-separated fields (Tom Lane)
•
Prevent crash in close_ps() (the point ## lseg operator) for NaN input coordinates (Tom Lane) Make it return NULL instead of crashing.
•
Fix several one-byte buffer over-reads in to_number() (Peter Eisentraut) In several cases the to_number() function would read one more character than it should from the input string. There is a small chance of a crash, if the input happens to be adjacent to the end of memory.
•
Avoid unsafe intermediate state during expensive paths through heap_update() (Masahiko Sawada, Andres Freund) Previously, these cases locked the target tuple (by setting its XMAX) but did not WAL-log that action, thus risking data integrity problems if the page were spilled to disk and then a database crash occurred before the tuple update could be completed.
•
Avoid consuming a transaction ID during VACUUM (Alexander Korotkov) Some cases in VACUUM unnecessarily caused an XID to be assigned to the current transaction. Normally this is negligible, but if one is up against the XID wraparound limit, consuming more XIDs during anti-wraparound vacuums is a very bad thing.
•
Avoid canceling hot-standby queries during VACUUM FREEZE (Simon Riggs, Álvaro Herrera) VACUUM FREEZE on an otherwise-idle master server could result in unnecessary cancellations of
queries on its standby servers.
2149
Appendix E. Release Notes •
When
a manual ANALYZE specifies a changes_since_analyze counter (Tom Lane)
column
list,
don’t
reset
the
table’s
If we’re only analyzing some columns, we should not prevent routine auto-analyze from happening for the other columns. •
Fix ANALYZE’s overestimation of n_distinct for a unique or nearly-unique column with many null entries (Tom Lane) The nulls could get counted as though they were themselves distinct values, leading to serious planner misestimates in some types of queries.
•
Prevent autovacuum from starting multiple workers for the same shared catalog (Álvaro Herrera) Normally this isn’t much of a problem because the vacuum doesn’t take long anyway; but in the case of a severely bloated catalog, it could result in all but one worker uselessly waiting instead of doing useful work on other tables.
•
Fix contrib/btree_gin to handle the smallest possible bigint value correctly (Peter Eisentraut)
•
Teach libpq to correctly decode server version from future servers (Peter Eisentraut) It’s planned to switch to two-part instead of three-part server version numbers for releases after 9.6. Make sure that PQserverVersion() returns the correct value for such cases.
•
Fix ecpg’s code for unsigned long long array elements (Michael Meskes)
•
Make pg_basebackup accept -Z 0 as specifying no compression (Fujii Masao)
•
Revert to the old heuristic timeout for pg_ctl start -w (Tom Lane) The new method adopted as of release 9.1.20 does not work when silent_mode is enabled, so go back to the old way.
•
Fix makefiles’ rule for building AIX shared libraries to be safe for parallel make (Noah Misch)
•
Fix TAP tests and MSVC scripts to work when build directory’s path name contains spaces (Michael Paquier, Kyotaro Horiguchi)
•
Make regression tests safe for Danish and Welsh locales (Jeff Janes, Tom Lane) Change some test data that triggered the unusual sorting rules of these locales.
•
Update our copy of the timezone code to match IANA’s tzcode release 2016c (Tom Lane) This is needed to cope with anticipated future changes in the time zone data files. It also fixes some corner-case bugs in coping with unusual time zones.
•
Update time zone data files to tzdata release 2016f for DST law changes in Kemerovo and Novosibirsk, plus historical corrections for Azerbaijan, Belarus, and Morocco.
E.47. Release 9.1.22 Release date: 2016-05-12 This release contains a variety of fixes from 9.1.21. For information about new features in the 9.1 major release, see Section E.69. The PostgreSQL community will stop releasing updates for the 9.1.X release series in September 2016. Users are encouraged to update to a newer release branch soon.
2150
Appendix E. Release Notes
E.47.1. Migration to Version 9.1.22 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.47.2. Changes •
Clear the OpenSSL error queue before OpenSSL calls, rather than assuming it’s clear already; and make sure we leave it clear afterwards (Peter Geoghegan, Dave Vitek, Peter Eisentraut) This change prevents problems when there are multiple connections using OpenSSL within a single process and not all the code involved follows the same rules for when to clear the error queue. Failures have been reported specifically when a client application uses SSL connections in libpq concurrently with SSL connections using the PHP, Python, or Ruby wrappers for OpenSSL. It’s possible for similar problems to arise within the server as well, if an extension module establishes an outgoing SSL connection.
•
Fix “failed to build any N -way joins” planner error with a full join enclosed in the right-hand side of a left join (Tom Lane)
•
Fix possible misbehavior of TH, th, and Y,YYY format codes in to_timestamp() (Tom Lane) These could advance off the end of the input string, causing subsequent format codes to read garbage.
•
Fix dumping of rules and views in which the array argument of a value operator ANY (array ) construct is a sub-SELECT (Tom Lane)
•
Make pg_regress use a startup timeout from the PGCTLTIMEOUT environment variable, if that’s set (Tom Lane) This is for consistency with a behavior recently added to pg_ctl; it eases automated testing on slow machines.
•
Fix pg_upgrade to correctly restore extension membership for operator families containing only one operator class (Tom Lane) In such a case, the operator family was restored into the new database, but it was no longer marked as part of the extension. This had no immediate ill effects, but would cause later pg_dump runs to emit output that would cause (harmless) errors on restore.
•
Rename internal function strtoi() to strtoint() to avoid conflict with a NetBSD library function (Thomas Munro)
•
Fix reporting of errors from bind() and listen() system calls on Windows (Tom Lane)
•
Reduce verbosity of compiler output when building with Microsoft Visual Studio (Christian Ullrich)
•
Avoid possibly-unsafe use of Windows’ FormatMessage() function (Christian Ullrich) Use the FORMAT_MESSAGE_IGNORE_INSERTS flag where appropriate. No live bug is known to exist here, but it seems like a good idea to be careful.
•
Update time zone data files to tzdata release 2016d for DST law changes in Russia and Venezuela. There are new zone names Europe/Kirov and Asia/Tomsk to reflect the fact that these regions now have different time zone histories from adjacent regions.
2151
Appendix E. Release Notes
E.48. Release 9.1.21 Release date: 2016-03-31 This release contains a variety of fixes from 9.1.20. For information about new features in the 9.1 major release, see Section E.69.
E.48.1. Migration to Version 9.1.21 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.48.2. Changes •
Fix incorrect handling of NULL index entries in indexed ROW() comparisons (Tom Lane) An index search using a row comparison such as ROW(a, b) > ROW(’x’, ’y’) would stop upon reaching a NULL entry in the b column, ignoring the fact that there might be non-NULL b values associated with later values of a.
•
Avoid unlikely data-loss scenarios due to renaming files without adequate fsync() calls before and after (Michael Paquier, Tomas Vondra, Andres Freund)
•
Correctly handle cases where pg_subtrans is close to XID wraparound during server startup (Jeff Janes)
•
Fix corner-case crash due to trying to free localeconv() output strings more than once (Tom Lane)
•
Fix parsing of affix files for ispell dictionaries (Tom Lane) The code could go wrong if the affix file contained any characters whose byte length changes during case-folding, for example I in Turkish UTF8 locales.
•
Avoid use of sscanf() to parse ispell dictionary files (Artur Zakirov) This dodges a portability problem on FreeBSD-derived platforms (including macOS).
•
Avoid a crash on old Windows versions (before 7SP1/2008R2SP1) with an AVX2-capable CPU and a Postgres build done with Visual Studio 2013 (Christian Ullrich) This is a workaround for a bug in Visual Studio 2013’s runtime library, which Microsoft have stated they will not fix in that version.
•
Fix psql’s tab completion logic to handle multibyte characters properly (Kyotaro Horiguchi, Robert Haas)
•
Fix psql’s tab completion for SECURITY LABEL (Tom Lane) Pressing TAB after SECURITY LABEL might cause a crash or offering of inappropriate keywords.
•
Make pg_ctl accept a wait timeout from the PGCTLTIMEOUT environment variable, if none is specified on the command line (Noah Misch) This eases testing of slower buildfarm members by allowing them to globally specify a longer-thannormal timeout for postmaster startup and shutdown.
•
Fix incorrect test for Windows service status in pg_ctl (Manuel Mathar)
2152
Appendix E. Release Notes The previous set of minor releases attempted to fix pg_ctl to properly determine whether to send log messages to Window’s Event Log, but got the test backwards. •
Fix pgbench to correctly handle the combination of -C and -M prepared options (Tom Lane)
•
In PL/Perl, properly translate empty Postgres arrays into empty Perl arrays (Alex Hunsaker)
•
Make PL/Python cope with function names that aren’t valid Python identifiers (Jim Nasby)
•
Fix multiple mistakes in the statistics returned by contrib/pgstattuple’s pgstatindex() function (Tom Lane)
•
Remove dependency on psed in MSVC builds, since it’s no longer provided by core Perl (Michael Paquier, Andrew Dunstan)
•
Update time zone data files to tzdata release 2016c for DST law changes in Azerbaijan, Chile, Haiti, Palestine, and Russia (Altai, Astrakhan, Kirov, Sakhalin, Ulyanovsk regions), plus historical corrections for Lithuania, Moldova, and Russia (Kaliningrad, Samara, Volgograd).
E.49. Release 9.1.20 Release date: 2016-02-11 This release contains a variety of fixes from 9.1.19. For information about new features in the 9.1 major release, see Section E.69.
E.49.1. Migration to Version 9.1.20 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.49.2. Changes •
Fix infinite loops and buffer-overrun problems in regular expressions (Tom Lane) Very large character ranges in bracket expressions could cause infinite loops in some cases, and memory overwrites in other cases. (CVE-2016-0773)
•
Perform an immediate shutdown if the postmaster.pid file is removed (Tom Lane) The postmaster now checks every minute or so that postmaster.pid is still there and still contains its own PID. If not, it performs an immediate shutdown, as though it had received SIGQUIT. The main motivation for this change is to ensure that failed buildfarm runs will get cleaned up without manual intervention; but it also serves to limit the bad effects if a DBA forcibly removes postmaster.pid and then starts a new postmaster.
•
In SERIALIZABLE transaction isolation mode, serialization anomalies could be missed due to race conditions during insertions (Kevin Grittner, Thomas Munro)
•
Fix failure to emit appropriate WAL records when doing ALTER TABLE ... SET TABLESPACE for unlogged relations (Michael Paquier, Andres Freund) Even though the relation’s data is unlogged, the move must be logged or the relation will be inaccessible after a standby is promoted to master.
2153
Appendix E. Release Notes •
Fix possible misinitialization of unlogged relations at the end of crash recovery (Andres Freund, Michael Paquier)
•
Fix ALTER COLUMN TYPE to reconstruct inherited check constraints properly (Tom Lane)
•
Fix REASSIGN OWNED to change ownership of composite types properly (Álvaro Herrera)
•
Fix REASSIGN OWNED and ALTER OWNER to correctly update granted-permissions lists when changing owners of data types, foreign data wrappers, or foreign servers (Bruce Momjian, Álvaro Herrera)
•
Fix REASSIGN OWNED to ignore foreign user mappings, rather than fail (Álvaro Herrera)
•
Add more defenses against bad planner cost estimates for GIN index scans when the index’s internal statistics are very out-of-date (Tom Lane)
•
Make planner cope with hypothetical GIN indexes suggested by an index advisor plug-in (Julien Rouhaud)
•
Fix dumping of whole-row Vars in ROW() and VALUES() lists (Tom Lane)
•
Fix possible internal overflow in numeric division (Dean Rasheed)
•
Fix enforcement of restrictions inside parentheses within regular expression lookahead constraints (Tom Lane) Lookahead constraints aren’t allowed to contain backrefs, and parentheses within them are always considered non-capturing, according to the manual. However, the code failed to handle these cases properly inside a parenthesized subexpression, and would give unexpected results.
•
Conversion of regular expressions to indexscan bounds could produce incorrect bounds from regexps containing lookahead constraints (Tom Lane)
•
Fix regular-expression compiler to handle loops of constraint arcs (Tom Lane) The code added for CVE-2007-4772 was both incomplete, in that it didn’t handle loops involving more than one state, and incorrect, in that it could cause assertion failures (though there seem to be no bad consequences of that in a non-assert build). Multi-state loops would cause the compiler to run until the query was canceled or it reached the too-many-states error condition.
•
Improve memory-usage accounting in regular-expression compiler (Tom Lane) This causes the code to emit “regular expression is too complex” errors in some cases that previously used unreasonable amounts of time and memory.
•
Improve performance of regular-expression compiler (Tom Lane)
•
Make %h and %r escapes in log_line_prefix work for messages emitted due to log_connections (Tom Lane) Previously, %h/%r started to work just after a new session had emitted the “connection received” log message; now they work for that message too.
•
On Windows, ensure the shared-memory mapping handle gets closed in child processes that don’t need it (Tom Lane, Amit Kapila) This oversight resulted in failure to recover from crashes whenever logging_collector is turned on.
•
Fix possible failure to detect socket EOF in non-blocking mode on Windows (Tom Lane) It’s not entirely clear whether this problem can happen in pre-9.5 branches, but if it did, the symptom would be that a walsender process would wait indefinitely rather than noticing a loss of connection.
•
Avoid leaking a token handle during SSPI authentication (Christian Ullrich)
2154
Appendix E. Release Notes •
In psql, ensure that libreadline’s idea of the screen size is updated when the terminal window size changes (Merlin Moncure) Previously, libreadline did not notice if the window was resized during query output, leading to strange behavior during later input of multiline queries.
•
Fix psql’s \det command to interpret its pattern argument the same way as other \d commands with potentially schema-qualified patterns do (Reece Hart)
•
Avoid possible crash in psql’s \c command when previous connection was via Unix socket and command specifies a new hostname and same username (Tom Lane)
•
In pg_ctl start -w, test child process status directly rather than relying on heuristics (Tom Lane, Michael Paquier) Previously, pg_ctl relied on an assumption that the new postmaster would always create postmaster.pid within five seconds. But that can fail on heavily-loaded systems, causing pg_ctl to report incorrectly that the postmaster failed to start. Except on Windows, this change also means that a pg_ctl start -w done immediately after another such command will now reliably fail, whereas previously it would report success if done within two seconds of the first command.
•
In pg_ctl start -w, don’t attempt to use a wildcard listen address to connect to the postmaster (Kondo Yuta) On Windows, pg_ctl would fail to detect postmaster startup if listen_addresses is set to 0.0.0.0 or ::, because it would try to use that value verbatim as the address to connect to, which doesn’t work. Instead assume that 127.0.0.1 or ::1, respectively, is the right thing to use.
•
In pg_ctl on Windows, check service status to decide where to send output, rather than checking if standard output is a terminal (Michael Paquier)
•
In pg_dump and pg_basebackup, adopt the GNU convention for handling tar-archive members exceeding 8GB (Tom Lane) The POSIX standard for tar file format does not allow archive member files to exceed 8GB, but most modern implementations of tar support an extension that fixes that. Adopt this extension so that pg_dump with -Ft no longer fails on tables with more than 8GB of data, and so that pg_basebackup can handle files larger than 8GB. In addition, fix some portability issues that could cause failures for members between 4GB and 8GB on some platforms. Potentially these problems could cause unrecoverable data loss due to unreadable backup files.
•
Fix assorted corner-case bugs in pg_dump’s processing of extension member objects (Tom Lane)
•
Make pg_dump mark a view’s triggers as needing to be processed after its rule, to prevent possible failure during parallel pg_restore (Tom Lane)
•
Ensure that relation option values are properly quoted in pg_dump (Kouhei Sutou, Tom Lane) A reloption value that isn’t a simple identifier or number could lead to dump/reload failures due to syntax errors in CREATE statements issued by pg_dump. This is not an issue with any reloption currently supported by core PostgreSQL, but extensions could allow reloptions that cause the problem.
•
Fix pg_upgrade’s file-copying code to handle errors properly on Windows (Bruce Momjian)
•
Install guards in pgbench against corner-case overflow conditions during evaluation of scriptspecified division or modulo operators (Fabien Coelho, Michael Paquier)
•
Prevent certain PL/Java parameters from being set by non-superusers (Noah Misch)
2155
Appendix E. Release Notes This change mitigates a PL/Java security bug (CVE-2016-0766), which was fixed in PL/Java by marking these parameters as superuser-only. To fix the security hazard for sites that update PostgreSQL more frequently than PL/Java, make the core code aware of them also. •
Improve libpq’s handling of out-of-memory situations (Michael Paquier, Amit Kapila, Heikki Linnakangas)
•
Fix order of arguments in ecpg-generated typedef statements (Michael Meskes)
•
Use %g not %f format in ecpg’s PGTYPESnumeric_from_double() (Tom Lane)
•
Fix ecpg-supplied header files to not contain comments continued from a preprocessor directive line onto the next line (Michael Meskes) Such a comment is rejected by ecpg. It’s not yet clear whether ecpg itself should be changed.
•
Ensure that contrib/pgcrypto’s crypt() function can be interrupted by query cancel (Andreas Karlsson)
•
Accept flex versions later than 2.5.x (Tom Lane, Michael Paquier) Now that flex 2.6.0 has been released, the version checks in our build scripts needed to be adjusted.
•
Install our missing script where PGXS builds can find it (Jim Nasby) This allows sane behavior in a PGXS build done on a machine where build tools such as bison are missing.
•
Ensure that dynloader.h is included in the installed header files in MSVC builds (Bruce Momjian, Michael Paquier)
•
Add variant regression test expected-output file to match behavior of current libxml2 (Tom Lane) The fix for libxml2’s CVE-2015-7499 causes it not to output error context reports in some cases where it used to do so. This seems to be a bug, but we’ll probably have to live with it for some time, so work around it.
•
Update time zone data files to tzdata release 2016a for DST law changes in Cayman Islands, Metlakatla, and Trans-Baikal Territory (Zabaykalsky Krai), plus historical corrections for Pakistan.
E.50. Release 9.1.19 Release date: 2015-10-08 This release contains a variety of fixes from 9.1.18. For information about new features in the 9.1 major release, see Section E.69.
E.50.1. Migration to Version 9.1.19 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.50.2. Changes •
Fix contrib/pgcrypto to detect and report too-short crypt() salts (Josh Kupershmidt)
2156
Appendix E. Release Notes Certain invalid salt arguments crashed the server or disclosed a few bytes of server memory. We have not ruled out the viability of attacks that arrange for presence of confidential information in the disclosed bytes, but they seem unlikely. (CVE-2015-5288) •
Fix subtransaction cleanup after a portal (cursor) belonging to an outer subtransaction fails (Tom Lane, Michael Paquier) A function executed in an outer-subtransaction cursor could cause an assertion failure or crash by referencing a relation created within an inner subtransaction.
•
Fix insertion of relations into the relation cache “init file” (Tom Lane) An
oversight
in
a
patch
in
the
most
recent
minor
releases
caused
pg_trigger_tgrelid_tgname_index to be omitted from the init file. Subsequent sessions
detected this, then deemed the init file to be broken and silently ignored it, resulting in a significant degradation in session startup time. In addition to fixing the bug, install some guards so that any similar future mistake will be more obvious. •
Avoid O(N^2) behavior when inserting many tuples into a SPI query result (Neil Conway)
•
Improve LISTEN startup time when there are many unread notifications (Matt Newell)
•
Back-patch 9.3-era addition of per-resource-owner lock caches (Jeff Janes) This substantially improves performance when pg_dump tries to dump a large number of tables.
•
Disable SSL renegotiation by default (Michael Paquier, Andres Freund) While use of SSL renegotiation is a good idea in theory, we have seen too many bugs in practice, both in the underlying OpenSSL library and in our usage of it. Renegotiation will be removed entirely in 9.5 and later. In the older branches, just change the default value of ssl_renegotiation_limit to zero (disabled).
•
Lower the minimum values of the *_freeze_max_age parameters (Andres Freund) This is mainly to make tests of related behavior less time-consuming, but it may also be of value for installations with limited disk space.
•
Limit the maximum value of wal_buffers to 2GB to avoid server crashes (Josh Berkus)
•
Fix rare internal overflow in multiplication of numeric values (Dean Rasheed)
•
Guard against hard-to-reach stack overflows involving record types, range types, json, jsonb, tsquery, ltxtquery and query_int (Noah Misch)
•
Fix handling of DOW and DOY in datetime input (Greg Stark) These tokens aren’t meant to be used in datetime values, but previously they resulted in opaque internal error messages rather than “invalid input syntax”.
•
Add more query-cancel checks to regular expression matching (Tom Lane)
•
Add recursion depth protections to regular expression, SIMILAR TO, and LIKE matching (Tom Lane) Suitable search patterns and a low stack depth limit could lead to stack-overrun crashes.
•
Fix potential infinite loop in regular expression execution (Tom Lane) A search pattern that can apparently match a zero-length string, but actually doesn’t match because of a back reference, could lead to an infinite loop.
•
Fix low-memory failures in regular expression compilation (Andreas Seltenreich)
•
Fix low-probability memory leak during regular expression execution (Tom Lane)
•
Fix rare low-memory failure in lock cleanup during transaction abort (Tom Lane)
2157
Appendix E. Release Notes •
Fix “unexpected out-of-memory situation during sort” errors when using tuplestores with small work_mem settings (Tom Lane)
•
Fix very-low-probability stack overrun in qsort (Tom Lane)
•
Fix “invalid memory alloc request size” failure in hash joins with large work_mem settings (Tomas Vondra, Tom Lane)
•
Fix assorted planner bugs (Tom Lane) These mistakes could lead to incorrect query plans that would give wrong answers, or to assertion failures in assert-enabled builds, or to odd planner errors such as “could not devise a query plan for the given query”, “could not find pathkey item to sort”, “plan should not reference subplan’s variable”, or “failed to assign all NestLoopParams to plan nodes”. Thanks are due to Andreas Seltenreich and Piotr Stefaniak for fuzz testing that exposed these problems.
•
Use fuzzy path cost tiebreaking rule in all supported branches (Tom Lane) This change is meant to avoid platform-specific behavior when alternative plan choices have effectively-identical estimated costs.
•
Ensure standby promotion trigger files are removed at postmaster startup (Michael Paquier, Fujii Masao) This prevents unwanted promotion from occurring if these files appear in a database backup that is used to initialize a new standby server.
•
During postmaster shutdown, ensure that per-socket lock files are removed and listen sockets are closed before we remove the postmaster.pid file (Tom Lane) This avoids race-condition failures if an external script attempts to start a new postmaster as soon as pg_ctl stop returns.
•
Fix postmaster’s handling of a startup-process crash during crash recovery (Tom Lane) If, during a crash recovery cycle, the startup process crashes without having restored database consistency, we’d try to launch a new startup process, which typically would just crash again, leading to an infinite loop.
•
Do not print a WARNING when an autovacuum worker is already gone when we attempt to signal it, and reduce log verbosity for such signals (Tom Lane)
•
Prevent autovacuum launcher from sleeping unduly long if the server clock is moved backwards a large amount (Álvaro Herrera)
•
Ensure that cleanup of a GIN index’s pending-insertions list is interruptable by cancel requests (Jeff Janes)
•
Allow all-zeroes pages in GIN indexes to be reused (Heikki Linnakangas) Such a page might be left behind after a crash.
•
Fix off-by-one error that led to otherwise-harmless warnings about “apparent wraparound” in subtrans/multixact truncation (Thomas Munro)
•
Fix misreporting of CONTINUE and MOVE statement types in PL/pgSQL’s error context messages (Pavel Stehule, Tom Lane)
Fix PL/Python crash when returning the string representation of a record result (Tom Lane)
•
Fix some places in PL/Tcl that neglected to check for failure of malloc() calls (Michael Paquier, Álvaro Herrera)
•
In contrib/isn, fix output of ISBN-13 numbers that begin with 979 (Fabien Coelho)
2158
Appendix E. Release Notes EANs beginning with 979 (but not 9790) are considered ISBNs, but they must be printed in the new 13-digit format, not the 10-digit format. •
Improve libpq’s handling of out-of-memory conditions (Michael Paquier, Heikki Linnakangas)
•
Fix memory leaks and missing out-of-memory checks in ecpg (Michael Paquier)
•
Fix psql’s code for locale-aware formatting of numeric output (Tom Lane) The formatting code invoked by \pset numericlocale on did the wrong thing for some uncommon cases such as numbers with an exponent but no decimal point. It could also mangle already-localized output from the money data type.
•
Prevent crash in psql’s \c command when there is no current connection (Noah Misch)
•
Fix selection of default zlib compression level in pg_dump’s directory output format (Andrew Dunstan)
•
Ensure that temporary files created during a pg_dump run with tar-format output are not worldreadable (Michael Paquier)
•
Fix pg_dump and pg_upgrade to support cases where the postgres or template1 database is in a non-default tablespace (Marti Raudsepp, Bruce Momjian)
•
Fix pg_dump to handle object privileges sanely when dumping from a server too old to have a particular privilege type (Tom Lane) When dumping functions or procedural languages from pre-7.3 servers, pg_dump would produce GRANT/REVOKE commands that revoked the owner’s grantable privileges and instead granted all privileges to PUBLIC. Since the privileges involved are just USAGE and EXECUTE, this isn’t a security problem, but it’s certainly a surprising representation of the older systems’ behavior. Fix it to leave the default privilege state alone in these cases.
•
Fix pg_dump to dump shell types (Tom Lane) Shell types (that is, not-yet-fully-defined types) aren’t useful for much, but nonetheless pg_dump should dump them.
•
Fix assorted minor memory leaks in pg_dump and other client-side programs (Michael Paquier)
•
Fix spinlock assembly code for PPC hardware to be compatible with AIX’s native assembler (Tom Lane) Building with gcc didn’t work if gcc had been configured to use the native assembler, which is becoming more common.
•
On AIX, test the -qlonglong compiler option rather than just assuming it’s safe to use (Noah Misch)
•
On AIX, use -Wl,-brtllib link option to allow symbols to be resolved at runtime (Noah Misch) Perl relies on this ability in 5.8.0 and later.
•
Avoid use of inline functions when compiling with 32-bit xlc, due to compiler bugs (Noah Misch)
•
Use librt for sched_yield() when necessary, which it is on some Solaris versions (Oskari Saarenmaa)
•
Fix Windows install.bat script to handle target directory names that contain spaces (Heikki Linnakangas)
•
Make the numeric form of the PostgreSQL version number (e.g., 90405) readily available to extension Makefiles, as a variable named VERSION_NUM (Michael Paquier)
2159
Appendix E. Release Notes •
Update time zone data files to tzdata release 2015g for DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, and Uruguay. There is a new zone name America/Fort_Nelson for the Canadian Northern Rockies.
E.51. Release 9.1.18 Release date: 2015-06-12 This release contains a small number of fixes from 9.1.17. For information about new features in the 9.1 major release, see Section E.69.
E.51.1. Migration to Version 9.1.18 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
E.51.2. Changes •
Fix rare failure to invalidate relation cache init file (Tom Lane) With just the wrong timing of concurrent activity, a VACUUM FULL on a system catalog might fail to update the “init file” that’s used to avoid cache-loading work for new sessions. This would result in later sessions being unable to access that catalog at all. This is a very ancient bug, but it’s so hard to trigger that no reproducible case had been seen until recently.
•
Avoid deadlock between incoming sessions and CREATE/DROP DATABASE (Tom Lane) A new session starting in a database that is the target of a DROP DATABASE command, or is the template for a CREATE DATABASE command, could cause the command to wait for five seconds and then fail, even if the new session would have exited before that.
E.52. Release 9.1.17 Release date: 2015-06-04 This release contains a small number of fixes from 9.1.16. For information about new features in the 9.1 major release, see Section E.69.
E.52.1. Migration to Version 9.1.17 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.16, see Section E.53.
2160
Appendix E. Release Notes
E.52.2. Changes •
Avoid failures while fsync’ing data directory during crash restart (Abhijit Menon-Sen, Tom Lane) In the previous minor releases we added a patch to fsync everything in the data directory after a crash. Unfortunately its response to any error condition was to fail, thereby preventing the server from starting up, even when the problem was quite harmless. An example is that an unwritable file in the data directory would prevent restart on some platforms; but it is common to make SSL certificate files unwritable by the server. Revise this behavior so that permissions failures are ignored altogether, and other types of failures are logged but do not prevent continuing.
•
Remove configure’s check prohibiting linking to a threaded libpython on OpenBSD (Tom Lane) The failure this restriction was meant to prevent seems to not be a problem anymore on current OpenBSD versions.
•
Allow libpq to use TLS protocol versions beyond v1 (Noah Misch) For a long time, libpq was coded so that the only SSL protocol it would allow was TLS v1. Now that newer TLS versions are becoming popular, allow it to negotiate the highest commonly-supported TLS version with the server. (PostgreSQL servers were already capable of such negotiation, so no change is needed on the server side.) This is a back-patch of a change already released in 9.4.0.
E.53. Release 9.1.16 Release date: 2015-05-22 This release contains a variety of fixes from 9.1.15. For information about new features in the 9.1 major release, see Section E.69.
E.53.1. Migration to Version 9.1.16 A dump/restore is not required for those running 9.1.X. However, if you use contrib/citext’s regexp_matches() functions, see the changelog entry below about that. Also, if you are upgrading from a version earlier than 9.1.14, see Section E.55.
E.53.2. Changes •
Avoid possible crash when client disconnects just before the authentication timeout expires (Benkocs Norbert Attila) If the timeout interrupt fired partway through the session shutdown sequence, SSL-related state would be freed twice, typically causing a crash and hence denial of service to other sessions. Experimentation shows that an unauthenticated remote attacker could trigger the bug somewhat consistently, hence treat as security issue. (CVE-2015-3165)
•
Improve detection of system-call failures (Noah Misch) Our replacement implementation of snprintf() failed to check for errors reported by the underlying system library calls; the main case that might be missed is out-of-memory situations. In the
2161
Appendix E. Release Notes worst case this might lead to information exposure, due to our code assuming that a buffer had been overwritten when it hadn’t been. Also, there were a few places in which security-relevant calls of other system library functions did not check for failure. It remains possible that some calls of the *printf() family of functions are vulnerable to information disclosure if an out-of-memory error occurs at just the wrong time. We judge the risk to not be large, but will continue analysis in this area. (CVE-2015-3166) •
In contrib/pgcrypto, uniformly report decryption failures as “Wrong key or corrupt data” (Noah Misch) Previously, some cases of decryption with an incorrect key could report other error message texts. It has been shown that such variance in error reports can aid attackers in recovering keys from other systems. While it’s unknown whether pgcrypto’s specific behaviors are likewise exploitable, it seems better to avoid the risk by using a one-size-fits-all message. (CVE-2015-3167)
•
Fix incorrect declaration of contrib/citext’s regexp_matches() functions (Tom Lane) These functions should return setof text[], like the core functions they are wrappers for; but they were incorrectly declared as returning just text[]. This mistake had two results: first, if there was no match you got a scalar null result, whereas what you should get is an empty set (zero rows). Second, the g flag was effectively ignored, since you would get only one result array even if there were multiple matches. While the latter behavior is clearly a bug, there might be applications depending on the former behavior; therefore the function declarations will not be changed by default until PostgreSQL 9.5. In pre-9.5 branches, the old behavior exists in version 1.0 of the citext extension, while we have provided corrected declarations in version 1.1 (which is not installed by default). To adopt the fix in pre-9.5 branches, execute ALTER EXTENSION citext UPDATE TO ’1.1’ in each database in which citext is installed. (You can also “update” back to 1.0 if you need to undo that.) Be aware that either update direction will require dropping and recreating any views or rules that use citext’s regexp_matches() functions.
•
Fix incorrect checking of deferred exclusion constraints after a HOT update (Tom Lane) If a new row that potentially violates a deferred exclusion constraint is HOT-updated (that is, no indexed columns change and the row can be stored back onto the same table page) later in the same transaction, the exclusion constraint would be reported as violated when the check finally occurred, even if the row(s) the new row originally conflicted with had been deleted.
•
Prevent improper reordering of antijoins (NOT EXISTS joins) versus other outer joins (Tom Lane) This oversight in the planner has been observed to cause “could not find RelOptInfo for given relids” errors, but it seems possible that sometimes an incorrect query plan might get past that consistency check and result in silently-wrong query output.
•
Fix incorrect matching of subexpressions in outer-join plan nodes (Tom Lane) Previously, if textually identical non-strict subexpressions were used both above and below an outer join, the planner might try to re-use the value computed below the join, which would be incorrect because the executor would force the value to NULL in case of an unmatched outer row.
•
Fix GEQO planner to cope with failure of its join order heuristic (Tom Lane) This oversight has been seen to lead to “failed to join all relations together” errors in queries involving LATERAL, and that might happen in other cases as well.
•
Fix possible deadlock at startup when max_prepared_transactions is too small (Heikki Linnakangas)
•
Don’t archive useless preallocated WAL files after a timeline switch (Heikki Linnakangas)
2162
Appendix E. Release Notes •
Avoid “cannot GetMultiXactIdMembers() during recovery” error (Álvaro Herrera)
•
Recursively fsync() the data directory after a crash (Abhijit Menon-Sen, Robert Haas) This ensures consistency if another crash occurs shortly later. (The second crash would have to be a system-level crash, not just a database crash, for there to be a problem.)
•
Fix autovacuum launcher’s possible failure to shut down, if an error occurs after it receives SIGTERM (Álvaro Herrera)
•
Cope with unexpected signals in LockBufferForCleanup() (Andres Freund) This oversight could result in spurious errors about “multiple backends attempting to wait for pincount 1”.
•
Avoid waiting for WAL flush or synchronous replication during commit of a transaction that was read-only so far as the user is concerned (Andres Freund) Previously, a delay could occur at commit in transactions that had written WAL due to HOT page pruning, leading to undesirable effects such as sessions getting stuck at startup if all synchronous replicas are down. Sessions have also been observed to get stuck in catchup interrupt processing when using synchronous replication; this will fix that problem as well.
•
Fix crash when manipulating hash indexes on temporary tables (Heikki Linnakangas)
•
Fix possible failure during hash index bucket split, if other processes are modifying the index concurrently (Tom Lane)
•
Check for interrupts while analyzing index expressions (Jeff Janes) ANALYZE executes index expressions many times; if there are slow functions in such an expression, it’s desirable to be able to cancel the ANALYZE before that loop finishes.
•
Ensure tableoid of a foreign table is reported correctly when a READ COMMITTED recheck occurs after locking rows in SELECT FOR UPDATE, UPDATE, or DELETE (Etsuro Fujita)
•
Add the name of the target server to object description strings for foreign-server user mappings (Álvaro Herrera)
•
Recommend setting include_realm to 1 when using Kerberos/GSSAPI/SSPI authentication (Stephen Frost) Without this, identically-named users from different realms cannot be distinguished. For the moment this is only a documentation change, but it will become the default setting in PostgreSQL 9.5.
•
Remove code for matching IPv4 pg_hba.conf entries to IPv4-in-IPv6 addresses (Tom Lane) This hack was added in 2003 in response to a report that some Linux kernels of the time would report IPv4 connections as having IPv4-in-IPv6 addresses. However, the logic was accidentally broken in 9.0. The lack of any field complaints since then shows that it’s not needed anymore. Now we have reports that the broken code causes crashes on some systems, so let’s just remove it rather than fix it. (Had we chosen to fix it, that would make for a subtle and potentially security-sensitive change in the effective meaning of IPv4 pg_hba.conf entries, which does not seem like a good thing to do in minor releases.)
•
Report WAL flush, not insert, position in IDENTIFY_SYSTEM replication command (Heikki Linnakangas) This avoids a possible startup failure in pg_receivexlog.
•
While shutting down service on Windows, periodically send status updates to the Service Control Manager to prevent it from killing the service too soon; and ensure that pg_ctl will wait for shutdown (Krystian Bigaj)
2163
Appendix E. Release Notes •
Reduce risk of network deadlock when using libpq’s non-blocking mode (Heikki Linnakangas) When sending large volumes of data, it’s important to drain the input buffer every so often, in case the server has sent enough response data to cause it to block on output. (A typical scenario is that the server is sending a stream of NOTICE messages during COPY FROM STDIN.) This worked properly in the normal blocking mode, but not so much in non-blocking mode. We’ve modified libpq to opportunistically drain input when it can, but a full defense against this problem requires application cooperation: the application should watch for socket read-ready as well as write-ready conditions, and be sure to call PQconsumeInput() upon read-ready.
•
Fix array handling in ecpg (Michael Meskes)
•
Fix psql to sanely handle URIs and conninfo strings as the first parameter to \connect (David Fetter, Andrew Dunstan, Álvaro Herrera) This syntax has been accepted (but undocumented) for a long time, but previously some parameters might be taken from the old connection instead of the given string, which was agreed to be undesirable.
•
Suppress incorrect complaints from psql on some platforms that it failed to write ~/.psql_history at exit (Tom Lane) This misbehavior was caused by a workaround for a bug in very old (pre-2006) versions of libedit. We fixed it by removing the workaround, which will cause a similar failure to appear for anyone still using such versions of libedit. Recommendation: upgrade that library, or use libreadline.
•
Fix pg_dump’s rule for deciding which casts are system-provided casts that should not be dumped (Tom Lane)
•
In pg_dump, fix failure to honor -Z compression level option together with -Fd (Michael Paquier)
•
Make pg_dump consider foreign key relationships between extension configuration tables while choosing dump order (Gilles Darold, Michael Paquier, Stephen Frost) This oversight could result in producing dumps that fail to reload because foreign key constraints are transiently violated.
•
Fix dumping of views that are just VALUES(...) but have column aliases (Tom Lane)
•
In pg_upgrade, force timeline 1 in the new cluster (Bruce Momjian) This change prevents upgrade failures caused by bogus complaints about missing WAL history files.
•
In pg_upgrade, check for improperly non-connectable databases before proceeding (Bruce Momjian)
•
In pg_upgrade, quote directory paths properly in the generated delete_old_cluster script (Bruce Momjian)
•
In pg_upgrade, preserve database-level freezing info properly (Bruce Momjian) This oversight could cause missing-clog-file errors for tables within the postgres and template1 databases.
•
Run pg_upgrade and pg_resetxlog with restricted privileges on Windows, so that they don’t fail when run by an administrator (Muhammad Asif Naeem)
•
Improve handling of readdir() failures when scanning directories in initdb and pg_basebackup (Marco Nenciarini)
•
Fix slow sorting algorithm in contrib/intarray (Tom Lane)
•
Fix compile failure on Sparc V8 machines (Rob Rowan)
2164
Appendix E. Release Notes •
Update time zone data files to tzdata release 2015d for DST law changes in Egypt, Mongolia, and Palestine, plus historical changes in Canada and Chile. Also adopt revised zone abbreviations for the America/Adak zone (HST/HDT not HAST/HADT).
E.54. Release 9.1.15 Release date: 2015-02-05 This release contains a variety of fixes from 9.1.14. For information about new features in the 9.1 major release, see Section E.69.
E.54.1. Migration to Version 9.1.15 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.14, see Section E.55.
E.54.2. Changes •
Fix buffer overruns in to_char() (Bruce Momjian) When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241)
•
Fix buffer overrun in replacement *printf() functions (Tom Lane) PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well. This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242)
•
Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch) Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243)
•
Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas) If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message’s data as a new protocol message. An attacker able to submit crafted binary data within a command parameter
2165
Appendix E. Release Notes might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244) •
Fix information leak via constraint-violation error messages (Stephen Frost) Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161)
•
Lock down regression testing’s temporary installations on Windows (Noah Misch) Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067)
•
Avoid possible data corruption if ALTER DATABASE SET TABLESPACE is used to move a database to a new tablespace and then shortly later move it back to its original tablespace (Tom Lane)
•
Avoid corrupting tables when ANALYZE inside a transaction is rolled back (Andres Freund, Tom Lane, Michael Paquier) If the failing transaction had earlier removed the last index, rule, or trigger from the table, the table would be left in a corrupted state with the relevant pg_class flags not set though they should be.
•
Ensure that unlogged tables are copied correctly during CREATE DATABASE or ALTER DATABASE SET TABLESPACE (Pavan Deolasee, Andres Freund)
•
Fix DROP’s dependency searching to correctly handle the case where a table column is recursively visited before its table (Petr Jelinek, Tom Lane) This case is only known to arise when an extension creates both a datatype and a table using that datatype. The faulty code might refuse a DROP EXTENSION unless CASCADE is specified, which should not be required.
•
Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane) In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug.
•
Fix planning of SELECT FOR UPDATE when using a partial index on a child table (Kyotaro Horiguchi) In READ COMMITTED mode, SELECT FOR UPDATE must also recheck the partial index’s WHERE condition when rechecking a recently-updated row to see if it still satisfies the query’s WHERE condition. This requirement was missed if the index belonged to an inheritance child table, so that it was possible to incorrectly return rows that no longer satisfy the query condition.
•
Fix corner case wherein SELECT FOR UPDATE could return a row twice, and possibly miss returning other rows (Tom Lane) In READ COMMITTED mode, a SELECT FOR UPDATE that is scanning an inheritance tree could incorrectly return a row from a prior child table instead of the one it should return from a later child table.
•
Reject duplicate column names in the referenced-columns list of a FOREIGN KEY declaration (David Rowley) This restriction is per SQL standard. Previously we did not reject the case explicitly, but later on the code would fail with bizarre-looking errors.
2166
Appendix E. Release Notes •
Fix bugs in raising a numeric value to a large integral power (Tom Lane) The previous code could get a wrong answer, or consume excessive amounts of time and memory before realizing that the answer must overflow.
•
In numeric_recv(), truncate away any fractional digits that would be hidden according to the value’s dscale field (Tom Lane) A numeric value’s display scale (dscale) should never be less than the number of nonzero fractional digits; but apparently there’s at least one broken client application that transmits binary numeric values in which that’s true. This leads to strange behavior since the extra digits are taken into account by arithmetic operations even though they aren’t printed. The least risky fix seems to be to truncate away such “hidden” digits on receipt, so that the value is indeed what it prints as.
•
Reject out-of-range numeric timezone specifications (Tom Lane) Simple numeric timezone specifications exceeding +/- 168 hours (one week) would be accepted, but could then cause null-pointer dereference crashes in certain operations. There’s no use-case for such large UTC offsets, so reject them.
•
Fix bugs in tsquery @> tsquery operator (Heikki Linnakangas) Two different terms would be considered to match if they had the same CRC. Also, if the second operand had more terms than the first, it would be assumed not to be contained in the first; which is wrong since it might contain duplicate terms.
•
Improve ispell dictionary’s defenses against bad affix files (Tom Lane)
•
Allow more than 64K phrases in a thesaurus dictionary (David Boutin) The previous coding could crash on an oversize dictionary, so this was deemed a back-patchable bug fix rather than a feature addition.
•
Fix namespace handling in xpath() (Ali Akbar) Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation.
•
Fix planner problems with nested append relations, such as inherited tables within UNION ALL subqueries (Tom Lane)
•
Fail cleanly when a GiST index tuple doesn’t fit on a page, rather than going into infinite recursion (Andrew Gierth)
•
Exempt tables that have per-table cost_limit and/or cost_delay settings from autovacuum’s global cost balancing rules (Álvaro Herrera) The previous behavior resulted in basically ignoring these per-table settings, which was unintended. Now, a table having such settings will be vacuumed using those settings, independently of what is going on in other autovacuum workers. This may result in heavier total I/O load than before, so such settings should be re-examined for sanity.
•
Avoid wholesale autovacuuming when autovacuum is nominally off (Tom Lane) Even when autovacuum is nominally off, we will still launch autovacuum worker processes to vacuum tables that are at risk of XID wraparound. However, such a worker process then proceeded to vacuum all tables in the target database, if they met the usual thresholds for autovacuuming. This is at best pretty unexpected; at worst it delays response to the wraparound threat. Fix it so that if autovacuum is turned off, workers only do anti-wraparound vacuums and not any other work.
2167
Appendix E. Release Notes •
During crash recovery, ensure that unlogged relations are rewritten as empty and are synced to disk before recovery is considered complete (Abhijit Menon-Sen, Andres Freund) This prevents scenarios in which unlogged relations might contain garbage data following database crash recovery.
•
Fix race condition between hot standby queries and replaying a full-page image (Heikki Linnakangas) This mistake could result in transient errors in queries being executed in hot standby.
•
Fix several cases where recovery logic improperly ignored WAL records for COMMIT/ABORT PREPARED (Heikki Linnakangas) The most notable oversight was that recovery_target_xid could not be used to stop at a twophase commit.
•
Avoid creating unnecessary .ready marker files for timeline history files (Fujii Masao)
•
Fix possible null pointer dereference when an empty prepared statement is used and the log_statement setting is mod or ddl (Fujii Masao)
•
Change “pgstat wait timeout” warning message to be LOG level, and rephrase it to be more understandable (Tom Lane) This message was originally thought to be essentially a can’t-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads “using stale statistics instead of current ones because stats collector is not responding”.
•
Fix SPARC spinlock implementation to ensure correctness if the CPU is being run in a non-TSO coherency mode, as some non-Solaris kernels do (Andres Freund)
•
Warn if macOS’s setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch)
•
Fix processing of repeated dbname parameters in PQconnectdbParams() (Alex Shulgin) Unexpected behavior ensued if the first occurrence of dbname contained a connection string or URI to be expanded.
•
Ensure that libpq reports a suitable error message on unexpected socket EOF (Marko Tiikkaja, Tom Lane) Depending on kernel behavior, libpq might return an empty error string rather than something useful when the server unexpectedly closed the socket.
•
Clear any old error message during PQreset() (Heikki Linnakangas) If PQreset() is called repeatedly, and the connection cannot be re-established, error messages from the failed connection attempts kept accumulating in the PGconn’s error string.
•
Properly handle out-of-memory conditions while parsing connection options in libpq (Alex Shulgin, Heikki Linnakangas)
•
Fix array overrun in ecpg’s version of ParseDateTime() (Michael Paquier)
•
In initdb, give a clearer error message if a password file is specified but is empty (Mats Erik Andersson)
•
Fix psql’s \s command to work nicely with libedit, and add pager support (Stepan Rutz, Tom Lane) When using libedit rather than readline, \s printed the command history in a fairly unreadable encoded format, and on recent libedit versions might fail altogether. Fix that by printing the history
2168
Appendix E. Release Notes ourselves rather than having the library do it. A pleasant side-effect is that the pager is used if appropriate. This patch also fixes a bug that caused newline encoding to be applied inconsistently when saving the command history with libedit. Multiline history entries written by older psql versions will be read cleanly with this patch, but perhaps not vice versa, depending on the exact libedit versions involved. •
Improve consistency of parsing of psql’s special variables (Tom Lane) Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors.
•
Fix psql’s expanded-mode display to work consistently when using border = 3 and linestyle = ascii or unicode (Stephen Frost)
•
Improve performance of pg_dump when the database contains many instances of multiple dependency paths between the same two objects (Tom Lane)
•
Fix possible deadlock during parallel restore of a schema-only dump (Robert Haas, Tom Lane)
•
Fix core dump in pg_dump --binary-upgrade on zero-column composite type (Rushabh Lathia)
•
Prevent WAL files created by pg_basebackup -x/-X from being archived again when the standby is promoted (Andres Freund)
•
Fix upgrade-from-unpackaged script for contrib/citext (Tom Lane)
•
Fix block number checking in contrib/pageinspect’s get_raw_page() (Tom Lane) The incorrect checking logic could prevent access to some pages in non-main relation forks.
•
Fix contrib/pgcrypto’s pgp_sym_decrypt() to not fail on messages whose length is 6 less than a power of 2 (Marko Tiikkaja)
•
Fix file descriptor leak in contrib/pg_test_fsync (Jeff Janes) This could cause failure to remove temporary files on Windows.
•
Handle unexpected query results, especially NULLs, safely in contrib/tablefunc’s connectby() (Michael Paquier) connectby() previously crashed if it encountered a NULL key value. It now prints that row but
doesn’t recurse further. •
Avoid a possible crash in contrib/xml2’s xslt_process() (Mark Simonetti) libxslt seems to have an undocumented dependency on the order in which resources are freed; reorder our calls to avoid a crash.
•
Mark some contrib I/O functions with correct volatility properties (Tom Lane) The previous over-conservative marking was immaterial in normal use, but could cause optimization problems or rejection of valid index expression definitions. Since the consequences are not large, we’ve just adjusted the function definitions in the extension modules’ scripts, without changing version numbers.
•
Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier)
2169
Appendix E. Release Notes These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues. •
Detect incompatible OpenLDAP versions during build (Noah Misch) With OpenLDAP versions 2.4.24 through 2.4.31, inclusive, PostgreSQL backends can crash at exit. Raise a warning during configure based on the compile-time OpenLDAP version number, and test the crashing scenario in the contrib/dblink regression test.
•
In non-MSVC Windows builds, ensure libpq.dll is installed with execute permissions (Noah Misch)
•
Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane) This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations.
•
Support time zone abbreviations that change UTC offset from time to time (Tom Lane) Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date.
•
Update time zone abbreviations lists (Tom Lane) Add CST (China Standard Time) to our lists. Remove references to ADT as “Arabia Daylight Time”, an abbreviation that’s been out of use since 2007; therefore, claiming there is a conflict with “Atlantic Daylight Time” doesn’t seem especially helpful. Fix entirely incorrect GMT offsets for CKT (Cook Islands), FJT, and FJST (Fiji); we didn’t even have them on the proper side of the date line.
•
Update time zone data files to tzdata release 2015a. The IANA timezone database has adopted abbreviations of the form Ax ST/Ax DT for all Australian time zones, reflecting what they believe to be current majority practice Down Under. These names do not conflict with usage elsewhere (other than ACST for Acre Summer Time, which has been in disuse since 1994). Accordingly, adopt these names into our “Default” timezone abbreviation set. The “Australia” abbreviation set now contains only CST, EAST, EST, SAST, SAT, and WST, all of which are thought to be mostly historical usage. Note that SAST has also been changed to be South Africa Standard Time in the “Default” abbreviation set. Also, add zone abbreviations SRET (Asia/Srednekolymsk) and XJT (Asia/Urumqi), and use WSST/WSDT for western Samoa. Also, there were DST law changes in Chile, Mexico, the Turks & Caicos Islands (America/Grand_Turk), and Fiji. There is a new zone Pacific/Bougainville for portions of Papua New Guinea. Also, numerous corrections for historical (pre-1970) time zone data.
E.55. Release 9.1.14 Release date: 2014-07-24
2170
Appendix E. Release Notes This release contains a variety of fixes from 9.1.13. For information about new features in the 9.1 major release, see Section E.69.
E.55.1. Migration to Version 9.1.14 A dump/restore is not required for those running 9.1.X. However, this release corrects an index corruption problem in some GiST indexes. See the first changelog entry below to find out whether your installation has been affected and what steps you should take if so. Also, if you are upgrading from a version earlier than 9.1.11, see Section E.58.
E.55.2. Changes •
Correctly initialize padding bytes in contrib/btree_gist indexes on bit columns (Heikki Linnakangas) This error could result in incorrect query results due to values that should compare equal not being seen as equal. Users with GiST indexes on bit or bit varying columns should REINDEX those indexes after installing this update.
•
Protect against torn pages when deleting GIN list pages (Heikki Linnakangas) This fix prevents possible index corruption if a system crash occurs while the page update is being written to disk.
•
Don’t clear the right-link of a GiST index page while replaying updates from WAL (Heikki Linnakangas) This error could lead to transiently wrong answers from GiST index scans performed in Hot Standby.
•
Fix feedback status when hot_standby_feedback is turned off on-the-fly (Simon Riggs)
•
Fix
possibly-incorrect
cache
invalidation
during
nested
calls
to
ReceiveSharedInvalidMessages (Andres Freund) •
Fix “could not find pathkey item to sort” planner failures with UNION ALL over subqueries reading from tables with inheritance children (Tom Lane)
•
Don’t assume a subquery’s output is unique if there’s a set-returning function in its targetlist (David Rowley) This oversight could lead to misoptimization of constructs like WHERE x IN (SELECT y, generate_series(1,10) FROM t GROUP BY y).
•
Fix failure to detoast fields in composite elements of structured types (Tom Lane) This corrects cases where TOAST pointers could be copied into other tables without being dereferenced. If the original data is later deleted, it would lead to errors like “missing chunk number 0 for toast value ...” when the now-dangling pointer is used.
•
Fix “record type has not been registered” failures with whole-row references to the output of Append plan nodes (Tom Lane)
•
Fix possible crash when invoking a user-defined function while rewinding a cursor (Tom Lane)
•
Fix query-lifespan memory leak while evaluating the arguments for a function in FROM (Tom Lane)
2171
Appendix E. Release Notes •
Fix session-lifespan memory leaks in regular-expression processing (Tom Lane, Arthur O’Dwyer, Greg Stark)
•
Fix data encoding error in hungarian.stop (Tom Lane)
•
Prevent foreign tables from being created with OIDS when default_with_oids is true (Etsuro Fujita)
•
Fix liveness checks for rows that were inserted in the current transaction and then deleted by a now-rolled-back subtransaction (Andres Freund) This could cause problems (at least spurious warnings, and at worst an infinite loop) if CREATE INDEX or CLUSTER were done later in the same transaction.
•
Clear pg_stat_activity.xact_start during PREPARE TRANSACTION (Andres Freund) After the PREPARE, the originating session is no longer in a transaction, so it should not continue to display a transaction start time.
•
Fix REASSIGN OWNED to not fail for text search objects (Álvaro Herrera)
•
Block signals during postmaster startup (Tom Lane) This ensures that the postmaster will properly clean up after itself if, for example, it receives SIGINT while still starting up.
•
Fix client host name lookup when processing pg_hba.conf entries that specify host names instead of IP addresses (Tom Lane) Ensure that reverse-DNS lookup failures are reported, instead of just silently not matching such entries. Also ensure that we make only one reverse-DNS lookup attempt per connection, not one per host name entry, which is what previously happened if the lookup attempts failed.
•
Secure Unix-domain sockets of temporary postmasters started during make check (Noah Misch) Any local user able to access the socket file could connect as the server’s bootstrap superuser, then proceed to execute arbitrary code as the operating-system user running the test, as we previously noted in CVE-2014-0067. This change defends against that risk by placing the server’s socket in a temporary, mode 0700 subdirectory of /tmp. The hazard remains however on platforms where Unix sockets are not supported, notably Windows, because then the temporary postmaster must accept local TCP connections. A useful side effect of this change is to simplify make check testing in builds that override DEFAULT_PGSOCKET_DIR. Popular non-default values like /var/run/postgresql are often not writable by the build user, requiring workarounds that will no longer be necessary.
•
Fix tablespace creation WAL replay to work on Windows (MauMau)
•
Fix detection of socket creation failures on Windows (Bruce Momjian)
•
On Windows, allow new sessions to absorb values of PGC_BACKEND parameters (such as log_connections) from the configuration file (Amit Kapila) Previously, if such a parameter were changed in the file post-startup, the change would have no effect.
•
Properly quote executable path names on Windows (Nikhil Deshpande) This oversight could cause initdb and pg_upgrade to fail on Windows, if the installation path contained both spaces and @ signs.
•
Fix linking of libpython on macOS (Tom Lane) The method we previously used can fail with the Python library supplied by Xcode 5.0 and later.
•
Avoid buffer bloat in libpq when the server consistently sends data faster than the client can absorb it (Shin-ichi Morita, Tom Lane)
2172
Appendix E. Release Notes libpq could be coerced into enlarging its input buffer until it runs out of memory (which would be reported misleadingly as “lost synchronization with server”). Under ordinary circumstances it’s quite far-fetched that data could be continuously transmitted more quickly than the recv() loop can absorb it, but this has been observed when the client is artificially slowed by scheduler constraints. •
Ensure that LDAP lookup attempts in libpq time out as intended (Laurenz Albe)
•
Fix ecpg to do the right thing when an array of char * is the target for a FETCH statement returning more than one row, as well as some other array-handling fixes (Ashutosh Bapat)
•
Fix pg_restore’s processing of old-style large object comments (Tom Lane) A direct-to-database restore from an archive file generated by a pre-9.0 version of pg_dump would usually fail if the archive contained more than a few comments for large objects.
•
In contrib/pgcrypto functions, ensure sensitive information is cleared from stack variables before returning (Marko Kreen)
•
In contrib/uuid-ossp, cache the state of the OSSP UUID library across calls (Tom Lane) This improves the efficiency of UUID generation and reduces the amount of entropy drawn from /dev/urandom, on platforms that have that.
•
Update time zone data files to tzdata release 2014e for DST law changes in Crimea, Egypt, and Morocco.
E.56. Release 9.1.13 Release date: 2014-03-20 This release contains a variety of fixes from 9.1.12. For information about new features in the 9.1 major release, see Section E.69.
E.56.1. Migration to Version 9.1.13 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.11, see Section E.58.
E.56.2. Changes •
Restore GIN metapages unconditionally to avoid torn-page risk (Heikki Linnakangas) Although this oversight could theoretically result in a corrupted index, it is unlikely to have caused any problems in practice, since the active part of a GIN metapage is smaller than a standard 512byte disk sector.
•
Avoid race condition in checking transaction commit status during receipt of a NOTIFY message (Marko Tiikkaja) This prevents a scenario wherein a sufficiently fast client might respond to a notification before database updates made by the notifier have become visible to the recipient.
•
Allow regular-expression operators to be terminated early by query cancel requests (Tom Lane)
2173
Appendix E. Release Notes This prevents scenarios wherein a pathological regular expression could lock up a server process uninterruptibly for a long time. •
Remove incorrect code that tried to allow OVERLAPS with single-element row arguments (Joshua Yanovski) This code never worked correctly, and since the case is neither specified by the SQL standard nor documented, it seemed better to remove it than fix it.
•
Avoid getting more than AccessShareLock when de-parsing a rule or view (Dean Rasheed) This oversight resulted in pg_dump unexpectedly acquiring RowExclusiveLock locks on tables mentioned as the targets of INSERT/UPDATE/DELETE commands in rules. While usually harmless, that could interfere with concurrent transactions that tried to acquire, for example, ShareLock on those tables.
•
Improve performance of index endpoint probes during planning (Tom Lane) This change fixes a significant performance problem that occurred when there were many not-yetcommitted rows at the end of the index, which is a common situation for indexes on sequentiallyassigned values such as timestamps or sequence-generated identifiers.
•
Fix walsender’s failure to shut down cleanly when client is pg_receivexlog (Fujii Masao)
•
Fix test to see if hot standby connections can be allowed immediately after a crash (Heikki Linnakangas)
•
Prevent interrupts while reporting non-ERROR messages (Tom Lane) This guards against rare server-process freezeups due to recursive entry to syslog(), and perhaps other related problems.
•
Fix memory leak in PL/Perl when returning a composite result, including multiple-OUT-parameter cases (Alex Hunsaker)
•
Prevent intermittent “could not reserve shared memory region” failures on recent Windows versions (MauMau)
•
Update time zone data files to tzdata release 2014a for DST law changes in Fiji and Turkey, plus historical changes in Israel and Ukraine.
E.57. Release 9.1.12 Release date: 2014-02-20 This release contains a variety of fixes from 9.1.11. For information about new features in the 9.1 major release, see Section E.69.
E.57.1. Migration to Version 9.1.12 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.11, see Section E.58.
2174
Appendix E. Release Notes
E.57.2. Changes •
Shore up GRANT ... WITH ADMIN OPTION restrictions (Noah Misch) Granting a role without ADMIN OPTION is supposed to prevent the grantee from adding or removing members from the granted role, but this restriction was easily bypassed by doing SET ROLE first. The security impact is mostly that a role member can revoke the access of others, contrary to the wishes of his grantor. Unapproved role member additions are a lesser concern, since an uncooperative role member could provide most of his rights to others anyway by creating views or SECURITY DEFINER functions. (CVE-2014-0060)
•
Prevent privilege escalation via manual calls to PL validator functions (Andres Freund) The primary role of PL validator functions is to be called implicitly during CREATE FUNCTION, but they are also normal SQL functions that a user can call explicitly. Calling a validator on a function actually written in some other language was not checked for and could be exploited for privilege-escalation purposes. The fix involves adding a call to a privilege-checking function in each validator function. Non-core procedural languages will also need to make this change to their own validator functions, if any. (CVE-2014-0061)
•
Avoid multiple name lookups during table and index DDL (Robert Haas, Andres Freund) If the name lookups come to different conclusions due to concurrent activity, we might perform some parts of the DDL on a different table than other parts. At least in the case of CREATE INDEX, this can be used to cause the permissions checks to be performed against a different table than the index creation, allowing for a privilege escalation attack. (CVE-2014-0062)
•
Prevent buffer overrun with long datetime strings (Noah Misch) The MAXDATELEN constant was too small for the longest possible value of type interval, allowing a buffer overrun in interval_out(). Although the datetime input functions were more careful about avoiding buffer overrun, the limit was short enough to cause them to reject some valid inputs, such as input containing a very long timezone name. The ecpg library contained these vulnerabilities along with some of its own. (CVE-2014-0063)
•
Prevent buffer overrun due to integer overflow in size calculations (Noah Misch, Heikki Linnakangas) Several functions, mostly type input functions, calculated an allocation size without checking for overflow. If overflow did occur, a too-small buffer would be allocated and then written past. (CVE2014-0064)
•
Prevent overruns of fixed-size buffers (Peter Eisentraut, Jozef Mlich) Use strlcpy() and related functions to provide a clear guarantee that fixed-size buffers are not overrun. Unlike the preceding items, it is unclear whether these cases really represent live issues, since in most cases there appear to be previous constraints on the size of the input string. Nonetheless it seems prudent to silence all Coverity warnings of this type. (CVE-2014-0065)
•
Avoid crashing if crypt() returns NULL (Honza Horak, Bruce Momjian) There are relatively few scenarios in which crypt() could return NULL, but contrib/chkpass would crash if it did. One practical case in which this could be an issue is if libc is configured to refuse to execute unapproved hashing algorithms (e.g., “FIPS mode”). (CVE-2014-0066)
•
Document risks of make check in the regression testing instructions (Noah Misch, Tom Lane) Since the temporary server started by make check uses “trust” authentication, another user on the same machine could connect to it as database superuser, and then potentially exploit the privileges of the operating-system user who started the tests. A future release will probably incorporate
2175
Appendix E. Release Notes changes in the testing procedure to prevent this risk, but some public discussion is needed first. So for the moment, just warn people against using make check when there are untrusted users on the same machine. (CVE-2014-0067) •
Fix possible mis-replay of WAL records when some segments of a relation aren’t full size (Greg Stark, Tom Lane) The WAL update could be applied to the wrong page, potentially many pages past where it should have been. Aside from corrupting data, this error has been observed to result in significant “bloat” of standby servers compared to their masters, due to updates being applied far beyond where the end-of-file should have been. This failure mode does not appear to be a significant risk during crash recovery, only when initially synchronizing a standby created from a base backup taken from a quickly-changing master.
•
Fix bug in determining when recovery has reached consistency (Tomonari Katsumata, Heikki Linnakangas) In some cases WAL replay would mistakenly conclude that the database was already consistent at the start of replay, thus possibly allowing hot-standby queries before the database was really consistent. Other symptoms such as “PANIC: WAL contains references to invalid pages” were also possible.
•
Fix improper locking of btree index pages while replaying a VACUUM operation in hot-standby mode (Andres Freund, Heikki Linnakangas, Tom Lane) This error could result in “PANIC: WAL contains references to invalid pages” failures.
•
Ensure that insertions into non-leaf GIN index pages write a full-page WAL record when appropriate (Heikki Linnakangas) The previous coding risked index corruption in the event of a partial-page write during a system crash.
•
When pause_at_recovery_target and recovery_target_inclusive are both set, ensure the target record is applied before pausing, not after (Heikki Linnakangas)
•
Fix race conditions during server process exit (Robert Haas) Ensure that signal handlers don’t attempt to use the process’s MyProc pointer after it’s no longer valid.
•
Fix race conditions in walsender shutdown logic and walreceiver SIGHUP signal handler (Tom Lane)
•
Fix unsafe references to errno within error reporting logic (Christian Kruse) This would typically lead to odd behaviors such as missing or inappropriate HINT fields.
•
Fix possible crashes from using ereport() too early during server startup (Tom Lane) The principal case we’ve seen in the field is a crash if the server is started in a directory it doesn’t have permission to read.
•
Clear retry flags properly in OpenSSL socket write function (Alexander Kukushkin) This omission could result in a server lockup after unexpected loss of an SSL-encrypted connection.
•
Fix length checking for Unicode identifiers (U&"..." syntax) containing escapes (Tom Lane) A spurious truncation warning would be printed for such identifiers if the escaped form of the identifier was too long, but the identifier actually didn’t need truncation after de-escaping.
•
Allow keywords that are type names to be used in lists of roles (Stephen Frost)
2176
Appendix E. Release Notes A previous patch allowed such keywords to be used without quoting in places such as role identifiers; but it missed cases where a list of role identifiers was permitted, such as DROP ROLE. •
Fix parser crash for EXISTS(SELECT * FROM zero_column_table) (Tom Lane)
•
Fix possible crash due to invalid plan for nested sub-selects, such as WHERE (... x IN (SELECT ...) ...) IN (SELECT ...) (Tom Lane)
•
Ensure that ANALYZE creates statistics for a table column even when all the values in it are “too wide” (Tom Lane) ANALYZE intentionally omits very wide values from its histogram and most-common-values cal-
culations, but it neglected to do something sane in the case that all the sampled entries are too wide. •
In ALTER TABLE ... SET TABLESPACE, allow the database’s default tablespace to be used without a permissions check (Stephen Frost) CREATE TABLE has always allowed such usage, but ALTER TABLE didn’t get the memo.
•
Fix “cannot accept a set” error when some arms of a CASE return a set and others don’t (Tom Lane)
•
Fix checks for all-zero client addresses in pgstat functions (Kevin Grittner)
•
Fix possible misclassification of multibyte characters by the text search parser (Tom Lane) Non-ASCII characters could be misclassified when using C locale with a multibyte encoding. On Cygwin, non-C locales could fail as well.
•
Fix possible misbehavior in plainto_tsquery() (Heikki Linnakangas) Use memmove() not memcpy() for copying overlapping memory regions. There have been no field reports of this actually causing trouble, but it’s certainly risky.
•
Fix placement of permissions checks in pg_start_backup() and pg_stop_backup() (Andres Freund, Magnus Hagander) The previous coding might attempt to do catalog access when it shouldn’t.
•
Accept SHIFT_JIS as an encoding name for locale checking purposes (Tatsuo Ishii)
•
Fix misbehavior of PQhost() on Windows (Fujii Masao) It should return localhost if no host has been specified.
•
Improve error handling in libpq and psql for failures during COPY TO STDOUT/FROM STDIN (Tom Lane) In particular this fixes an infinite loop that could occur in 9.2 and up if the server connection was lost during COPY FROM STDIN. Variants of that scenario might be possible in older versions, or with other client applications.
•
Fix possible incorrect printing of filenames in pg_basebackup’s verbose mode (Magnus Hagander)
•
Avoid including tablespaces inside PGDATA twice in base backups (Dimitri Fontaine, Magnus Hagander)
•
Fix misaligned descriptors in ecpg (MauMau)
•
In ecpg, handle lack of a hostname in the connection parameters properly (Michael Meskes)
•
Fix performance regression in contrib/dblink connection startup (Joe Conway) Avoid an unnecessary round trip when client and server encodings match.
•
In contrib/isn, fix incorrect calculation of the check digit for ISMN values (Fabien Coelho)
•
Ensure client-code-only installation procedure works as documented (Peter Eisentraut)
2177
Appendix E. Release Notes •
In Mingw and Cygwin builds, install the libpq DLL in the bin directory (Andrew Dunstan) This duplicates what the MSVC build has long done. It should fix problems with programs like psql failing to start because they can’t find the DLL.
•
Avoid using the deprecated dllwrap tool in Cygwin builds (Marco Atzeri)
•
Don’t generate plain-text HISTORY and src/test/regress/README files anymore (Tom Lane) These text files duplicated the main HTML and PDF documentation formats. The trouble involved in maintaining them greatly outweighs the likely audience for plain-text format. Distribution tarballs will still contain files by these names, but they’ll just be stubs directing the reader to consult the main documentation. The plain-text INSTALL file will still be maintained, as there is arguably a use-case for that.
•
Update time zone data files to tzdata release 2013i for DST law changes in Jordan and historical changes in Cuba. In addition, the zones Asia/Riyadh87, Asia/Riyadh88, and Asia/Riyadh89 have been removed, as they are no longer maintained by IANA, and never represented actual civil timekeeping practice.
E.58. Release 9.1.11 Release date: 2013-12-05 This release contains a variety of fixes from 9.1.10. For information about new features in the 9.1 major release, see Section E.69.
E.58.1. Migration to Version 9.1.11 A dump/restore is not required for those running 9.1.X. However, this release corrects a number of potential data corruption issues. See the first two changelog entries below to find out whether your installation has been affected and what steps you can take if so. Also, if you are upgrading from a version earlier than 9.1.9, see Section E.60.
E.58.2. Changes •
Fix VACUUM’s tests to see whether it can update relfrozenxid (Andres Freund) In some cases VACUUM (either manual or autovacuum) could incorrectly advance a table’s relfrozenxid value, allowing tuples to escape freezing, causing those rows to become invisible once 2^31 transactions have elapsed. The probability of data loss is fairly low since multiple incorrect advancements would need to happen before actual loss occurs, but it’s not zero. Users upgrading from releases 9.0.4 or 8.4.8 or earlier are not affected, but all later versions contain the bug. The issue can be ameliorated by, after upgrading, vacuuming all tables in all databases while having vacuum_freeze_table_age set to zero. This will fix any latent corruption but will not be able to fix all pre-existing data errors. However, an installation can be presumed safe after performing this
2178
Appendix E. Release Notes vacuuming if it has executed fewer than 2^31 update transactions in its lifetime (check this with SELECT txid_current() < 2^31). •
Fix initialization of pg_clog and pg_subtrans during hot standby startup (Andres Freund, Heikki Linnakangas) This bug can cause data loss on standby servers at the moment they start to accept hot-standby queries, by marking committed transactions as uncommitted. The likelihood of such corruption is small unless, at the time of standby startup, the primary server has executed many updating transactions since its last checkpoint. Symptoms include missing rows, rows that should have been deleted being still visible, and obsolete versions of updated rows being still visible alongside their newer versions. This bug was introduced in versions 9.3.0, 9.2.5, 9.1.10, and 9.0.14. Standby servers that have only been running earlier releases are not at risk. It’s recommended that standby servers that have ever run any of the buggy releases be re-cloned from the primary (e.g., with a new base backup) after upgrading.
•
Truncate pg_multixact contents during WAL replay (Andres Freund) This avoids ever-increasing disk space consumption in standby servers.
•
Fix race condition in GIN index posting tree page deletion (Heikki Linnakangas) This could lead to transient wrong answers or query failures.
•
Avoid flattening a subquery whose SELECT list contains a volatile function wrapped inside a subSELECT (Tom Lane) This avoids unexpected results due to extra evaluations of the volatile function.
•
Fix planner’s processing of non-simple-variable subquery outputs nested within outer joins (Tom Lane) This error could lead to incorrect plans for queries involving multiple levels of subqueries within JOIN syntax.
•
Fix incorrect generation of optimized MIN()/MAX() plans for inheritance trees (Tom Lane) The planner could fail in cases where the MIN()/MAX() argument was an expression rather than a simple variable.
•
Fix premature deletion of temporary files (Andres Freund)
•
Fix possible read past end of memory in rule printing (Peter Eisentraut)
•
Fix array slicing of int2vector and oidvector values (Tom Lane) Expressions of this kind are now implicitly promoted to regular int2 or oid arrays.
•
Fix incorrect behaviors when using a SQL-standard, simple GMT offset timezone (Tom Lane) In some cases, the system would use the simple GMT offset value when it should have used the regular timezone setting that had prevailed before the simple offset was selected. This change also causes the timeofday function to honor the simple GMT offset zone.
•
Prevent possible misbehavior when logging translations of Windows error codes (Tom Lane)
•
Properly quote generated command lines in pg_ctl (Naoya Anzai and Tom Lane) This fix applies only to Windows.
•
Fix pg_dumpall to work when a source database sets default_transaction_read_only via ALTER DATABASE SET (Kevin Grittner) Previously, the generated script would fail during restore.
2179
Appendix E. Release Notes •
Make ecpg search for quoted cursor names case-sensitively (Zoltán Böszörményi)
•
Fix ecpg’s processing of lists of variables declared varchar (Zoltán Böszörményi)
•
Make contrib/lo defend against incorrect trigger definitions (Marc Cousin)
•
Update time zone data files to tzdata release 2013h for DST law changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco, and Palestine. Also, new timezone abbreviations WIB, WIT, WITA for Indonesia.
E.59. Release 9.1.10 Release date: 2013-10-10 This release contains a variety of fixes from 9.1.9. For information about new features in the 9.1 major release, see Section E.69.
E.59.1. Migration to Version 9.1.10 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.9, see Section E.60.
E.59.2. Changes •
Prevent corruption of multi-byte characters when attempting to case-fold identifiers (Andrew Dunstan) PostgreSQL case-folds non-ASCII characters only when using a single-byte server encoding.
•
Fix checkpoint memory leak in background writer when wal_level = hot_standby (Naoya Anzai)
•
Fix memory leak caused by lo_open() failure (Heikki Linnakangas)
•
Fix memory overcommit bug when work_mem is using more than 24GB of memory (Stephen Frost)
•
Serializable snapshot fixes (Kevin Grittner, Heikki Linnakangas)
•
Fix deadlock bug in libpq when using SSL (Stephen Frost)
•
Fix possible SSL state corruption in threaded libpq applications (Nick Phillips, Stephen Frost)
•
Properly compute row estimates for boolean columns containing many NULL values (Andrew Gierth) Previously tests like col IS NOT TRUE and col IS NOT FALSE did not properly factor in NULL values when estimating plan costs.
•
Prevent pushing down WHERE clauses into unsafe UNION/INTERSECT subqueries (Tom Lane) Subqueries of a UNION or INTERSECT that contain set-returning functions or volatile functions in their SELECT lists could be improperly optimized, leading to run-time errors or incorrect query results.
•
Fix rare case of “failed to locate grouping columns” planner failure (Tom Lane)
•
Fix pg_dump of foreign tables with dropped columns (Andrew Dunstan)
2180
Appendix E. Release Notes Previously such cases could cause a pg_upgrade error. •
Reorder pg_dump processing of extension-related rules and event triggers (Joe Conway)
•
Force dumping of extension tables if specified by pg_dump -t or -n (Joe Conway)
•
Improve view dumping code’s handling of dropped columns in referenced tables (Tom Lane)
•
Fix pg_restore -l with the directory archive to display the correct format name (Fujii Masao)
•
Properly record index comments created using UNIQUE and PRIMARY KEY syntax (Andres Freund) This fixes a parallel pg_restore failure.
•
Properly guarantee transmission of WAL files before clean switchover (Fujii Masao) Previously, the streaming replication connection might close before all WAL files had been replayed on the standby.
•
Fix WAL segment timeline handling during recovery (Mitsumasa Kondo, Heikki Linnakangas) WAL file recycling during standby recovery could lead to premature recovery completion, resulting in data loss.
•
Fix REINDEX TABLE and REINDEX DATABASE to properly revalidate constraints and mark invalidated indexes as valid (Noah Misch) REINDEX INDEX has always worked properly.
•
Fix possible deadlock during concurrent CREATE INDEX CONCURRENTLY operations (Tom Lane)
•
Fix regexp_matches() handling of zero-length matches (Jeevan Chalke) Previously, zero-length matches like ’^’ could return too many matches.
•
Fix crash for overly-complex regular expressions (Heikki Linnakangas)
•
Fix regular expression match failures for back references combined with non-greedy quantifiers (Jeevan Chalke)
•
Prevent CREATE FUNCTION from checking SET variables unless function body checking is enabled (Tom Lane)
•
Allow ALTER DEFAULT PRIVILEGES to operate on schemas without requiring CREATE permission (Tom Lane)
•
Loosen restriction on keywords used in queries (Tom Lane) Specifically, lessen keyword restrictions for role names, language names, EXPLAIN and COPY options, and SET values. This allows COPY ... (FORMAT BINARY) to work as expected; previously BINARY needed to be quoted.
•
Fix pgp_pub_decrypt() so it works for secret keys with passwords (Marko Kreen)
•
Make pg_upgrade use pg_dump --quote-all-identifiers to avoid problems with keyword changes between releases (Tom Lane)
•
Remove rare inaccurate warning during vacuum of index-less tables (Heikki Linnakangas)
•
Ensure that VACUUM ANALYZE still runs the ANALYZE phase if its attempt to truncate the file is cancelled due to lock conflicts (Kevin Grittner)
•
Avoid possible failure when performing transaction control commands (e.g ROLLBACK) in prepared queries (Tom Lane)
•
Ensure that floating-point data input accepts standard spellings of “infinity” on all platforms (Tom Lane)
2181
Appendix E. Release Notes The C99 standard says that allowable spellings are inf, +inf, -inf, infinity, +infinity, and -infinity. Make sure we recognize these even if the platform’s strtod function doesn’t. •
Expand ability to compare rows to records and arrays (Rafal Rzepecki, Tom Lane)
•
Update time zone data files to tzdata release 2013d for DST law changes in Israel, Morocco, Palestine, and Paraguay. Also, historical zone data corrections for Macquarie Island.
E.60. Release 9.1.9 Release date: 2013-04-04 This release contains a variety of fixes from 9.1.8. For information about new features in the 9.1 major release, see Section E.69.
E.60.1. Migration to Version 9.1.9 A dump/restore is not required for those running 9.1.X. However, this release corrects several errors in management of GiST indexes. After installing this update, it is advisable to REINDEX any GiST indexes that meet one or more of the conditions described below. Also, if you are upgrading from a version earlier than 9.1.6, see Section E.63.
E.60.2. Changes •
Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi) A connection request containing a database name that begins with “-” could be crafted to damage or destroy files within the server’s data directory, even if the request is eventually rejected. (CVE2013-1899)
•
Reset OpenSSL randomness state in each postmaster child process (Marko Kreen) This avoids a scenario wherein random numbers generated by contrib/pgcrypto functions might be relatively easy for another database user to guess. The risk is only significant when the postmaster is configured with ssl = on but most connections don’t use SSL encryption. (CVE2013-1900)
•
Make REPLICATION privilege checks test current user not authenticated user (Noah Misch) An unprivileged database user could exploit this mistake to call pg_start_backup() or pg_stop_backup(), thus possibly interfering with creation of routine backups. (CVE-2013-1901)
•
Fix GiST indexes to not use “fuzzy” geometric comparisons when it’s not appropriate to do so (Alexander Korotkov) The core geometric types perform comparisons using “fuzzy” equality, but gist_box_same must do exact comparisons, else GiST indexes using it might become inconsistent. After installing this update, users should REINDEX any GiST indexes on box, polygon, circle, or point columns, since all of these use gist_box_same.
2182
Appendix E. Release Notes •
Fix erroneous range-union and penalty logic in GiST indexes that use contrib/btree_gist for variable-width data types, that is text, bytea, bit, and numeric columns (Tom Lane) These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in useless index bloat. Users are advised to REINDEX such indexes after installing this update.
•
Fix bugs in GiST page splitting code for multi-column indexes (Tom Lane) These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in indexes that are unnecessarily inefficient to search. Users are advised to REINDEX multi-column GiST indexes after installing this update.
•
Fix gist_point_consistent to handle fuzziness consistently (Alexander Korotkov) Index scans on GiST indexes on point columns would sometimes yield results different from a sequential scan, because gist_point_consistent disagreed with the underlying operator code about whether to do comparisons exactly or fuzzily.
•
Fix buffer leak in WAL replay (Heikki Linnakangas) This bug could result in “incorrect local pin count” errors during replay, making recovery impossible.
•
Fix race condition in DELETE RETURNING (Tom Lane) Under the right circumstances, DELETE RETURNING could attempt to fetch data from a shared buffer that the current process no longer has any pin on. If some other process changed the buffer meanwhile, this would lead to garbage RETURNING output, or even a crash.
•
Fix infinite-loop risk in regular expression compilation (Tom Lane, Don Porter)
•
Fix potential null-pointer dereference in regular expression compilation (Tom Lane)
•
Fix to_char() to use ASCII-only case-folding rules where appropriate (Tom Lane) This fixes misbehavior of some template patterns that should be locale-independent, but mishandled “I” and “i” in Turkish locales.
•
Fix unwanted rejection of timestamp 1999-12-31 24:00:00 (Tom Lane)
•
Fix logic error when a single transaction does UNLISTEN then LISTEN (Tom Lane) The session wound up not listening for notify events at all, though it surely should listen in this case.
•
Fix possible planner crash after columns have been added to a view that’s depended on by another view (Tom Lane)
•
Remove useless “picksplit doesn’t support secondary split” log messages (Josh Hansen, Tom Lane) This message seems to have been added in expectation of code that was never written, and probably never will be, since GiST’s default handling of secondary splits is actually pretty good. So stop nagging end users about it.
•
Fix possible failure to send a session’s last few transaction commit/abort counts to the statistics collector (Tom Lane)
•
Eliminate memory leaks in PL/Perl’s spi_prepare() function (Alex Hunsaker, Tom Lane)
Avoid crash in pg_dump when an incorrect connection string is given (Heikki Linnakangas)
•
Ignore invalid indexes in pg_dump and pg_upgrade (Michael Paquier, Bruce Momjian)
2183
Appendix E. Release Notes Dumping invalid indexes can cause problems at restore time, for example if the reason the index creation failed was because it tried to enforce a uniqueness condition not satisfied by the table’s data. Also, if the index creation is in fact still in progress, it seems reasonable to consider it to be an uncommitted DDL change, which pg_dump wouldn’t be expected to dump anyway. pg_upgrade now also skips invalid indexes rather than failing. •
In pg_basebackup, include only the current server version’s subdirectory when backing up a tablespace (Heikki Linnakangas)
•
Add a server version check in pg_basebackup and pg_receivexlog, so they fail cleanly with version combinations that won’t work (Heikki Linnakangas)
•
Fix contrib/pg_trgm’s similarity() function to return zero for trigram-less strings (Tom Lane) Previously it returned NaN due to internal division by zero.
•
Update time zone data files to tzdata release 2013b for DST law changes in Chile, Haiti, Morocco, Paraguay, and some Russian areas. Also, historical zone data corrections for numerous places. Also, update the time zone abbreviation files for recent changes in Russia and elsewhere: CHOT, GET, IRKT, KGT, KRAT, MAGT, MAWT, MSK, NOVT, OMST, TKT, VLAT, WST, YAKT, YEKT now follow their current meanings, and VOLT (Europe/Volgograd) and MIST (Antarctica/Macquarie) are added to the default abbreviations list.
E.61. Release 9.1.8 Release date: 2013-02-07 This release contains a variety of fixes from 9.1.7. For information about new features in the 9.1 major release, see Section E.69.
E.61.1. Migration to Version 9.1.8 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.6, see Section E.63.
E.61.2. Changes •
Prevent execution of enum_recv from SQL (Tom Lane) The function was misdeclared, allowing a simple SQL command to crash the server. In principle an attacker might be able to use it to examine the contents of server memory. Our thanks to Sumit Soni (via Secunia SVCRP) for reporting this issue. (CVE-2013-0255)
•
Fix multiple problems in detection of when a consistent database state has been reached during WAL replay (Fujii Masao, Heikki Linnakangas, Simon Riggs, Andres Freund)
•
Update minimum recovery point when truncating a relation file (Heikki Linnakangas) Once data has been discarded, it’s no longer safe to stop recovery at an earlier point in the timeline.
•
Fix recycling of WAL segments after changing recovery target timeline (Heikki Linnakangas)
2184
Appendix E. Release Notes •
Fix missing cancellations in hot standby mode (Noah Misch, Simon Riggs) The need to cancel conflicting hot-standby queries would sometimes be missed, allowing those queries to see inconsistent data.
•
Prevent recovery pause feature from pausing before users can connect (Tom Lane)
•
Fix SQL grammar to allow subscripting or field selection from a sub-SELECT result (Tom Lane)
•
Fix performance problems with autovacuum truncation in busy workloads (Jan Wieck) Truncation of empty pages at the end of a table requires exclusive lock, but autovacuum was coded to fail (and release the table lock) when there are conflicting lock requests. Under load, it is easily possible that truncation would never occur, resulting in table bloat. Fix by performing a partial truncation, releasing the lock, then attempting to re-acquire the lock and continue. This fix also greatly reduces the average time before autovacuum releases the lock after a conflicting request arrives.
•
Protect against race conditions when scanning pg_tablespace (Stephen Frost, Tom Lane) CREATE DATABASE and DROP DATABASE could misbehave if there were concurrent updates of pg_tablespace entries.
•
Prevent DROP OWNED from trying to drop whole databases or tablespaces (Álvaro Herrera) For safety, ownership of these objects must be reassigned, not dropped.
•
Fix error in vacuum_freeze_table_age implementation (Andres Freund) In installations that have existed for more than vacuum_freeze_min_age transactions, this mistake prevented autovacuum from using partial-table scans, so that a full-table scan would always happen instead.
•
Prevent misbehavior when a RowExpr or XmlExpr is parse-analyzed twice (Andres Freund, Tom Lane) This mistake could be user-visible in contexts such as CREATE TABLE LIKE INCLUDING INDEXES.
•
Improve defenses against integer overflow in hashtable sizing calculations (Jeff Davis)
•
Fix failure to ignore leftover temporary tables after a server crash (Tom Lane)
•
Reject out-of-range dates in to_date() (Hitoshi Harada)
•
Fix pg_extension_config_dump() to handle extension-update cases properly (Tom Lane) This function will now replace any existing entry for the target table, making it usable in extension update scripts.
•
Fix PL/Python’s handling of functions used as triggers on multiple tables (Andres Freund)
•
Ensure that non-ASCII prompt strings are translated to the correct code page on Windows (Alexander Law, Noah Misch) This bug affected psql and some other client programs.
•
Fix possible crash in psql’s \? command when not connected to a database (Meng Qingzhong)
•
Fix possible error if a relation file is removed while pg_basebackup is running (Heikki Linnakangas)
•
Make pg_dump exclude data of unlogged tables when running on a hot-standby server (Magnus Hagander) This would fail anyway because the data is not available on the standby server, so it seems most convenient to assume --no-unlogged-table-data automatically.
2185
Appendix E. Release Notes •
Fix pg_upgrade to deal with invalid indexes safely (Bruce Momjian)
•
Fix one-byte buffer overrun in libpq’s PQprintTuples (Xi Wang) This ancient function is not used anywhere by PostgreSQL itself, but it might still be used by some client code.
•
Make ecpglib use translated messages properly (Chen Huajun)
•
Properly install ecpg_compat and pgtypes libraries on MSVC (Jiang Guiqing)
•
Include our version of isinf() in libecpg if it’s not provided by the system (Jiang Guiqing)
•
Rearrange configure’s tests for supplied functions so it is not fooled by bogus exports from libedit/libreadline (Christoph Berg)
•
Ensure Windows build number increases over time (Magnus Hagander)
•
Make pgxs build executables with the right .exe suffix when cross-compiling for Windows (Zoltan Boszormenyi)
•
Add new timezone abbreviation FET (Tom Lane) This is now used in some eastern-European time zones.
E.62. Release 9.1.7 Release date: 2012-12-06 This release contains a variety of fixes from 9.1.6. For information about new features in the 9.1 major release, see Section E.69.
E.62.1. Migration to Version 9.1.7 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.6, see Section E.63.
E.62.2. Changes •
Fix multiple bugs associated with CREATE INDEX CONCURRENTLY (Andres Freund, Tom Lane) Fix CREATE INDEX CONCURRENTLY to use in-place updates when changing the state of an index’s pg_index row. This prevents race conditions that could cause concurrent sessions to miss updating the target index, thus resulting in corrupt concurrently-created indexes. Also, fix various other operations to ensure that they ignore invalid indexes resulting from a failed CREATE INDEX CONCURRENTLY command. The most important of these is VACUUM, because an auto-vacuum could easily be launched on the table before corrective action can be taken to fix or remove the invalid index.
•
Fix buffer locking during WAL replay (Tom Lane) The WAL replay code was insufficiently careful about locking buffers when replaying WAL records that affect more than one page. This could result in hot standby queries transiently seeing inconsistent states, resulting in wrong answers or unexpected failures.
2186
Appendix E. Release Notes •
Fix an error in WAL generation logic for GIN indexes (Tom Lane) This could result in index corruption, if a torn-page failure occurred.
•
Properly remove startup process’s virtual XID lock when promoting a hot standby server to normal running (Simon Riggs) This oversight could prevent subsequent execution of certain operations such as CREATE INDEX CONCURRENTLY.
Prevent the postmaster from launching new child processes after it’s received a shutdown signal (Tom Lane) This mistake could result in shutdown taking longer than it should, or even never completing at all without additional user action.
•
Avoid corruption of internal hash tables when out of memory (Hitoshi Harada)
•
Prevent file descriptors for dropped tables from being held open past transaction end (Tom Lane) This should reduce problems with long-since-dropped tables continuing to occupy disk space.
•
Prevent database-wide crash and restart when a new child process is unable to create a pipe for its latch (Tom Lane) Although the new process must fail, there is no good reason to force a database-wide restart, so avoid that. This improves robustness when the kernel is nearly out of file descriptors.
•
Fix planning of non-strict equivalence clauses above outer joins (Tom Lane) The planner could derive incorrect constraints from a clause equating a non-strict construct to something else, for example WHERE COALESCE(foo, 0) = 0 when foo is coming from the nullable side of an outer join.
•
Fix SELECT DISTINCT with index-optimized MIN/MAX on an inheritance tree (Tom Lane) The planner would fail with “failed to re-find MinMaxAggInfo record” given this combination of factors.
•
Improve planner’s ability to prove exclusion constraints from equivalence classes (Tom Lane)
•
Fix partial-row matching in hashed subplans to handle cross-type cases correctly (Tom Lane) This affects multicolumn NOT IN subplans, such as WHERE (a, b) NOT IN (SELECT x, y FROM ...) when for instance b and y are int4 and int8 respectively. This mistake led to wrong answers or crashes depending on the specific datatypes involved.
•
Acquire buffer lock when re-fetching the old tuple for an AFTER ROW UPDATE/DELETE trigger (Andres Freund) In very unusual circumstances, this oversight could result in passing incorrect data to a trigger WHEN condition, or to the precheck logic for a foreign-key enforcement trigger. That could result in a crash, or in an incorrect decision about whether to fire the trigger.
•
Fix ALTER COLUMN TYPE to handle inherited check constraints properly (Pavan Deolasee) This worked correctly in pre-8.4 releases, and now works correctly in 8.4 and later.
•
Fix ALTER EXTENSION SET SCHEMA’s failure to move some subsidiary objects into the new schema (Álvaro Herrera, Dimitri Fontaine)
•
Fix REASSIGN OWNED to handle grants on tablespaces (Álvaro Herrera)
•
Ignore incorrect pg_attribute entries for system columns for views (Tom Lane)
2187
Appendix E. Release Notes Views do not have any system columns. However, we forgot to remove such entries when converting a table to a view. That’s fixed properly for 9.3 and later, but in previous branches we need to defend against existing mis-converted views. •
Fix rule printing to dump INSERT INTO table DEFAULT VALUES correctly (Tom Lane)
•
Guard against stack overflow when there are too many UNION/INTERSECT/EXCEPT clauses in a query (Tom Lane)
•
Prevent platform-dependent failures when dividing the minimum possible integer value by -1 (Xi Wang, Tom Lane)
•
Fix possible access past end of string in date parsing (Hitoshi Harada)
•
Fix failure to advance XID epoch if XID wraparound happens during a checkpoint and wal_level is hot_standby (Tom Lane, Andres Freund) While this mistake had no particular impact on PostgreSQL itself, it was bad for applications that rely on txid_current() and related functions: the TXID value would appear to go backwards.
•
Fix display of pg_stat_replication.sync_state at a page boundary (Kyotaro Horiguchi)
•
Produce an understandable error message if the length of the path name for a Unix-domain socket exceeds the platform-specific limit (Tom Lane, Andrew Dunstan) Formerly, this would result in something quite unhelpful, such as “Non-recoverable failure in name resolution”.
•
Fix memory leaks when sending composite column values to the client (Tom Lane)
•
Make pg_ctl more robust about reading the postmaster.pid file (Heikki Linnakangas) Fix race conditions and possible file descriptor leakage.
•
Fix possible crash in psql if incorrectly-encoded data is presented and the client_encoding setting is a client-only encoding, such as SJIS (Jiang Guiqing)
•
Make pg_dump dump SEQUENCE SET items in the data not pre-data section of the archive (Tom Lane) This change fixes dumping of sequences that are marked as extension configuration tables.
•
Fix bugs in the restore.sql script emitted by pg_dump in tar output format (Tom Lane) The script would fail outright on tables whose names include upper-case characters. Also, make the script capable of restoring data in --inserts mode as well as the regular COPY mode.
•
Fix pg_restore to accept POSIX-conformant tar files (Brian Weaver, Tom Lane) The original coding of pg_dump’s tar output mode produced files that are not fully conformant with the POSIX standard. This has been corrected for version 9.3. This patch updates previous branches so that they will accept both the incorrect and the corrected formats, in hopes of avoiding compatibility problems when 9.3 comes out.
•
Fix tar files emitted by pg_basebackup to be POSIX conformant (Brian Weaver, Tom Lane)
•
Fix pg_resetxlog to locate postmaster.pid correctly when given a relative path to the data directory (Tom Lane) This mistake could lead to pg_resetxlog not noticing that there is an active postmaster using the data directory.
•
Fix libpq’s lo_import() and lo_export() functions to report file I/O errors properly (Tom Lane)
•
Fix ecpg’s processing of nested structure pointer variables (Muhammad Usama)
2188
Appendix E. Release Notes •
Fix ecpg’s ecpg_get_data function to handle arrays properly (Michael Meskes)
•
Make contrib/pageinspect’s btree page inspection functions take buffer locks while examining pages (Tom Lane)
•
Ensure that make install for an extension creates the extension installation directory (Cédric Villemain) Previously, this step was missed if MODULEDIR was set in the extension’s Makefile.
•
Fix pgxs support for building loadable modules on AIX (Tom Lane) Building modules outside the original source tree didn’t work on AIX.
•
Update time zone data files to tzdata release 2012j for DST law changes in Cuba, Israel, Jordan, Libya, Palestine, Western Samoa, and portions of Brazil.
E.63. Release 9.1.6 Release date: 2012-09-24 This release contains a variety of fixes from 9.1.5. For information about new features in the 9.1 major release, see Section E.69.
E.63.1. Migration to Version 9.1.6 A dump/restore is not required for those running 9.1.X. However, you may need to perform REINDEX operations to recover from the effects of the data corruption bug described in the first changelog item below. Also, if you are upgrading from a version earlier than 9.1.4, see Section E.65.
E.63.2. Changes •
Fix persistence marking of shared buffers during WAL replay (Jeff Davis) This mistake can result in buffers not being written out during checkpoints, resulting in data corruption if the server later crashes without ever having written those buffers. Corruption can occur on any server following crash recovery, but it is significantly more likely to occur on standby slave servers since those perform much more WAL replay. There is a low probability of corruption of btree and GIN indexes. There is a much higher probability of corruption of table “visibility maps”. Fortunately, visibility maps are non-critical data in 9.1, so the worst consequence of such corruption in 9.1 installations is transient inefficiency of vacuuming. Table data proper cannot be corrupted by this bug. While no index corruption due to this bug is known to have occurred in the field, as a precautionary measure it is recommended that production installations REINDEX all btree and GIN indexes at a convenient time after upgrading to 9.1.6. Also, if you intend to do an in-place upgrade to 9.2.X, before doing so it is recommended to perform a VACUUM of all tables while having vacuum_freeze_table_age set to zero. This will ensure that any lingering wrong data in the visibility maps is corrected before 9.2.X can depend on
2189
Appendix E. Release Notes it. vacuum_cost_delay can be adjusted to reduce the performance impact of vacuuming, while causing it to take longer to finish. •
Fix planner’s assignment of executor parameters, and fix executor’s rescan logic for CTE plan nodes (Tom Lane) These errors could result in wrong answers from queries that scan the same WITH subquery multiple times.
•
Fix misbehavior when default_transaction_isolation is set to serializable (Kevin Grittner, Tom Lane, Heikki Linnakangas) Symptoms include crashes at process start on Windows, and crashes in hot standby operation.
•
Improve selectivity estimation for text search queries involving prefixes, i.e. word:* patterns (Tom Lane)
•
Improve page-splitting decisions in GiST indexes (Alexander Korotkov, Robert Haas, Tom Lane) Multi-column GiST indexes might suffer unexpected bloat due to this error.
•
Fix cascading privilege revoke to stop if privileges are still held (Tom Lane) If we revoke a grant option from some role X , but X still holds that option via a grant from someone else, we should not recursively revoke the corresponding privilege from role(s) Y that X had granted it to.
•
Disallow extensions from containing the schema they are assigned to (Thom Brown) This situation creates circular dependencies that confuse pg_dump and probably other things. It’s confusing for humans too, so disallow it.
•
Improve error messages for Hot Standby misconfiguration errors (Gurjeet Singh)
•
Make configure probe for mbstowcs_l (Tom Lane) This fixes build failures on some versions of AIX.
•
Fix handling of SIGFPE when PL/Perl is in use (Andres Freund) Perl resets the process’s SIGFPE handler to SIG_IGN, which could result in crashes later on. Restore the normal Postgres signal handler after initializing PL/Perl.
•
Prevent PL/Perl from crashing if a recursive PL/Perl function is redefined while being executed (Tom Lane)
•
Work around possible misoptimization in PL/Perl (Tom Lane) Some Linux distributions contain an incorrect version of pthread.h that results in incorrect compiled code in PL/Perl, leading to crashes if a PL/Perl function calls another one that throws an error.
•
Fix bugs in contrib/pg_trgm’s LIKE pattern analysis code (Fujii Masao) LIKE queries using a trigram index could produce wrong results if the pattern contained LIKE
escape characters. •
Fix pg_upgrade’s handling of line endings on Windows (Andrew Dunstan) Previously, pg_upgrade might add or remove carriage returns in places such as function bodies.
•
On Windows, make pg_upgrade use backslash path separators in the scripts it emits (Andrew Dunstan)
•
Remove unnecessary dependency on pg_config from pg_upgrade (Peter Eisentraut)
•
Update time zone data files to tzdata release 2012f for DST law changes in Fiji
2190
Appendix E. Release Notes
E.64. Release 9.1.5 Release date: 2012-08-17 This release contains a variety of fixes from 9.1.4. For information about new features in the 9.1 major release, see Section E.69.
E.64.1. Migration to Version 9.1.5 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.4, see Section E.65.
E.64.2. Changes •
Prevent access to external files/URLs via XML entity references (Noah Misch, Tom Lane) xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn’t get returned directly to the user, portions of it could be exposed in error messages if the data didn’t parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. (CVE-20123489)
•
Prevent access to external files/URLs via contrib/xml2’s xslt_process() (Peter Eisentraut) libxslt offers the ability to read and write both files and URLs through stylesheet commands, thus allowing unprivileged database users to both read and write data with the privileges of the database server. Disable that through proper use of libxslt’s security options. (CVE-2012-3488) Also, remove xslt_process()’s ability to fetch documents and stylesheets from external files/URLs. While this was a documented “feature”, it was long regarded as a bad idea. The fix for CVE-2012-3489 broke that capability, and rather than expend effort on trying to fix it, we’re just going to summarily remove it.
•
Prevent too-early recycling of btree index pages (Noah Misch) When we allowed read-only transactions to skip assigning XIDs, we introduced the possibility that a deleted btree page could be recycled while a read-only transaction was still in flight to it. This would result in incorrect index search results. The probability of such an error occurring in the field seems very low because of the timing requirements, but nonetheless it should be fixed.
•
Fix crash-safety bug with newly-created-or-reset sequences (Tom Lane) If ALTER SEQUENCE was executed on a freshly created or reset sequence, and then precisely one nextval() call was made on it, and then the server crashed, WAL replay would restore the sequence to a state in which it appeared that no nextval() had been done, thus allowing the first sequence value to be returned again by the next nextval() call. In particular this could manifest for serial columns, since creation of a serial column’s sequence includes an ALTER SEQUENCE OWNED BY step.
•
Fix race condition in enum-type value comparisons (Robert Haas, Tom Lane) Comparisons could fail when encountering an enum value added since the current query started.
•
Fix txid_current() to report the correct epoch when not in hot standby (Heikki Linnakangas) This fixes a regression introduced in the previous minor release.
2191
Appendix E. Release Notes •
Prevent selection of unsuitable replication connections as the synchronous standby (Fujii Masao) The master might improperly choose pseudo-servers such as pg_receivexlog or pg_basebackup as the synchronous standby, and then wait indefinitely for them.
•
Fix bug in startup of Hot Standby when a master transaction has many subtransactions (Andres Freund) This mistake led to failures reported as “out-of-order XID insertion in KnownAssignedXids”.
•
Ensure the backup_label file is fsync’d after pg_start_backup() (Dave Kerr)
•
Fix timeout handling in walsender processes (Tom Lane) WAL sender background processes neglected to establish a SIGALRM handler, meaning they would wait forever in some corner cases where a timeout ought to happen.
•
Wake walsenders after each background flush by walwriter (Andres Freund, Simon Riggs) This greatly reduces replication delay when the workload contains only asynchronously-committed transactions.
•
Fix LISTEN/NOTIFY to cope better with I/O problems, such as out of disk space (Tom Lane) After a write failure, all subsequent attempts to send more NOTIFY messages would fail with messages like “Could not read from file "pg_notify/nnnn" at offset nnnnn: Success”.
•
Only allow autovacuum to be auto-canceled by a directly blocked process (Tom Lane) The original coding could allow inconsistent behavior in some cases; in particular, an autovacuum could get canceled after less than deadlock_timeout grace period.
•
Improve logging of autovacuum cancels (Robert Haas)
•
Fix log collector so that log_truncate_on_rotation works during the very first log rotation after server start (Tom Lane)
•
Fix WITH attached to a nested set operation (UNION/INTERSECT/EXCEPT) (Tom Lane)
•
Ensure that a whole-row reference to a subquery doesn’t include any extra GROUP BY or ORDER BY columns (Tom Lane)
•
Fix dependencies generated during ALTER TABLE ... ADD CONSTRAINT USING INDEX (Tom Lane) This command left behind a redundant pg_depend entry for the index, which could confuse later operations, notably ALTER TABLE ... ALTER COLUMN TYPE on one of the indexed columns.
•
Fix REASSIGN OWNED to work on extensions (Alvaro Herrera)
•
Disallow copying whole-row references in CHECK constraints and index definitions during CREATE TABLE (Tom Lane) This situation can arise in CREATE TABLE with LIKE or INHERITS. The copied whole-row variable was incorrectly labeled with the row type of the original table not the new one. Rejecting the case seems reasonable for LIKE, since the row types might well diverge later. For INHERITS we should ideally allow it, with an implicit coercion to the parent table’s row type; but that will require more work than seems safe to back-patch.
•
Fix memory leak in ARRAY(SELECT ...) subqueries (Heikki Linnakangas, Tom Lane)
•
Fix planner to pass correct collation to operator selectivity estimators (Tom Lane) This was not previously required by any core selectivity estimation function, but third-party code might need it.
•
Fix extraction of common prefixes from regular expressions (Tom Lane)
2192
Appendix E. Release Notes The code could get confused by quantified parenthesized subexpressions, such as ^(foo)?bar. This would lead to incorrect index optimization of searches for such patterns. •
Fix bugs with parsing signed hh:mm and hh:mm:ss fields in interval constants (Amit Kapila, Tom Lane)
•
Fix pg_dump to better handle views containing partial GROUP BY lists (Tom Lane) A view that lists only a primary key column in GROUP BY, but uses other table columns as if they were grouped, gets marked as depending on the primary key. Improper handling of such primary key dependencies in pg_dump resulted in poorly-ordered dumps, which at best would be inefficient to restore and at worst could result in outright failure of a parallel pg_restore run.
•
In PL/Perl, avoid setting UTF8 flag when in SQL_ASCII encoding (Alex Hunsaker, Kyotaro Horiguchi, Alvaro Herrera)
•
Use Postgres’ encoding conversion functions, not Python’s, when converting a Python Unicode string to the server encoding in PL/Python (Jan Urbanski) This avoids some corner-case problems, notably that Python doesn’t support all the encodings Postgres does. A notable functional change is that if the server encoding is SQL_ASCII, you will get the UTF-8 representation of the string; formerly, any non-ASCII characters in the string would result in an error.
•
Fix mapping of PostgreSQL encodings to Python encodings in PL/Python (Jan Urbanski)
•
Report errors properly in contrib/xml2’s xslt_process() (Tom Lane)
•
Update time zone data files to tzdata release 2012e for DST law changes in Morocco and Tokelau
E.65. Release 9.1.4 Release date: 2012-06-04 This release contains a variety of fixes from 9.1.3. For information about new features in the 9.1 major release, see Section E.69.
E.65.1. Migration to Version 9.1.4 A dump/restore is not required for those running 9.1.X. However, if you use the citext data type, and you upgraded from a previous major release by running pg_upgrade, you should run CREATE EXTENSION citext FROM unpackaged to avoid collation-related failures in citext operations. The same is necessary if you restore a dump from a pre-9.1 database that contains an instance of the citext data type. If you’ve already run the CREATE EXTENSION command before upgrading to 9.1.4, you will instead need to do manual catalog updates as explained in the third changelog item below. Also, if you are upgrading from a version earlier than 9.1.2, see Section E.67.
E.65.2. Changes •
Fix incorrect password transformation in contrib/pgcrypto’s DES crypt() function (Solar Designer)
2193
Appendix E. Release Notes If a password string contained the byte value 0x80, the remainder of the password was ignored, causing the password to be much weaker than it appeared. With this fix, the rest of the string is properly included in the DES hash. Any stored password values that are affected by this bug will thus no longer match, so the stored values may need to be updated. (CVE-2012-2143) •
Ignore SECURITY DEFINER and SET attributes for a procedural language’s call handler (Tom Lane) Applying such attributes to a call handler could crash the server. (CVE-2012-2655)
•
Make contrib/citext’s upgrade script fix collations of citext arrays and domains over citext (Tom Lane) Release 9.1.2 provided a fix for collations of citext columns and indexes in databases upgraded or reloaded from pre-9.1 installations, but that fix was incomplete: it neglected to handle arrays and domains over citext. This release extends the module’s upgrade script to handle these cases. As before, if you have already run the upgrade script, you’ll need to run the collation update commands by hand instead. See the 9.1.2 release notes for more information about doing this.
•
Allow numeric timezone offsets in timestamp input to be up to 16 hours away from UTC (Tom Lane) Some historical time zones have offsets larger than 15 hours, the previous limit. This could result in dumped data values being rejected during reload.
•
Fix timestamp conversion to cope when the given time is exactly the last DST transition time for the current timezone (Tom Lane) This oversight has been there a long time, but was not noticed previously because most DST-using zones are presumed to have an indefinite sequence of future DST transitions.
•
Fix text to name and char to name casts to perform string truncation correctly in multibyte encodings (Karl Schnaitter)
•
Fix memory copying bug in to_tsquery() (Heikki Linnakangas)
•
Ensure txid_current() reports the correct epoch when executed in hot standby (Simon Riggs)
•
Fix planner’s handling of outer PlaceHolderVars within subqueries (Tom Lane) This bug concerns sub-SELECTs that reference variables coming from the nullable side of an outer join of the surrounding query. In 9.1, queries affected by this bug would fail with “ERROR: Upper-level PlaceHolderVar found where not expected”. But in 9.0 and 8.4, you’d silently get possibly-wrong answers, since the value transmitted into the subquery wouldn’t go to null when it should.
•
Fix planning of UNION ALL subqueries with output columns that are not simple variables (Tom Lane) Planning of such cases got noticeably worse in 9.1 as a result of a misguided fix for “MergeAppend child’s targetlist doesn’t match MergeAppend” errors. Revert that fix and do it another way.
•
Fix slow session startup when pg_attribute is very large (Tom Lane) If pg_attribute exceeds one-fourth of shared_buffers, cache rebuilding code that is sometimes needed during session start would trigger the synchronized-scan logic, causing it to take many times longer than normal. The problem was particularly acute if many new sessions were starting at once.
•
Ensure sequential scans check for query cancel reasonably often (Merlin Moncure) A scan encountering many consecutive pages that contain no live tuples would not respond to interrupts meanwhile.
2194
Appendix E. Release Notes •
Ensure the Windows implementation of PGSemaphoreLock() clears ImmediateInterruptOK before returning (Tom Lane) This oversight meant that a query-cancel interrupt received later in the same query could be accepted at an unsafe time, with unpredictable but not good consequences.
•
Show whole-row variables safely when printing views or rules (Abbas Butt, Tom Lane) Corner cases involving ambiguous names (that is, the name could be either a table or column name of the query) were printed in an ambiguous way, risking that the view or rule would be interpreted differently after dump and reload. Avoid the ambiguous case by attaching a no-op cast.
•
Fix COPY FROM to properly handle null marker strings that correspond to invalid encoding (Tom Lane) A null marker string such as E’\\0’ should work, and did work in the past, but the case got broken in 8.4.
Fix PREPARE TRANSACTION to work correctly in the presence of advisory locks (Tom Lane) Historically, PREPARE TRANSACTION has simply ignored any session-level advisory locks the session holds, but this case was accidentally broken in 9.1.
•
Fix truncation of unlogged tables (Robert Haas)
•
Ignore missing schemas during non-interactive assignments of search_path (Tom Lane) This re-aligns 9.1’s behavior with that of older branches. Previously 9.1 would throw an error for nonexistent schemas mentioned in search_path settings obtained from places such as ALTER DATABASE SET.
•
Fix bugs with temporary or transient tables used in extension scripts (Tom Lane) This includes cases such as a rewriting ALTER TABLE within an extension update script, since that uses a transient table behind the scenes.
•
Ensure autovacuum worker processes perform stack depth checking properly (Heikki Linnakangas) Previously, infinite recursion in a function invoked by auto-ANALYZE could crash worker processes.
•
Fix logging collector to not lose log coherency under high load (Andrew Dunstan) The collector previously could fail to reassemble large messages if it got too busy.
•
Fix logging collector to ensure it will restart file rotation after receiving SIGHUP (Tom Lane)
•
Fix “too many LWLocks taken” failure in GiST indexes (Heikki Linnakangas)
•
Fix WAL replay logic for GIN indexes to not fail if the index was subsequently dropped (Tom Lane)
•
Correctly detect SSI conflicts of prepared transactions after a crash (Dan Ports)
•
Avoid synchronous replication delay when committing a transaction that only modified temporary tables (Heikki Linnakangas) In such a case the transaction’s commit record need not be flushed to standby servers, but some of the code didn’t know that and waited for it to happen anyway.
•
Fix error handling in pg_basebackup (Thomas Ogrisegg, Fujii Masao)
•
Fix walsender to not go into a busy loop if connection is terminated (Fujii Masao)
•
Fix memory leak in PL/pgSQL’s RETURN NEXT command (Joe Conway)
2195
Appendix E. Release Notes •
Fix PL/pgSQL’s GET DIAGNOSTICS command when the target is the function’s first variable (Tom Lane)
•
Ensure that PL/Perl package-qualifies the _TD variable (Alex Hunsaker) This bug caused trigger invocations to fail when they are nested within a function invocation that changes the current package.
•
Fix PL/Python functions returning composite types to accept a string for their result value (Jan Urbanski) This case was accidentally broken by the 9.1 additions to allow a composite result value to be supplied in other formats, such as dictionaries.
•
Fix potential access off the end of memory in psql’s expanded display (\x) mode (Peter Eisentraut)
•
Fix several performance problems in pg_dump when the database contains many objects (Jeff Janes, Tom Lane) pg_dump could get very slow if the database contained many schemas, or if many objects are in dependency loops, or if there are many owned sequences.
•
Fix memory and file descriptor leaks in pg_restore when reading a directory-format archive (Peter Eisentraut)
•
Fix pg_upgrade for the case that a database stored in a non-default tablespace contains a table in the cluster’s default tablespace (Bruce Momjian)
•
In ecpg, fix rare memory leaks and possible overwrite of one byte after the sqlca_t structure (Peter Eisentraut)
•
Fix contrib/dblink’s dblink_exec() to not leak temporary database connections upon error (Tom Lane)
•
Fix contrib/dblink to report the correct connection name in error messages (Kyotaro Horiguchi)
•
Fix contrib/vacuumlo to use multiple transactions when dropping many large objects (Tim Lewis, Robert Haas, Tom Lane) This change avoids exceeding max_locks_per_transaction when many objects need to be dropped. The behavior can be adjusted with the new -l (limit) option.
•
Update time zone data files to tzdata release 2012c for DST law changes in Antarctica, Armenia, Chile, Cuba, Falkland Islands, Gaza, Haiti, Hebron, Morocco, Syria, and Tokelau Islands; also historical corrections for Canada.
E.66. Release 9.1.3 Release date: 2012-02-27 This release contains a variety of fixes from 9.1.2. For information about new features in the 9.1 major release, see Section E.69.
E.66.1. Migration to Version 9.1.3 A dump/restore is not required for those running 9.1.X. However, if you are upgrading from a version earlier than 9.1.2, see Section E.67.
2196
Appendix E. Release Notes
E.66.2. Changes •
Require execute permission on the trigger function for CREATE TRIGGER (Robert Haas) This missing check could allow another user to execute a trigger function with forged input data, by installing it on a table he owns. This is only of significance for trigger functions marked SECURITY DEFINER, since otherwise trigger functions run as the table owner anyway. (CVE-2012-0866)
•
Remove arbitrary limitation on length of common name in SSL certificates (Heikki Linnakangas) Both libpq and the server truncated the common name extracted from an SSL certificate at 32 bytes. Normally this would cause nothing worse than an unexpected verification failure, but there are some rather-implausible scenarios in which it might allow one certificate holder to impersonate another. The victim would have to have a common name exactly 32 bytes long, and the attacker would have to persuade a trusted CA to issue a certificate in which the common name has that string as a prefix. Impersonating a server would also require some additional exploit to redirect client connections. (CVE-2012-0867)
•
Convert newlines to spaces in names written in pg_dump comments (Robert Haas) pg_dump was incautious about sanitizing object names that are emitted within SQL comments in its output script. A name containing a newline would at least render the script syntactically incorrect. Maliciously crafted object names could present a SQL injection risk when the script is reloaded. (CVE-2012-0868)
•
Fix btree index corruption from insertions concurrent with vacuuming (Tom Lane) An index page split caused by an insertion could sometimes cause a concurrently-running VACUUM to miss removing index entries that it should remove. After the corresponding table rows are removed, the dangling index entries would cause errors (such as “could not read block N in file ...”) or worse, silently wrong query results after unrelated rows are re-inserted at the now-free table locations. This bug has been present since release 8.2, but occurs so infrequently that it was not diagnosed until now. If you have reason to suspect that it has happened in your database, reindexing the affected index will fix things.
•
Fix transient zeroing of shared buffers during WAL replay (Tom Lane) The replay logic would sometimes zero and refill a shared buffer, so that the contents were transiently invalid. In hot standby mode this can result in a query that’s executing in parallel seeing garbage data. Various symptoms could result from that, but the most common one seems to be “invalid memory alloc request size”.
•
Fix handling of data-modifying WITH subplans in READ COMMITTED rechecking (Tom Lane) A WITH clause containing INSERT/UPDATE/DELETE would crash if the parent UPDATE or DELETE command needed to be re-evaluated at one or more rows due to concurrent updates in READ COMMITTED mode.
•
Fix corner case in SSI transaction cleanup (Dan Ports) When finishing up a read-write serializable transaction, a crash could occur if all remaining active serializable transactions are read-only.
•
Fix postmaster to attempt restart after a hot-standby crash (Tom Lane) A logic error caused the postmaster to terminate, rather than attempt to restart the cluster, if any backend process crashed while operating in hot standby mode.
•
Fix CLUSTER/VACUUM FULL handling of toast values owned by recently-updated rows (Tom Lane)
2197
Appendix E. Release Notes This oversight could lead to “duplicate key value violates unique constraint” errors being reported against the toast table’s index during one of these commands. •
Update per-column permissions, not only per-table permissions, when changing table owner (Tom Lane) Failure to do this meant that any previously granted column permissions were still shown as having been granted by the old owner. This meant that neither the new owner nor a superuser could revoke the now-untraceable-to-table-owner permissions.
•
Support foreign data wrappers and foreign servers in REASSIGN OWNED (Alvaro Herrera) This command failed with “unexpected classid” errors if it needed to change the ownership of any such objects.
•
Allow non-existent values for some settings in ALTER USER/DATABASE SET (Heikki Linnakangas) Allow default_text_search_config, default_tablespace, and temp_tablespaces to be set to names that are not known. This is because they might be known in another database where the setting is intended to be used, or for the tablespace cases because the tablespace might not be created yet. The same issue was previously recognized for search_path, and these settings now act like that one.
•
Fix “unsupported node type” error caused by COLLATE in an INSERT expression (Tom Lane)
•
Avoid crashing when we have problems deleting table files post-commit (Tom Lane) Dropping a table should lead to deleting the underlying disk files only after the transaction commits. In event of failure then (for instance, because of wrong file permissions) the code is supposed to just emit a warning message and go on, since it’s too late to abort the transaction. This logic got broken as of release 8.4, causing such situations to result in a PANIC and an unrestartable database.
•
Recover from errors occurring during WAL replay of DROP TABLESPACE (Tom Lane) Replay will attempt to remove the tablespace’s directories, but there are various reasons why this might fail (for example, incorrect ownership or permissions on those directories). Formerly the replay code would panic, rendering the database unrestartable without manual intervention. It seems better to log the problem and continue, since the only consequence of failure to remove the directories is some wasted disk space.
•
Fix race condition in logging AccessExclusiveLocks for hot standby (Simon Riggs) Sometimes a lock would be logged as being held by “transaction zero”. This is at least known to produce assertion failures on slave servers, and might be the cause of more serious problems.
•
Track the OID counter correctly during WAL replay, even when it wraps around (Tom Lane) Previously the OID counter would remain stuck at a high value until the system exited replay mode. The practical consequences of that are usually nil, but there are scenarios wherein a standby server that’s been promoted to master might take a long time to advance the OID counter to a reasonable value once values are needed.
•
Prevent emitting misleading “consistent recovery state reached” log message at the beginning of crash recovery (Heikki Linnakangas)
•
Fix initial value of pg_stat_replication.replay_location (Fujii Masao) Previously, the value shown would be wrong until at least one WAL record had been replayed.
•
Fix regular expression back-references with * attached (Tom Lane) Rather than enforcing an exact string match, the code would effectively accept any string that satisfies the pattern sub-expression referenced by the back-reference symbol.
2198
Appendix E. Release Notes A similar problem still afflicts back-references that are embedded in a larger quantified expression, rather than being the immediate subject of the quantifier. This will be addressed in a future PostgreSQL release. •
Fix recently-introduced memory leak in processing of inet/cidr values (Heikki Linnakangas) A patch in the December 2011 releases of PostgreSQL caused memory leakage in these operations, which could be significant in scenarios such as building a btree index on such a column.
•
Fix planner’s ability to push down index-expression restrictions through UNION ALL (Tom Lane) This type of optimization was inadvertently disabled by a fix for another problem in 9.1.2.
•
Fix planning of WITH clauses referenced in UPDATE/DELETE on an inherited table (Tom Lane) This bug led to “could not find plan for CTE” failures.
•
Fix GIN cost estimation to handle column IN (...) index conditions (Marti Raudsepp) This oversight would usually lead to crashes if such a condition could be used with a GIN index.
•
Prevent assertion failure when exiting a session with an open, failed transaction (Tom Lane) This bug has no impact on normal builds with asserts not enabled.
•
Fix dangling pointer after CREATE TABLE AS/SELECT INTO in a SQL-language function (Tom Lane) In most cases this only led to an assertion failure in assert-enabled builds, but worse consequences seem possible.
•
Avoid double close of file handle in syslogger on Windows (MauMau) Ordinarily this error was invisible, but it would cause an exception when running on a debug version of Windows.
•
Fix I/O-conversion-related memory leaks in plpgsql (Andres Freund, Jan Urbanski, Tom Lane) Certain operations would leak memory until the end of the current function.
•
Work around bug in perl’s SvPVutf8() function (Andrew Dunstan) This function crashes when handed a typeglob or certain read-only objects such as $^V. Make plperl avoid passing those to it.
•
In pg_dump, don’t dump contents of an extension’s configuration tables if the extension itself is not being dumped (Tom Lane)
•
Improve pg_dump’s handling of inherited table columns (Tom Lane) pg_dump mishandled situations where a child column has a different default expression than its parent column. If the default is textually identical to the parent’s default, but not actually the same (for instance, because of schema search path differences) it would not be recognized as different, so that after dump and restore the child would be allowed to inherit the parent’s default. Child columns that are NOT NULL where their parent is not could also be restored subtly incorrectly.
•
Fix pg_restore’s direct-to-database mode for INSERT-style table data (Tom Lane) Direct-to-database restores from archive files made with --inserts or --column-inserts options fail when using pg_restore from a release dated September or December 2011, as a result of an oversight in a fix for another problem. The archive file itself is not at fault, and text-mode output is okay.
•
Teach pg_upgrade to handle renaming of plpython’s shared library (Bruce Momjian) Upgrading a pre-9.1 database that included plpython would fail because of this oversight.
•
Allow pg_upgrade to process tables containing regclass columns (Bruce Momjian)
2199
Appendix E. Release Notes Since pg_upgrade now takes care to preserve pg_class OIDs, there was no longer any reason for this restriction. •
Make libpq ignore ENOTDIR errors when looking for an SSL client certificate file (Magnus Hagander) This allows SSL connections to be established, though without a certificate, even when the user’s home directory is set to something like /dev/null.
•
Fix some more field alignment issues in ecpg’s SQLDA area (Zoltan Boszormenyi)
•
Allow AT option in ecpg DEALLOCATE statements (Michael Meskes) The infrastructure to support this has been there for awhile, but through an oversight there was still an error check rejecting the case.
•
Do not use the variable name when defining a varchar structure in ecpg (Michael Meskes)
•
Fix contrib/auto_explain’s JSON output mode to produce valid JSON (Andrew Dunstan) The output used brackets at the top level, when it should have used braces.
•
Fix error in contrib/intarray’s int[] & int[] operator (Guillaume Lelarge) If the smallest integer the two input arrays have in common is 1, and there are smaller values in either array, then 1 would be incorrectly omitted from the result.
•
Fix error detection in contrib/pgcrypto’s encrypt_iv() and decrypt_iv() (Marko Kreen) These functions failed to report certain types of invalid-input errors, and would instead return random garbage values for incorrect input.
•
Fix one-byte buffer overrun in contrib/test_parser (Paul Guyot) The code would try to read one more byte than it should, which would crash in corner cases. Since contrib/test_parser is only example code, this is not a security issue in itself, but bad example code is still bad.
•
Use __sync_lock_test_and_set() for spinlocks on ARM, if available (Martin Pitt) This function replaces our previous use of the SWPB instruction, which is deprecated and not available on ARMv6 and later. Reports suggest that the old code doesn’t fail in an obvious way on recent ARM boards, but simply doesn’t interlock concurrent accesses, leading to bizarre failures in multiprocess operation.
•
Use -fexcess-precision=standard option when building with gcc versions that accept it (Andrew Dunstan) This prevents assorted scenarios wherein recent versions of gcc will produce creative results.
•
Allow use of threaded Python on FreeBSD (Chris Rees) Our configure script previously believed that this combination wouldn’t work; but FreeBSD fixed the problem, so remove that error check.
•
Allow MinGW builds to use standardly-named OpenSSL libraries (Tomasz Ostrowski)
E.67. Release 9.1.2 Release date: 2011-12-05 This release contains a variety of fixes from 9.1.1. For information about new features in the 9.1 major release, see Section E.69.
2200
Appendix E. Release Notes
E.67.1. Migration to Version 9.1.2 A dump/restore is not required for those running 9.1.X. However,
a
longstanding
error
was
discovered in the definition of the information_schema.referential_constraints view. If you rely on correct results from that view, you should replace its definition as explained in the first changelog item below. Also, if you use the citext data type, and you upgraded from a previous major release by running pg_upgrade, you should run CREATE EXTENSION citext FROM unpackaged to avoid collationrelated failures in citext operations. The same is necessary if you restore a dump from a pre9.1 database that contains an instance of the citext data type. If you’ve already run the CREATE EXTENSION command before upgrading to 9.1.2, you will instead need to do manual catalog updates as explained in the second changelog item.
E.67.2. Changes •
Fix bugs in information_schema.referential_constraints view (Tom Lane) This view was being insufficiently careful about matching the foreign-key constraint to the depended-on primary or unique key constraint. That could result in failure to show a foreign key constraint at all, or showing it multiple times, or claiming that it depends on a different constraint than the one it really does. Since the view definition is installed by initdb, merely upgrading will not fix the problem. If you need to fix this in an existing installation, you can (as a superuser) drop the information_schema schema then re-create it by sourcing SHAREDIR/information_schema.sql. (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) This must be repeated in each database to be fixed.
•
Make contrib/citext’s upgrade script fix collations of citext columns and indexes (Tom Lane) Existing citext columns and indexes aren’t correctly marked as being of a collatable data type during pg_upgrade from a pre-9.1 server, or when a pre-9.1 dump containing the citext type is loaded into a 9.1 server. That leads to operations on these columns failing with errors such as “could not determine which collation to use for string comparison”. This change allows them to be fixed by the same script that upgrades the citext module into a proper 9.1 extension during CREATE EXTENSION citext FROM unpackaged. If you have a previously-upgraded database that is suffering from this problem, and you already ran the CREATE EXTENSION command, you can manually run (as superuser) the UPDATE commands found at the end of SHAREDIR/extension/citext--unpackaged--1.0.sql. (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) There is no harm in doing this again if unsure.
•
Fix possible crash during UPDATE or DELETE that joins to the output of a scalar-returning function (Tom Lane) A crash could only occur if the target row had been concurrently updated, so this problem surfaced only intermittently.
•
Fix incorrect replay of WAL records for GIN index updates (Tom Lane) This could result in transiently failing to find index entries after a crash, or on a hot-standby server. The problem would be repaired by the next VACUUM of the index, however.
2201
Appendix E. Release Notes •
Fix TOAST-related data corruption during CREATE TABLE dest AS SELECT * FROM src or INSERT INTO dest SELECT * FROM src (Tom Lane) If a table has been modified by ALTER TABLE ADD COLUMN, attempts to copy its data verbatim to another table could produce corrupt results in certain corner cases. The problem can only manifest in this precise form in 8.4 and later, but we patched earlier versions as well in case there are other code paths that could trigger the same bug.
•
Fix possible failures during hot standby startup (Simon Riggs)
•
Start hot standby faster when initial snapshot is incomplete (Simon Riggs)
•
Fix race condition during toast table access from stale syscache entries (Tom Lane) The typical symptom was transient errors like “missing chunk number 0 for toast value NNNNN in pg_toast_2619”, where the cited toast table would always belong to a system catalog.
•
Track dependencies of functions on items used in parameter default expressions (Tom Lane) Previously, a referenced object could be dropped without having dropped or modified the function, leading to misbehavior when the function was used. Note that merely installing this update will not fix the missing dependency entries; to do that, you’d need to CREATE OR REPLACE each such function afterwards. If you have functions whose defaults depend on non-built-in objects, doing so is recommended.
•
Fix incorrect management of placeholder variables in nestloop joins (Tom Lane) This bug is known to lead to “variable not found in subplan target list” planner errors, and could possibly result in wrong query output when outer joins are involved.
•
Fix window functions that sort by expressions involving aggregates (Tom Lane) Previously these could fail with “could not find pathkey item to sort” planner errors.
Fix index matching for operators with both collatable and noncollatable inputs (Tom Lane) In 9.1.0, an indexable operator that has a non-collatable left-hand input type and a collatable righthand input type would not be recognized as matching the left-hand column’s index. An example is the hstore ? text operator.
•
Allow inlining of set-returning SQL functions with multiple OUT parameters (Tom Lane)
•
Don’t trust deferred-unique indexes for join removal (Tom Lane and Marti Raudsepp) A deferred uniqueness constraint might not hold intra-transaction, so assuming that it does could give incorrect query results.
•
Make DatumGetInetP() unpack inet datums that have a 1-byte header, and add a new macro, DatumGetInetPP(), that does not (Heikki Linnakangas) This change affects no core code, but might prevent crashes in add-on code that expects DatumGetInetP() to produce an unpacked datum as per usual convention.
•
Improve locale support in money type’s input and output (Tom Lane) Aside from not supporting all standard lc_monetary formatting options, the input and output functions were inconsistent, meaning there were locales in which dumped money values could not be re-read.
•
Don’t let transform_null_equals affect CASE foo WHEN NULL ... constructs (Heikki Linnakangas) transform_null_equals is only supposed to affect foo = NULL expressions written directly by the user, not equality checks generated internally by this form of CASE.
2202
Appendix E. Release Notes •
Change foreign-key trigger creation order to better support self-referential foreign keys (Tom Lane) For a cascading foreign key that references its own table, a row update will fire both the ON UPDATE trigger and the CHECK trigger as one event. The ON UPDATE trigger must execute first, else the CHECK will check a non-final state of the row and possibly throw an inappropriate error. However, the firing order of these triggers is determined by their names, which generally sort in creation order since the triggers have auto-generated names following the convention “RI_ConstraintTrigger_NNNN”. A proper fix would require modifying that convention, which we will do in 9.2, but it seems risky to change it in existing releases. So this patch just changes the creation order of the triggers. Users encountering this type of error should drop and re-create the foreign key constraint to get its triggers into the right order.
•
Fix IF EXISTS to work correctly in DROP OPERATOR FAMILY (Robert Haas)
•
Disallow dropping of an extension from within its own script (Tom Lane) This prevents odd behavior in case of incorrect management of extension dependencies.
•
Don’t mark auto-generated types as extension members (Robert Haas) Relation rowtypes and automatically-generated array types do not need to have their own extension membership entries in pg_depend, and creating such entries complicates matters for extension upgrades.
•
Cope with invalid pre-existing search_path settings during CREATE EXTENSION (Tom Lane)
•
Avoid floating-point underflow while tracking buffer allocation rate (Greg Matthews) While harmless in itself, on certain platforms this would result in annoying kernel log messages.
•
Prevent autovacuum transactions from running in serializable mode (Tom Lane) Autovacuum formerly used the cluster-wide default transaction isolation level, but there is no need for it to use anything higher than READ COMMITTED, and using SERIALIZABLE could result in unnecessary delays for other processes.
•
Ensure walsender processes respond promptly to SIGTERM (Magnus Hagander)
•
Exclude postmaster.opts from base backups (Magnus Hagander)
•
Preserve configuration file name and line number values when starting child processes under Windows (Tom Lane) Formerly, these would not be displayed correctly in the pg_settings view.
•
Fix incorrect field alignment in ecpg’s SQLDA area (Zoltan Boszormenyi)
•
Preserve blank lines within commands in psql’s command history (Robert Haas) The former behavior could cause problems if an empty line was removed from within a string literal, for example.
•
Avoid platform-specific infinite loop in pg_dump (Steve Singer)
•
Fix compression of plain-text output format in pg_dump (Adrian Klaver and Tom Lane) pg_dump has historically understood -Z with no -F switch to mean that it should emit a gzipcompressed version of its plain text output. Restore that behavior.
•
Fix pg_dump to dump user-defined casts between auto-generated types, such as table rowtypes (Tom Lane)
•
Fix missed quoting of foreign server names in pg_dump (Tom Lane)
•
Assorted fixes for pg_upgrade (Bruce Momjian)
2203
Appendix E. Release Notes Handle exclusion constraints correctly, avoid failures on Windows, don’t complain about mismatched toast table names in 8.4 databases. •
In PL/pgSQL, allow foreign tables to define row types (Alexander Soudakov)
•
Fix up conversions of PL/Perl functions’ results (Alex Hunsaker and Tom Lane) Restore the pre-9.1 behavior that PL/Perl functions returning void ignore the result value of their last Perl statement; 9.1.0 would throw an error if that statement returned a reference. Also, make sure it works to return a string value for a composite type, so long as the string meets the type’s input format. In addition, throw errors for attempts to return Perl arrays or hashes when the function’s declared result type is not an array or composite type, respectively. (Pre-9.1 versions rather uselessly returned strings like ARRAY(0x221a9a0) or HASH(0x221aa90) in such cases.)
•
Ensure PL/Perl strings are always correctly UTF8-encoded (Amit Khandekar and Alex Hunsaker)
•
Use the preferred version of xsubpp to build PL/Perl, not necessarily the operating system’s main copy (David Wheeler and Alex Hunsaker)
•
Correctly propagate SQLSTATE in PL/Python exceptions (Mika Eloranta and Jan Urbanski)
•
Do not install PL/Python extension files for Python major versions other than the one built against (Peter Eisentraut)
•
Change all the contrib extension script files to report a useful error message if they are fed to psql (Andrew Dunstan and Tom Lane) This should help teach people about the new method of using CREATE EXTENSION to load these files. In most cases, sourcing the scripts directly would fail anyway, but with harder-to-interpret messages.
•
Fix incorrect coding in contrib/dict_int and contrib/dict_xsyn (Tom Lane) Some functions incorrectly assumed that memory returned by palloc() is guaranteed zeroed.
•
Remove contrib/sepgsql tests from the regular regression test mechanism (Tom Lane) Since these tests require root privileges for setup, they’re impractical to run automatically. Switch over to a manual approach instead, and provide a testing script to help with that.
•
Fix assorted errors in contrib/unaccent’s configuration file parsing (Tom Lane)
•
Honor query cancel interrupts promptly in pgstatindex() (Robert Haas)
•
Fix incorrect quoting of log file name in macOS start script (Sidar Lopez)
•
Revert unintentional enabling of WAL_DEBUG (Robert Haas) Fortunately, as debugging tools go, this one is pretty cheap; but it’s not intended to be enabled by default, so revert.
•
Ensure VPATH builds properly install all server header files (Peter Eisentraut)
•
Shorten file names reported in verbose error messages (Peter Eisentraut) Regular builds have always reported just the name of the C file containing the error message call, but VPATH builds formerly reported an absolute path name.
•
Fix interpretation of Windows timezone names for Central America (Tom Lane) Map “Central America Standard Time” to CST6, not CST6CDT, because DST is generally not observed anywhere in Central America.
•
Update time zone data files to tzdata release 2011n for DST law changes in Brazil, Cuba, Fiji, Palestine, Russia, and Samoa; also historical corrections for Alaska and British East Africa.
2204
Appendix E. Release Notes
E.68. Release 9.1.1 Release date: 2011-09-26 This release contains a small number of fixes from 9.1.0. For information about new features in the 9.1 major release, see Section E.69.
E.68.1. Migration to Version 9.1.1 A dump/restore is not required for those running 9.1.X.
E.68.2. Changes •
Make pg_options_to_table return NULL for an option with no value (Tom Lane) Previously such cases would result in a server crash.
•
Fix memory leak at end of a GiST index scan (Tom Lane) Commands that perform many separate GiST index scans, such as verification of a new GiSTbased exclusion constraint on a table already containing many rows, could transiently require large amounts of memory due to this leak.
•
Fix explicit reference to pg_temp schema in CREATE TEMPORARY TABLE (Robert Haas) This used to be allowed, but failed in 9.1.0.
E.69. Release 9.1 Release date: 2011-09-12
E.69.1. Overview This release shows PostgreSQL moving beyond the traditional relational-database feature set with new, ground-breaking functionality that is unique to PostgreSQL. The streaming replication feature introduced in release 9.0 is significantly enhanced by adding a synchronous-replication option, streaming backups, and monitoring improvements. Major enhancements include: •
Allow synchronous replication
•
Add support for foreign tables
•
Add per-column collation support
•
Add extensions which simplify packaging of additions to PostgreSQL
•
Add a true serializable isolation level
•
Support unlogged tables using the UNLOGGED option in CREATE TABLE
•
Allow data-modification commands (INSERT/UPDATE/DELETE) in WITH clauses
•
Add nearest-neighbor (order-by-operator) searching to GiST indexes
•
Add a SECURITY LABEL command and support for SELinux permissions control
2205
Appendix E. Release Notes •
Update the PL/Python server-side language
The above items are explained in more detail in the sections below.
E.69.2. Migration to Version 9.1 A dump/restore using pg_dump, or use of pg_upgrade, is required for those wishing to migrate data from any previous release. Version 9.1 contains a number of changes that may affect compatibility with previous releases. Observe the following incompatibilities:
E.69.2.1. Strings •
Change the default value of standard_conforming_strings to on (Robert Haas) By default, backslashes are now ordinary characters in string literals, not escape characters. This change removes a long-standing incompatibility with the SQL standard. escape_string_warning has produced warnings about this usage for years. E” strings are the proper way to embed backslash escapes in strings and are unaffected by this change.
Warning This change can break applications that are not expecting it and do their own string escaping according to the old rules. The consequences could be as severe as introducing SQL-injection security holes. Be sure to test applications that are exposed to untrusted input, to ensure that they correctly handle single quotes and backslashes in text strings.
E.69.2.2. Casting •
Disallow function-style and attribute-style data type casts for composite types (Tom Lane) For example, disallow composite_value.text and text(composite_value). Unintentional uses of this syntax have frequently resulted in bug reports; although it was not a bug, it seems better to go back to rejecting such expressions. The CAST and :: syntaxes are still available for use when a cast of an entire composite value is actually intended.
•
Tighten casting checks for domains based on arrays (Tom Lane) When a domain is based on an array type, it is allowed to “look through” the domain type to access the array elements, including subscripting the domain value to fetch or assign an element. Assignment to an element of such a domain value, for instance via UPDATE ... SET domaincol[5] = ..., will now result in rechecking the domain type’s constraints, whereas before the checks were skipped.
E.69.2.3. Arrays •
Change string_to_array() to return an empty array for a zero-length string (Pavel Stehule)
2206
Appendix E. Release Notes Previously this returned a null value. •
Change string_to_array() so a NULL separator splits the string into characters (Pavel Stehule) Previously this returned a null value.
E.69.2.4. Object Modification •
Fix improper checks for before/after triggers (Tom Lane) Triggers can now be fired in three cases: BEFORE, AFTER, or INSTEAD OF some action. Trigger function authors should verify that their logic behaves sanely in all three cases.
•
Require superuser or CREATEROLE permissions in order to set comments on roles (Tom Lane)
E.69.2.5. Server Settings •
Change pg_last_xlog_receive_location() so it never moves backwards (Fujii Masao) Previously, the value of pg_last_xlog_receive_location() could move backward when streaming replication is restarted.
•
Have logging of replication connections honor log_connections (Magnus Hagander) Previously, replication connections were always logged.
E.69.2.6. PL/pgSQL Server-Side Language •
Change PL/pgSQL’s RAISE command without parameters to be catchable by the attached exception block (Piyush Newe) Previously RAISE in a code block was always scoped to an attached exception block, so it was uncatchable at the same scope.
•
Adjust PL/pgSQL’s error line numbering code to be consistent with other PLs (Pavel Stehule) Previously, PL/pgSQL would ignore (not count) an empty line at the start of the function body. Since this was inconsistent with all other languages, the special case was removed.
•
Make PL/pgSQL complain about conflicting IN and OUT parameter names (Tom Lane) Formerly, the collision was not detected, and the name would just silently refer to only the OUT parameter.
•
Type modifiers of PL/pgSQL variables are now visible to the SQL parser (Tom Lane) A type modifier (such as a varchar length limit) attached to a PL/pgSQL variable was formerly enforced during assignments, but was ignored for all other purposes. Such variables will now behave more like table columns declared with the same modifier. This is not expected to make any visible difference in most cases, but it could result in subtle changes for some SQL commands issued by PL/pgSQL functions.
2207
Appendix E. Release Notes
E.69.2.7. Contrib •
All contrib modules are now installed with CREATE EXTENSION rather than by manually invoking their SQL scripts (Dimitri Fontaine, Tom Lane) To update an existing database containing the 9.0 version of a contrib module, use CREATE EXTENSION ... FROM unpackaged to wrap the existing contrib module’s objects into an extension. When updating from a pre-9.0 version, drop the contrib module’s objects using its old uninstall script, then use CREATE EXTENSION.
E.69.2.8. Other Incompatibilities •
Make pg_stat_reset() reset all database-level statistics (Tomas Vondra) Some pg_stat_database counters were not being reset.
•
Fix some information_schema.triggers column names to match the new SQL-standard names (Dean Rasheed)
•
Treat ECPG cursor names as case-insensitive (Zoltan Boszormenyi)
E.69.3. Changes Below you will find a detailed account of the changes between PostgreSQL 9.1 and the previous major release.
E.69.3.1. Server E.69.3.1.1. Performance •
Support unlogged tables using the UNLOGGED option in CREATE TABLE (Robert Haas) Such tables provide better update performance than regular tables, but are not crash-safe: their contents are automatically cleared in case of a server crash. Their contents do not propagate to replication slaves, either.
•
Allow FULL OUTER JOIN to be implemented as a hash join, and allow either side of a LEFT OUTER JOIN or RIGHT OUTER JOIN to be hashed (Tom Lane) Previously FULL OUTER JOIN could only be implemented as a merge join, and LEFT OUTER JOIN and RIGHT OUTER JOIN could hash only the nullable side of the join. These changes provide additional query optimization possibilities.
•
Merge duplicate fsync requests (Robert Haas, Greg Smith) This greatly improves performance under heavy write loads.
•
Improve performance of commit_siblings (Greg Smith) This allows the use of commit_siblings with less overhead.
•
Reduce the memory requirement for large ispell dictionaries (Pavel Stehule, Tom Lane)
•
Avoid leaving data files open after “blind writes” (Alvaro Herrera)
2208
Appendix E. Release Notes This fixes scenarios in which backends might hold files open long after they were deleted, preventing the kernel from reclaiming disk space.
E.69.3.1.2. Optimizer •
Allow inheritance table scans to return meaningfully-sorted results (Greg Stark, Hans-Jurgen Schonig, Robert Haas, Tom Lane) This allows better optimization of queries that use ORDER BY, LIMIT, or MIN/MAX with inherited tables.
•
Improve GIN index scan cost estimation (Teodor Sigaev)
•
Improve cost estimation for aggregates and window functions (Tom Lane)
E.69.3.1.3. Authentication •
Support host names and host suffixes (e.g. .example.com) in pg_hba.conf (Peter Eisentraut) Previously only host IP addresses and CIDR values were supported.
•
Support the key word all in the host column of pg_hba.conf (Peter Eisentraut) Previously people used 0.0.0.0/0 or ::/0 for this.
•
Reject local lines in pg_hba.conf on platforms that don’t support Unix-socket connections (Magnus Hagander) Formerly, such lines were silently ignored, which could be surprising. This makes the behavior more like other unsupported cases.
•
Allow GSSAPI to be used to authenticate to servers via SSPI (Christian Ullrich) Specifically this allows Unix-based GSSAPI clients to do SSPI authentication with Windows servers.
• ident
authentication over local sockets is now known as peer (Magnus Hagander)
The old term is still accepted for backward compatibility, but since the two methods are fundamentally different, it seemed better to adopt different names for them. •
Rewrite peer authentication to avoid use of credential control messages (Tom Lane) This change makes the peer authentication code simpler and better-performing. However, it requires the platform to provide the getpeereid function or an equivalent socket operation. So far as is known, the only platform for which peer authentication worked before and now will not is pre-5.0 NetBSD.
E.69.3.1.4. Monitoring •
Add details to the logging of restartpoints and checkpoints, which is controlled by log_checkpoints (Fujii Masao, Greg Smith) New details include WAL file and sync activity.
•
Add log_file_mode which controls the permissions on log files created by the logging collector (Martin Pihlak)
•
Reduce the default maximum line length for syslog logging to 900 bytes plus prefixes (Noah Misch)
2209
Appendix E. Release Notes This avoids truncation of long log lines on syslog implementations that have a 1KB length limit, rather than the more common 2KB.
E.69.3.1.5. Statistical Views •
Add client_hostname column to pg_stat_activity (Peter Eisentraut) Previously only the client address was reported.
•
Add pg_stat_xact_* statistics functions and views (Joel Jacobson) These are like the database-wide statistics counter views, but reflect counts for only the current transaction.
•
Add time of last reset in database-level and background writer statistics views (Tomas Vondra)
•
Add columns showing the number of vacuum and analyze operations in pg_stat_*_tables views (Magnus Hagander)
•
Add buffers_backend_fsync column to pg_stat_bgwriter (Greg Smith) This new column counts the number of times a backend fsyncs a buffer.
E.69.3.1.6. Server Settings •
Provide auto-tuning of wal_buffers (Greg Smith) By default, the value of wal_buffers is now chosen automatically based on the value of shared_buffers.
•
Increase the maximum values for deadlock_timeout, log_min_duration_statement, and log_autovacuum_min_duration (Peter Eisentraut) The maximum value for each of these parameters was previously only about 35 minutes. Much larger values are now allowed.
E.69.3.2. Replication and Recovery E.69.3.2.1. Streaming Replication and Continuous Archiving •
Allow synchronous replication (Simon Riggs, Fujii Masao) This allows the primary server to wait for a standby to write a transaction’s information to disk before acknowledging the commit. One standby at a time can take the role of the synchronous standby, as controlled by the synchronous_standby_names setting. Synchronous replication can be enabled or disabled on a per-transaction basis using the synchronous_commit setting.
•
Add protocol support for sending file system backups to standby servers using the streaming replication network connection (Magnus Hagander, Heikki Linnakangas) This avoids the requirement of manually transferring a file system backup when setting up a standby server.
•
Add replication_timeout setting (Fujii Masao, Heikki Linnakangas)
2210
Appendix E. Release Notes Replication connections that are idle for more than the replication_timeout interval will be terminated automatically. Formerly, a failed connection was typically not detected until the TCP timeout elapsed, which is inconveniently long in many situations. •
Add command-line tool pg_basebackup for creating a new standby server or database backup (Magnus Hagander)
•
Add a replication permission for roles (Magnus Hagander) This is a read-only permission used for streaming replication. It allows a non-superuser role to be used for replication connections. Previously only superusers could initiate replication connections; superusers still have this permission by default.
E.69.3.2.2. Replication Monitoring •
Add system view pg_stat_replication which displays activity of WAL sender processes (Itagaki Takahiro, Simon Riggs) This reports the status of all connected standby servers.
•
Add monitoring function pg_last_xact_replay_timestamp() (Fujii Masao) This returns the time at which the primary generated the most recent commit or abort record applied on the standby.
E.69.3.2.3. Hot Standby •
Add configuration parameter hot_standby_feedback to enable standbys to postpone cleanup of old row versions on the primary (Simon Riggs) This helps avoid canceling long-running queries on the standby.
•
Add the pg_stat_database_conflicts system view to show queries that have been canceled and the reason (Magnus Hagander) Cancellations can occur because of dropped tablespaces, lock timeouts, old snapshots, pinned buffers, and deadlocks.
•
Add a conflicts count to pg_stat_database (Magnus Hagander) This is the number of conflicts that occurred in the database.
•
Increase
the
maximum
values
for
max_standby_archive_delay
and
max_standby_streaming_delay
The maximum value for each of these parameters was previously only about 35 minutes. Much larger values are now allowed. •
Add ERRCODE_T_R_DATABASE_DROPPED error code to report recovery conflicts due to dropped databases (Tatsuo Ishii) This is useful for connection pooling software.
E.69.3.2.4. Recovery Control •
Add functions to control streaming replication replay (Simon Riggs)
2211
Appendix E. Release Notes The new functions are pg_xlog_replay_pause(), pg_xlog_replay_resume(), and the status function pg_is_xlog_replay_paused(). •
Add recovery.conf setting pause_at_recovery_target to pause recovery at target (Simon Riggs) This allows a recovery server to be queried to check whether the recovery point is the one desired.
•
Add the ability to create named restore points using pg_create_restore_point() (Jaime Casanova) These named restore points can be specified as recovery targets using the new recovery.conf setting recovery_target_name.
•
Allow standby recovery to switch to a new timeline automatically (Heikki Linnakangas) Now standby servers scan the archive directory for new timelines periodically.
•
Add restart_after_crash setting which disables automatic server restart after a backend crash (Robert Haas) This allows external cluster management software to control whether the database server restarts or not.
•
Allow recovery.conf to use the same quoting behavior as postgresql.conf (Dimitri Fontaine) Previously all values had to be quoted.
E.69.3.3. Queries •
Add a true serializable isolation level (Kevin Grittner, Dan Ports) Previously, asking for serializable isolation guaranteed only that a single MVCC snapshot would be used for the entire transaction, which allowed certain documented anomalies. The old snapshot isolation behavior is still available by requesting the REPEATABLE READ isolation level.
•
Allow data-modification commands (INSERT/UPDATE/DELETE) in WITH clauses (Marko Tiikkaja, Hitoshi Harada) These commands can use RETURNING to pass data up to the containing query.
•
Allow WITH clauses to be attached to INSERT, UPDATE, DELETE statements (Marko Tiikkaja, Hitoshi Harada)
•
Allow non-GROUP BY columns in the query target list when the primary key is specified in the GROUP BY clause (Peter Eisentraut) The SQL standard allows this behavior, and because of the primary key, the result is unambiguous.
•
Allow use of the key word DISTINCT in UNION/INTERSECT/EXCEPT clauses (Tom Lane) DISTINCT is the default behavior so use of this key word is redundant, but the SQL standard allows
it. •
Fix ordinary queries with rules to use the same snapshot behavior as EXPLAIN ANALYZE (Marko Tiikkaja) Previously EXPLAIN ANALYZE used slightly different snapshot timing for queries involving rules. The EXPLAIN ANALYZE behavior was judged to be more logical.
2212
Appendix E. Release Notes E.69.3.3.1. Strings •
Add per-column collation support (Peter Eisentraut, Tom Lane) Previously collation (the sort ordering of text strings) could only be chosen at database creation. Collation can now be set per column, domain, index, or expression, via the SQL-standard COLLATE clause.
E.69.3.4. Object Manipulation •
Add extensions which simplify packaging of additions to PostgreSQL (Dimitri Fontaine, Tom Lane) Extensions are controlled by the new CREATE/ALTER/DROP EXTENSION commands. This replaces ad-hoc methods of grouping objects that are added to a PostgreSQL installation.
•
Add support for foreign tables (Shigeru Hanada, Robert Haas, Jan Urbanski, Heikki Linnakangas) This allows data stored outside the database to be used like native PostgreSQL-stored data. Foreign tables are currently read-only, however.
•
Allow new values to be added to an existing enum type via ALTER TYPE (Andrew Dunstan)
•
Add ALTER TYPE ... ADD/DROP/ALTER/RENAME ATTRIBUTE (Peter Eisentraut) This allows modification of composite types.
E.69.3.4.1. ALTER Object •
Add RESTRICT/CASCADE to ALTER TYPE operations on typed tables (Peter Eisentraut) This controls ADD/DROP/ALTER/RENAME ATTRIBUTE cascading behavior.
•
Support ALTER TABLE name {OF | NOT OF} type (Noah Misch) This syntax allows a standalone table to be made into a typed table, or a typed table to be made standalone.
•
Add support for more object types in ALTER ... SET SCHEMA commands (Dimitri Fontaine) This command is now supported for conversions, operators, operator classes, operator families, text search configurations, text search dictionaries, text search parsers, and text search templates.
E.69.3.4.2. CREATE/ALTER TABLE •
Add ALTER TABLE ... ADD UNIQUE/PRIMARY KEY USING INDEX (Gurjeet Singh) This allows a primary key or unique constraint to be defined using an existing unique index, including a concurrently created unique index.
•
Allow ALTER TABLE to add foreign keys without validation (Simon Riggs) The new option is called NOT VALID. The constraint’s state can later be modified to VALIDATED and validation checks performed. Together these allow you to add a foreign key with minimal impact on read and write operations.
2213
Appendix E. Release Notes •
Allow ALTER TABLE ... SET DATA TYPE to avoid table rewrites in appropriate cases (Noah Misch, Robert Haas) For example, converting a varchar column to text no longer requires a rewrite of the table. However, increasing the length constraint on a varchar column still requires a table rewrite.
•
Add CREATE TABLE IF NOT EXISTS syntax (Robert Haas) This allows table creation without causing an error if the table already exists.
•
Fix possible “tuple concurrently updated” error when two backends attempt to add an inheritance child to the same table at the same time (Robert Haas) ALTER TABLE now takes a stronger lock on the parent table, so that the sessions cannot try to
update it simultaneously.
E.69.3.4.3. Object Permissions •
Add a SECURITY LABEL command (KaiGai Kohei) This allows security labels to be assigned to objects.
E.69.3.5. Utility Operations •
Add transaction-level advisory locks (Marko Tiikkaja) These are similar to the existing session-level advisory locks, but such locks are automatically released at transaction end.
•
Make TRUNCATE ... RESTART IDENTITY restart sequences transactionally (Steve Singer) Previously the counter could have been left out of sync if a backend crashed between the on-commit truncation activity and commit completion.
E.69.3.5.1. COPY •
Add ENCODING option to COPY TO/FROM (Hitoshi Harada, Itagaki Takahiro) This allows the encoding of the COPY file to be specified separately from client encoding.
•
Add bidirectional COPY protocol support (Fujii Masao) This is currently only used by streaming replication.
E.69.3.5.2. EXPLAIN •
Make EXPLAIN VERBOSE show the function call expression in a FunctionScan node (Tom Lane)
E.69.3.5.3. VACUUM •
Add additional details to the output of VACUUM FULL VERBOSE and CLUSTER VERBOSE (Itagaki Takahiro)
2214
Appendix E. Release Notes New information includes the live and dead tuple count and whether CLUSTER is using an index to rebuild. •
Prevent autovacuum from waiting if it cannot acquire a table lock (Robert Haas) It will try to vacuum that table later.
E.69.3.5.4. CLUSTER •
Allow CLUSTER to sort the table rather than scanning the index when it seems likely to be cheaper (Leonardo Francalanci)
E.69.3.5.5. Indexes •
Add nearest-neighbor (order-by-operator) searching to GiST indexes (Teodor Sigaev, Tom Lane) This allows GiST indexes to quickly return the N closest values in a query with LIMIT. For example SELECT * FROM places ORDER BY location point ’(101,456)’ LIMIT 10;
finds the ten places closest to a given target point. •
Allow GIN indexes to index null and empty values (Tom Lane) This allows full GIN index scans, and fixes various corner cases in which GIN scans would fail.
•
Allow GIN indexes to better recognize duplicate search entries (Tom Lane) This reduces the cost of index scans, especially in cases where it avoids unnecessary full index scans.
•
Fix GiST indexes to be fully crash-safe (Heikki Linnakangas) Previously there were rare cases where a REINDEX would be required (you would be informed).
E.69.3.6. Data Types •
Allow numeric to use a more compact, two-byte header in common cases (Robert Haas) Previously all numeric values had four-byte headers; this change saves on disk storage.
•
Add support for dividing money by money (Andy Balholm)
•
Allow binary I/O on type void (Radoslaw Smogura)
•
Improve hypotenuse calculations for geometric operators (Paul Matthews) This avoids unnecessary overflows, and may also be more accurate.
•
Support hashing array values (Tom Lane) This provides additional query optimization possibilities.
•
Don’t treat a composite type as sortable unless all its column types are sortable (Tom Lane) This avoids possible “could not identify a comparison function” failures at runtime, if it is possible to implement the query without sorting. Also, ANALYZE won’t try to use inappropriate statisticsgathering methods for columns of such composite types.
2215
Appendix E. Release Notes E.69.3.6.1. Casting •
Add support for casting between money and numeric (Andy Balholm)
•
Add support for casting from int4 and int8 to money (Joey Adams)
•
Allow casting a table’s row type to the table’s supertype if it’s a typed table (Peter Eisentraut) This is analogous to the existing facility that allows casting a row type to a supertable’s row type.
E.69.3.6.2. XML •
Add XML function XMLEXISTS and xpath_exists() functions (Mike Fowler) These are used for XPath matching.
•
Add
XML functions xml_is_well_formed(), xml_is_well_formed_content() (Mike Fowler)
xml_is_well_formed_document(),
These check whether the input is properly-formed XML. They provide functionality that was previously available only in the deprecated contrib/xml2 module.
E.69.3.7. Functions •
Add SQL function format(text, ...), which behaves analogously to C’s printf() (Pavel Stehule, Robert Haas) It currently supports formats for strings, SQL literals, and SQL identifiers.
•
Add string functions concat(), concat_ws(), left(), right(), and reverse() (Pavel Stehule) These improve compatibility with other database products.
•
Add function pg_read_binary_file() to read binary files (Dimitri Fontaine, Itagaki Takahiro)
•
Add a single-parameter version of function pg_read_file() to read an entire file (Dimitri Fontaine, Itagaki Takahiro)
•
Add three-parameter forms of array_to_string() and string_to_array() for null value processing control (Pavel Stehule)
E.69.3.7.1. Object Information Functions •
Add the pg_describe_object() function (Alvaro Herrera) This function is used to obtain a human-readable string describing an object, based on the pg_class OID, object OID, and sub-object ID. It can be used to help interpret the contents of pg_depend.
•
Update comments for built-in operators and their underlying functions (Tom Lane) Functions that are meant to be used via an associated operator are now commented as such.
•
Add variable quote_all_identifiers to force the quoting of all identifiers in EXPLAIN and in system catalog functions like pg_get_viewdef() (Robert Haas) This makes exporting schemas to tools and other databases with different quoting rules easier.
2216
Appendix E. Release Notes •
Add columns to the information_schema.sequences system view (Peter Eisentraut) Previously, though the view existed, the columns about the sequence parameters were unimplemented.
•
Allow public as a pseudo-role name in has_table_privilege() and related functions (Alvaro Herrera) This allows checking for public permissions.
E.69.3.7.2. Function and Trigger Creation •
Support INSTEAD OF triggers on views (Dean Rasheed) This feature can be used to implement fully updatable views.
E.69.3.8. Server-Side Languages E.69.3.8.1. PL/pgSQL Server-Side Language •
Add FOREACH IN ARRAY to PL/pgSQL (Pavel Stehule) This is more efficient and readable than previous methods of iterating through the elements of an array value.
•
Allow RAISE without parameters to be caught in the same places that could catch a RAISE ERROR from the same location (Piyush Newe) The previous coding threw the error from the block containing the active exception handler. The new behavior is more consistent with other DBMS products.
E.69.3.8.2. PL/Perl Server-Side Language •
Allow generic record arguments to PL/Perl functions (Andrew Dunstan) PL/Perl functions can now be declared to accept type record. The behavior is the same as for any named composite type.
•
Convert PL/Perl array arguments to Perl arrays (Alexey Klyukin, Alex Hunsaker) String representations are still available.
•
Convert PL/Perl composite-type arguments to Perl hashes (Alexey Klyukin, Alex Hunsaker) String representations are still available.
E.69.3.8.3. PL/Python Server-Side Language •
Add table function support for PL/Python (Jan Urbanski) PL/Python can now return multiple OUT parameters and record sets.
•
Add a validator to PL/Python (Jan Urbanski) This allows PL/Python functions to be syntax-checked at function creation time.
2217
Appendix E. Release Notes •
Allow exceptions for SQL queries in PL/Python (Jan Urbanski) This allows access to SQL-generated exception error codes from PL/Python exception blocks.
•
Add explicit subtransactions to PL/Python (Jan Urbanski)
•
Add PL/Python functions for quoting strings (Jan Urbanski) These functions are plpy.quote_ident, plpy.quote_literal, and plpy.quote_nullable.
•
Add traceback information to PL/Python errors (Jan Urbanski)
•
Report PL/Python errors from iterators with PLy_elog (Jan Urbanski)
•
Fix exception handling with Python 3 (Jan Urbanski) Exception classes were previously not available in plpy under Python 3.
E.69.3.9. Client Applications •
Mark createlang and droplang as deprecated now that they just invoke extension commands (Tom Lane)
E.69.3.9.1. psql •
Add psql command \conninfo to show current connection information (David Christensen)
•
Add psql command \sf to show a function’s definition (Pavel Stehule)
•
Add psql command \dL to list languages (Fernando Ike)
•
Add the S (“system”) option to psql’s \dn (list schemas) command (Tom Lane) \dn without S now suppresses system schemas.
•
Allow psql’s \e and \ef commands to accept a line number to be used to position the cursor in the editor (Pavel Stehule) This is passed to the editor according to the PSQL_EDITOR_LINENUMBER_ARG environment variable.
•
Have psql set the client encoding from the operating system locale by default (Heikki Linnakangas) This only happens if the PGCLIENTENCODING environment variable is not set.
•
Make \d distinguish between unique indexes and unique constraints (Josh Kupershmidt)
•
Make \dt+ report pg_table_size instead of pg_relation_size when talking to 9.0 or later servers (Bernd Helmle) This is a more useful measure of table size, but note that it is not identical to what was previously reported in the same display.
•
Additional tab completion support (Itagaki Takahiro, Pavel Stehule, Andrey Popp, Christoph Berg, David Fetter, Josh Kupershmidt)
E.69.3.9.2. pg_dump •
Add pg_dump and pg_dumpall option --quote-all-identifiers to force quoting of all identifiers (Robert Haas)
2218
Appendix E. Release Notes •
Add directory format to pg_dump (Joachim Wieland, Heikki Linnakangas) This is internally similar to the tar pg_dump format.
E.69.3.9.3. pg_ctl •
Fix pg_ctl so it no longer incorrectly reports that the server is not running (Bruce Momjian) Previously this could happen if the server was running but pg_ctl could not authenticate.
•
Improve pg_ctl start’s “wait” (-w) option (Bruce Momjian, Tom Lane) The wait mode is now significantly more robust. It will not get confused by non-default postmaster port numbers, non-default Unix-domain socket locations, permission problems, or stale postmaster lock files.
•
Add promote option to pg_ctl to switch a standby server to primary (Fujii Masao)
E.69.3.10. Development Tools E.69.3.10.1. libpq •
Add a libpq connection option client_encoding which behaves like the PGCLIENTENCODING environment variable (Heikki Linnakangas) The value auto sets the client encoding based on the operating system locale.
•
Add PQlibVersion() function which returns the libpq library version (Magnus Hagander) libpq already had PQserverVersion() which returns the server version.
•
Allow libpq-using clients to check the user name of the server process when connecting via Unixdomain sockets, with the new requirepeer connection option (Peter Eisentraut) PostgreSQL already allowed servers to check the client user name when connecting via Unixdomain sockets.
•
Add PQping() and PQpingParams() to libpq (Bruce Momjian, Tom Lane) These functions allow detection of the server’s status without trying to open a new session.
E.69.3.10.2. ECPG •
Allow ECPG to accept dynamic cursor names even in WHERE CURRENT OF clauses (Zoltan Boszormenyi)
•
Make ecpglib write double values with a precision of 15 digits, not 14 as formerly (Akira Kurosawa)
E.69.3.11. Build Options •
Use +Olibmerrno compile flag with HP-UX C compilers that accept it (Ibrar Ahmed)
2219
Appendix E. Release Notes This avoids possible misbehavior of math library calls on recent HP platforms. E.69.3.11.1. Makefiles •
Improved parallel make support (Peter Eisentraut) This allows for faster compiles. Also, make -k now works more consistently.
•
Require GNU make 3.80 or newer (Peter Eisentraut) This is necessary because of the parallel-make improvements.
•
Add make maintainer-check target (Peter Eisentraut) This target performs various source code checks that are not appropriate for either the build or the regression tests. Currently: duplicate_oids, SGML syntax and tabs check, NLS syntax check.
•
Support make check in contrib (Peter Eisentraut) Formerly only make installcheck worked, but now there is support for testing in a temporary installation. The top-level make check-world target now includes testing contrib this way.
E.69.3.11.2. Windows •
On Windows, allow pg_ctl to register the service as auto-start or start-on-demand (Quan Zongliang)
•
Add support for collecting crash dumps on Windows (Craig Ringer, Magnus Hagander) minidumps can now be generated by non-debug Windows binaries and analyzed by standard debugging tools.
•
Enable building with the MinGW64 compiler (Andrew Dunstan) This allows building 64-bit Windows binaries even on non-Windows platforms via cross-compiling.
E.69.3.12. Source Code •
Revise the API for GUC variable assign hooks (Tom Lane) The previous functions of assign hooks are now split between check hooks and assign hooks, where the former can fail but the latter shouldn’t. This change will impact add-on modules that define custom GUC parameters.
•
Add latches to the source code to support waiting for events (Heikki Linnakangas)
•
Centralize data modification permissions-checking logic (KaiGai Kohei)
•
Add missing get_object_oid() functions, for consistency (Robert Haas)
•
Improve ability to use C++ compilers for compiling add-on modules by removing conflicting key words (Tom Lane)
•
Add support for DragonFly BSD (Rumko)
•
Expose quote_literal_cstr() for backend use (Robert Haas)
•
Run regression tests in the default encoding (Peter Eisentraut) Regression tests were previously always run with SQL_ASCII encoding.
•
Add src/tools/git_changelog to replace cvs2cl and pgcvslog (Robert Haas, Tom Lane)
2220
Appendix E. Release Notes •
Add git-external-diff script to src/tools (Bruce Momjian) This is used to generate context diffs from git.
•
Improve support for building with Clang (Peter Eisentraut)
E.69.3.12.1. Server Hooks •
Add source code hooks to check permissions (Robert Haas, Stephen Frost)
•
Add post-object-creation function hooks for use by security frameworks (KaiGai Kohei)
•
Add a client authentication hook (KaiGai Kohei)
E.69.3.13. Contrib •
Modify contrib modules and procedural languages to install via the new extension mechanism (Tom Lane, Dimitri Fontaine)
•
Add contrib/file_fdw foreign-data wrapper (Shigeru Hanada) Foreign tables using this foreign data wrapper can read flat files in a manner very similar to COPY.
•
Add nearest-neighbor search support to contrib/pg_trgm and contrib/btree_gist (Teodor Sigaev)
•
Add contrib/btree_gist support for searching on not-equals (Jeff Davis)
•
Fix contrib/fuzzystrmatch’s levenshtein() function to handle multibyte characters (Alexander Korotkov)
•
Add ssl_cipher() and ssl_version() functions to contrib/sslinfo (Robert Haas)
•
Fix contrib/intarray and contrib/hstore to give consistent results with indexed empty arrays (Tom Lane) Previously an empty-array query that used an index might return different results from one that used a sequential scan.
•
Allow contrib/intarray to work properly on multidimensional arrays (Tom Lane)
•
In contrib/intarray, avoid errors complaining about the presence of nulls in cases where no nulls are actually present (Tom Lane)
•
In contrib/intarray, fix behavior of containment operators with respect to empty arrays (Tom Lane) Empty arrays are now correctly considered to be contained in any other array.
•
Remove contrib/xml2’s arbitrary limit on the number of parameter =value pairs that can be handled by xslt_process() (Pavel Stehule) The previous limit was 10.
•
In contrib/pageinspect, fix heap_page_item to return infomasks as 32-bit values (Alvaro Herrera) This avoids returning negative values, which was confusing. The underlying value is a 16-bit unsigned integer.
2221
Appendix E. Release Notes E.69.3.13.1. Security •
Add contrib/sepgsql to interface permission checks with SELinux (KaiGai Kohei) This uses the new SECURITY LABEL facility.
•
Add contrib module auth_delay (KaiGai Kohei) This causes the server to pause before returning authentication failure; it is designed to make brute force password attacks more difficult.
•
Add dummy_seclabel contrib module (KaiGai Kohei) This is used for permission regression testing.
E.69.3.13.2. Performance •
Add support for LIKE and ILIKE index searches to contrib/pg_trgm (Alexander Korotkov)
•
Add levenshtein_less_equal() function to contrib/fuzzystrmatch, which is optimized for small distances (Alexander Korotkov)
•
Improve performance of index lookups on contrib/seg columns (Alexander Korotkov)
•
Improve performance of pg_upgrade for databases with many relations (Bruce Momjian)
•
Add flag to contrib/pgbench to report per-statement latencies (Florian Pflug)
E.69.3.13.3. Fsync Testing •
Move src/tools/test_fsync to contrib/pg_test_fsync (Bruce Momjian, Tom Lane)
•
Add O_DIRECT support to contrib/pg_test_fsync (Bruce Momjian) This matches the use of O_DIRECT by wal_sync_method.
•
Add new tests to contrib/pg_test_fsync (Bruce Momjian)
Add documentation for exit_on_error (Robert Haas) This parameter causes sessions to exit on any error.
•
Add documentation for pg_options_to_table() (Josh Berkus) This function shows table storage options in a readable form.
•
Document that it is possible to access all composite type fields using (compositeval).* syntax (Peter Eisentraut)
•
Document that translate() removes characters in from that don’t have a corresponding to character (Josh Kupershmidt)
2222
Appendix E. Release Notes •
Merge documentation for CREATE CONSTRAINT TRIGGER and CREATE TRIGGER (Alvaro Herrera)
•
Centralize permission and upgrade documentation (Bruce Momjian)
•
Add kernel tuning documentation for Solaris 10 (Josh Berkus) Previously only Solaris 9 kernel tuning was documented.
•
Handle non-ASCII characters consistently in HISTORY file (Peter Eisentraut) While the HISTORY file is in English, we do have to deal with non-ASCII letters in contributor names. These are now transliterated so that they are reasonably legible without assumptions about character set.
E.70. Release 9.0.23 Release date: 2015-10-08 This release contains a variety of fixes from 9.0.22. For information about new features in the 9.0 major release, see Section E.93. This is expected to be the last PostgreSQL release in the 9.0.X series. Users are encouraged to update to a newer release branch soon.
E.70.1. Migration to Version 9.0.23 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.18, see Section E.75.
E.70.2. Changes •
Fix contrib/pgcrypto to detect and report too-short crypt() salts (Josh Kupershmidt) Certain invalid salt arguments crashed the server or disclosed a few bytes of server memory. We have not ruled out the viability of attacks that arrange for presence of confidential information in the disclosed bytes, but they seem unlikely. (CVE-2015-5288)
•
Fix subtransaction cleanup after a portal (cursor) belonging to an outer subtransaction fails (Tom Lane, Michael Paquier) A function executed in an outer-subtransaction cursor could cause an assertion failure or crash by referencing a relation created within an inner subtransaction.
•
Fix insertion of relations into the relation cache “init file” (Tom Lane) An
oversight
in
a
patch
in
the
most
recent
minor
releases
caused
pg_trigger_tgrelid_tgname_index to be omitted from the init file. Subsequent sessions
detected this, then deemed the init file to be broken and silently ignored it, resulting in a significant degradation in session startup time. In addition to fixing the bug, install some guards so that any similar future mistake will be more obvious. •
Avoid O(N^2) behavior when inserting many tuples into a SPI query result (Neil Conway)
2223
Appendix E. Release Notes •
Improve LISTEN startup time when there are many unread notifications (Matt Newell)
•
Disable SSL renegotiation by default (Michael Paquier, Andres Freund) While use of SSL renegotiation is a good idea in theory, we have seen too many bugs in practice, both in the underlying OpenSSL library and in our usage of it. Renegotiation will be removed entirely in 9.5 and later. In the older branches, just change the default value of ssl_renegotiation_limit to zero (disabled).
•
Lower the minimum values of the *_freeze_max_age parameters (Andres Freund) This is mainly to make tests of related behavior less time-consuming, but it may also be of value for installations with limited disk space.
•
Limit the maximum value of wal_buffers to 2GB to avoid server crashes (Josh Berkus)
•
Fix rare internal overflow in multiplication of numeric values (Dean Rasheed)
•
Guard against hard-to-reach stack overflows involving record types, range types, json, jsonb, tsquery, ltxtquery and query_int (Noah Misch)
•
Fix handling of DOW and DOY in datetime input (Greg Stark) These tokens aren’t meant to be used in datetime values, but previously they resulted in opaque internal error messages rather than “invalid input syntax”.
•
Add more query-cancel checks to regular expression matching (Tom Lane)
•
Add recursion depth protections to regular expression, SIMILAR TO, and LIKE matching (Tom Lane) Suitable search patterns and a low stack depth limit could lead to stack-overrun crashes.
•
Fix potential infinite loop in regular expression execution (Tom Lane) A search pattern that can apparently match a zero-length string, but actually doesn’t match because of a back reference, could lead to an infinite loop.
•
Fix low-memory failures in regular expression compilation (Andreas Seltenreich)
•
Fix low-probability memory leak during regular expression execution (Tom Lane)
•
Fix rare low-memory failure in lock cleanup during transaction abort (Tom Lane)
•
Fix “unexpected out-of-memory situation during sort” errors when using tuplestores with small work_mem settings (Tom Lane)
•
Fix very-low-probability stack overrun in qsort (Tom Lane)
•
Fix “invalid memory alloc request size” failure in hash joins with large work_mem settings (Tomas Vondra, Tom Lane)
•
Fix assorted planner bugs (Tom Lane) These mistakes could lead to incorrect query plans that would give wrong answers, or to assertion failures in assert-enabled builds, or to odd planner errors such as “could not devise a query plan for the given query”, “could not find pathkey item to sort”, “plan should not reference subplan’s variable”, or “failed to assign all NestLoopParams to plan nodes”. Thanks are due to Andreas Seltenreich and Piotr Stefaniak for fuzz testing that exposed these problems.
•
Use fuzzy path cost tiebreaking rule in all supported branches (Tom Lane) This change is meant to avoid platform-specific behavior when alternative plan choices have effectively-identical estimated costs.
•
During postmaster shutdown, ensure that per-socket lock files are removed and listen sockets are closed before we remove the postmaster.pid file (Tom Lane)
2224
Appendix E. Release Notes This avoids race-condition failures if an external script attempts to start a new postmaster as soon as pg_ctl stop returns. •
Fix postmaster’s handling of a startup-process crash during crash recovery (Tom Lane) If, during a crash recovery cycle, the startup process crashes without having restored database consistency, we’d try to launch a new startup process, which typically would just crash again, leading to an infinite loop.
•
Do not print a WARNING when an autovacuum worker is already gone when we attempt to signal it, and reduce log verbosity for such signals (Tom Lane)
•
Prevent autovacuum launcher from sleeping unduly long if the server clock is moved backwards a large amount (Álvaro Herrera)
•
Ensure that cleanup of a GIN index’s pending-insertions list is interruptable by cancel requests (Jeff Janes)
•
Allow all-zeroes pages in GIN indexes to be reused (Heikki Linnakangas) Such a page might be left behind after a crash.
•
Fix off-by-one error that led to otherwise-harmless warnings about “apparent wraparound” in subtrans/multixact truncation (Thomas Munro)
•
Fix misreporting of CONTINUE and MOVE statement types in PL/pgSQL’s error context messages (Pavel Stehule, Tom Lane)
•
Fix some places in PL/Tcl that neglected to check for failure of malloc() calls (Michael Paquier, Álvaro Herrera)
•
Improve libpq’s handling of out-of-memory conditions (Michael Paquier, Heikki Linnakangas)
•
Fix memory leaks and missing out-of-memory checks in ecpg (Michael Paquier)
•
Fix psql’s code for locale-aware formatting of numeric output (Tom Lane) The formatting code invoked by \pset numericlocale on did the wrong thing for some uncommon cases such as numbers with an exponent but no decimal point. It could also mangle already-localized output from the money data type.
•
Prevent crash in psql’s \c command when there is no current connection (Noah Misch)
•
Ensure that temporary files created during a pg_dump run with tar-format output are not worldreadable (Michael Paquier)
•
Fix pg_dump and pg_upgrade to support cases where the postgres or template1 database is in a non-default tablespace (Marti Raudsepp, Bruce Momjian)
•
Fix pg_dump to handle object privileges sanely when dumping from a server too old to have a particular privilege type (Tom Lane) When dumping functions or procedural languages from pre-7.3 servers, pg_dump would produce GRANT/REVOKE commands that revoked the owner’s grantable privileges and instead granted all privileges to PUBLIC. Since the privileges involved are just USAGE and EXECUTE, this isn’t a security problem, but it’s certainly a surprising representation of the older systems’ behavior. Fix it to leave the default privilege state alone in these cases.
•
Fix pg_dump to dump shell types (Tom Lane) Shell types (that is, not-yet-fully-defined types) aren’t useful for much, but nonetheless pg_dump should dump them.
•
Fix spinlock assembly code for PPC hardware to be compatible with AIX’s native assembler (Tom Lane)
2225
Appendix E. Release Notes Building with gcc didn’t work if gcc had been configured to use the native assembler, which is becoming more common. •
On AIX, test the -qlonglong compiler option rather than just assuming it’s safe to use (Noah Misch)
•
On AIX, use -Wl,-brtllib link option to allow symbols to be resolved at runtime (Noah Misch) Perl relies on this ability in 5.8.0 and later.
•
Avoid use of inline functions when compiling with 32-bit xlc, due to compiler bugs (Noah Misch)
•
Use librt for sched_yield() when necessary, which it is on some Solaris versions (Oskari Saarenmaa)
•
Fix Windows install.bat script to handle target directory names that contain spaces (Heikki Linnakangas)
•
Make the numeric form of the PostgreSQL version number (e.g., 90405) readily available to extension Makefiles, as a variable named VERSION_NUM (Michael Paquier)
•
Update time zone data files to tzdata release 2015g for DST law changes in Cayman Islands, Fiji, Moldova, Morocco, Norfolk Island, North Korea, Turkey, and Uruguay. There is a new zone name America/Fort_Nelson for the Canadian Northern Rockies.
E.71. Release 9.0.22 Release date: 2015-06-12 This release contains a small number of fixes from 9.0.21. For information about new features in the 9.0 major release, see Section E.93. The PostgreSQL community will stop releasing updates for the 9.0.X release series in September 2015. Users are encouraged to update to a newer release branch soon.
E.71.1. Migration to Version 9.0.22 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.18, see Section E.75.
E.71.2. Changes •
Fix rare failure to invalidate relation cache init file (Tom Lane) With just the wrong timing of concurrent activity, a VACUUM FULL on a system catalog might fail to update the “init file” that’s used to avoid cache-loading work for new sessions. This would result in later sessions being unable to access that catalog at all. This is a very ancient bug, but it’s so hard to trigger that no reproducible case had been seen until recently.
•
Avoid deadlock between incoming sessions and CREATE/DROP DATABASE (Tom Lane) A new session starting in a database that is the target of a DROP DATABASE command, or is the template for a CREATE DATABASE command, could cause the command to wait for five seconds and then fail, even if the new session would have exited before that.
2226
Appendix E. Release Notes
E.72. Release 9.0.21 Release date: 2015-06-04 This release contains a small number of fixes from 9.0.20. For information about new features in the 9.0 major release, see Section E.93. The PostgreSQL community will stop releasing updates for the 9.0.X release series in September 2015. Users are encouraged to update to a newer release branch soon.
E.72.1. Migration to Version 9.0.21 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.18, see Section E.75.
E.72.2. Changes •
Avoid failures while fsync’ing data directory during crash restart (Abhijit Menon-Sen, Tom Lane) In the previous minor releases we added a patch to fsync everything in the data directory after a crash. Unfortunately its response to any error condition was to fail, thereby preventing the server from starting up, even when the problem was quite harmless. An example is that an unwritable file in the data directory would prevent restart on some platforms; but it is common to make SSL certificate files unwritable by the server. Revise this behavior so that permissions failures are ignored altogether, and other types of failures are logged but do not prevent continuing.
•
Remove configure’s check prohibiting linking to a threaded libpython on OpenBSD (Tom Lane) The failure this restriction was meant to prevent seems to not be a problem anymore on current OpenBSD versions.
•
Allow libpq to use TLS protocol versions beyond v1 (Noah Misch) For a long time, libpq was coded so that the only SSL protocol it would allow was TLS v1. Now that newer TLS versions are becoming popular, allow it to negotiate the highest commonly-supported TLS version with the server. (PostgreSQL servers were already capable of such negotiation, so no change is needed on the server side.) This is a back-patch of a change already released in 9.4.0.
E.73. Release 9.0.20 Release date: 2015-05-22 This release contains a variety of fixes from 9.0.19. For information about new features in the 9.0 major release, see Section E.93. The PostgreSQL community will stop releasing updates for the 9.0.X release series in September 2015. Users are encouraged to update to a newer release branch soon.
E.73.1. Migration to Version 9.0.20 A dump/restore is not required for those running 9.0.X.
2227
Appendix E. Release Notes However, if you are upgrading from a version earlier than 9.0.18, see Section E.75.
E.73.2. Changes •
Avoid possible crash when client disconnects just before the authentication timeout expires (Benkocs Norbert Attila) If the timeout interrupt fired partway through the session shutdown sequence, SSL-related state would be freed twice, typically causing a crash and hence denial of service to other sessions. Experimentation shows that an unauthenticated remote attacker could trigger the bug somewhat consistently, hence treat as security issue. (CVE-2015-3165)
•
Improve detection of system-call failures (Noah Misch) Our replacement implementation of snprintf() failed to check for errors reported by the underlying system library calls; the main case that might be missed is out-of-memory situations. In the worst case this might lead to information exposure, due to our code assuming that a buffer had been overwritten when it hadn’t been. Also, there were a few places in which security-relevant calls of other system library functions did not check for failure. It remains possible that some calls of the *printf() family of functions are vulnerable to information disclosure if an out-of-memory error occurs at just the wrong time. We judge the risk to not be large, but will continue analysis in this area. (CVE-2015-3166)
•
In contrib/pgcrypto, uniformly report decryption failures as “Wrong key or corrupt data” (Noah Misch) Previously, some cases of decryption with an incorrect key could report other error message texts. It has been shown that such variance in error reports can aid attackers in recovering keys from other systems. While it’s unknown whether pgcrypto’s specific behaviors are likewise exploitable, it seems better to avoid the risk by using a one-size-fits-all message. (CVE-2015-3167)
•
Fix incorrect checking of deferred exclusion constraints after a HOT update (Tom Lane) If a new row that potentially violates a deferred exclusion constraint is HOT-updated (that is, no indexed columns change and the row can be stored back onto the same table page) later in the same transaction, the exclusion constraint would be reported as violated when the check finally occurred, even if the row(s) the new row originally conflicted with had been deleted.
•
Prevent improper reordering of antijoins (NOT EXISTS joins) versus other outer joins (Tom Lane) This oversight in the planner has been observed to cause “could not find RelOptInfo for given relids” errors, but it seems possible that sometimes an incorrect query plan might get past that consistency check and result in silently-wrong query output.
•
Fix incorrect matching of subexpressions in outer-join plan nodes (Tom Lane) Previously, if textually identical non-strict subexpressions were used both above and below an outer join, the planner might try to re-use the value computed below the join, which would be incorrect because the executor would force the value to NULL in case of an unmatched outer row.
•
Fix GEQO planner to cope with failure of its join order heuristic (Tom Lane) This oversight has been seen to lead to “failed to join all relations together” errors in queries involving LATERAL, and that might happen in other cases as well.
•
Fix possible deadlock at startup when max_prepared_transactions is too small (Heikki Linnakangas)
2228
Appendix E. Release Notes •
Don’t archive useless preallocated WAL files after a timeline switch (Heikki Linnakangas)
•
Avoid “cannot GetMultiXactIdMembers() during recovery” error (Álvaro Herrera)
•
Recursively fsync() the data directory after a crash (Abhijit Menon-Sen, Robert Haas) This ensures consistency if another crash occurs shortly later. (The second crash would have to be a system-level crash, not just a database crash, for there to be a problem.)
•
Fix autovacuum launcher’s possible failure to shut down, if an error occurs after it receives SIGTERM (Álvaro Herrera)
•
Cope with unexpected signals in LockBufferForCleanup() (Andres Freund) This oversight could result in spurious errors about “multiple backends attempting to wait for pincount 1”.
•
Avoid waiting for WAL flush or synchronous replication during commit of a transaction that was read-only so far as the user is concerned (Andres Freund) Previously, a delay could occur at commit in transactions that had written WAL due to HOT page pruning, leading to undesirable effects such as sessions getting stuck at startup if all synchronous replicas are down. Sessions have also been observed to get stuck in catchup interrupt processing when using synchronous replication; this will fix that problem as well.
•
Fix crash when manipulating hash indexes on temporary tables (Heikki Linnakangas)
•
Fix possible failure during hash index bucket split, if other processes are modifying the index concurrently (Tom Lane)
•
Check for interrupts while analyzing index expressions (Jeff Janes) ANALYZE executes index expressions many times; if there are slow functions in such an expression, it’s desirable to be able to cancel the ANALYZE before that loop finishes.
•
Add the name of the target server to object description strings for foreign-server user mappings (Álvaro Herrera)
•
Recommend setting include_realm to 1 when using Kerberos/GSSAPI/SSPI authentication (Stephen Frost) Without this, identically-named users from different realms cannot be distinguished. For the moment this is only a documentation change, but it will become the default setting in PostgreSQL 9.5.
•
Remove code for matching IPv4 pg_hba.conf entries to IPv4-in-IPv6 addresses (Tom Lane) This hack was added in 2003 in response to a report that some Linux kernels of the time would report IPv4 connections as having IPv4-in-IPv6 addresses. However, the logic was accidentally broken in 9.0. The lack of any field complaints since then shows that it’s not needed anymore. Now we have reports that the broken code causes crashes on some systems, so let’s just remove it rather than fix it. (Had we chosen to fix it, that would make for a subtle and potentially security-sensitive change in the effective meaning of IPv4 pg_hba.conf entries, which does not seem like a good thing to do in minor releases.)
•
While shutting down service on Windows, periodically send status updates to the Service Control Manager to prevent it from killing the service too soon; and ensure that pg_ctl will wait for shutdown (Krystian Bigaj)
•
Reduce risk of network deadlock when using libpq’s non-blocking mode (Heikki Linnakangas) When sending large volumes of data, it’s important to drain the input buffer every so often, in case the server has sent enough response data to cause it to block on output. (A typical scenario is that the server is sending a stream of NOTICE messages during COPY FROM STDIN.) This worked
2229
Appendix E. Release Notes properly in the normal blocking mode, but not so much in non-blocking mode. We’ve modified libpq to opportunistically drain input when it can, but a full defense against this problem requires application cooperation: the application should watch for socket read-ready as well as write-ready conditions, and be sure to call PQconsumeInput() upon read-ready. •
Fix array handling in ecpg (Michael Meskes)
•
Fix psql to sanely handle URIs and conninfo strings as the first parameter to \connect (David Fetter, Andrew Dunstan, Álvaro Herrera) This syntax has been accepted (but undocumented) for a long time, but previously some parameters might be taken from the old connection instead of the given string, which was agreed to be undesirable.
•
Suppress incorrect complaints from psql on some platforms that it failed to write ~/.psql_history at exit (Tom Lane) This misbehavior was caused by a workaround for a bug in very old (pre-2006) versions of libedit. We fixed it by removing the workaround, which will cause a similar failure to appear for anyone still using such versions of libedit. Recommendation: upgrade that library, or use libreadline.
•
Fix pg_dump’s rule for deciding which casts are system-provided casts that should not be dumped (Tom Lane)
•
Fix dumping of views that are just VALUES(...) but have column aliases (Tom Lane)
•
In pg_upgrade, force timeline 1 in the new cluster (Bruce Momjian) This change prevents upgrade failures caused by bogus complaints about missing WAL history files.
•
In pg_upgrade, check for improperly non-connectable databases before proceeding (Bruce Momjian)
•
In pg_upgrade, quote directory paths properly in the generated delete_old_cluster script (Bruce Momjian)
•
In pg_upgrade, preserve database-level freezing info properly (Bruce Momjian) This oversight could cause missing-clog-file errors for tables within the postgres and template1 databases.
•
Run pg_upgrade and pg_resetxlog with restricted privileges on Windows, so that they don’t fail when run by an administrator (Muhammad Asif Naeem)
•
Fix slow sorting algorithm in contrib/intarray (Tom Lane)
•
Fix compile failure on Sparc V8 machines (Rob Rowan)
•
Update time zone data files to tzdata release 2015d for DST law changes in Egypt, Mongolia, and Palestine, plus historical changes in Canada and Chile. Also adopt revised zone abbreviations for the America/Adak zone (HST/HDT not HAST/HADT).
E.74. Release 9.0.19 Release date: 2015-02-05 This release contains a variety of fixes from 9.0.18. For information about new features in the 9.0 major release, see Section E.93.
2230
Appendix E. Release Notes
E.74.1. Migration to Version 9.0.19 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.18, see Section E.75.
E.74.2. Changes •
Fix buffer overruns in to_char() (Bruce Momjian) When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241)
•
Fix buffer overrun in replacement *printf() functions (Tom Lane) PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well. This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242)
•
Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch) Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243)
•
Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas) If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message’s data as a new protocol message. An attacker able to submit crafted binary data within a command parameter might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244)
•
Fix information leak via constraint-violation error messages (Stephen Frost) Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161)
•
Lock down regression testing’s temporary installations on Windows (Noah Misch)
2231
Appendix E. Release Notes Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067) •
Avoid possible data corruption if ALTER DATABASE SET TABLESPACE is used to move a database to a new tablespace and then shortly later move it back to its original tablespace (Tom Lane)
•
Avoid corrupting tables when ANALYZE inside a transaction is rolled back (Andres Freund, Tom Lane, Michael Paquier) If the failing transaction had earlier removed the last index, rule, or trigger from the table, the table would be left in a corrupted state with the relevant pg_class flags not set though they should be.
•
Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane) In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug.
•
Fix planning of SELECT FOR UPDATE when using a partial index on a child table (Kyotaro Horiguchi) In READ COMMITTED mode, SELECT FOR UPDATE must also recheck the partial index’s WHERE condition when rechecking a recently-updated row to see if it still satisfies the query’s WHERE condition. This requirement was missed if the index belonged to an inheritance child table, so that it was possible to incorrectly return rows that no longer satisfy the query condition.
•
Fix corner case wherein SELECT FOR UPDATE could return a row twice, and possibly miss returning other rows (Tom Lane) In READ COMMITTED mode, a SELECT FOR UPDATE that is scanning an inheritance tree could incorrectly return a row from a prior child table instead of the one it should return from a later child table.
•
Reject duplicate column names in the referenced-columns list of a FOREIGN KEY declaration (David Rowley) This restriction is per SQL standard. Previously we did not reject the case explicitly, but later on the code would fail with bizarre-looking errors.
•
Fix bugs in raising a numeric value to a large integral power (Tom Lane) The previous code could get a wrong answer, or consume excessive amounts of time and memory before realizing that the answer must overflow.
•
In numeric_recv(), truncate away any fractional digits that would be hidden according to the value’s dscale field (Tom Lane) A numeric value’s display scale (dscale) should never be less than the number of nonzero fractional digits; but apparently there’s at least one broken client application that transmits binary numeric values in which that’s true. This leads to strange behavior since the extra digits are taken into account by arithmetic operations even though they aren’t printed. The least risky fix seems to be to truncate away such “hidden” digits on receipt, so that the value is indeed what it prints as.
•
Reject out-of-range numeric timezone specifications (Tom Lane) Simple numeric timezone specifications exceeding +/- 168 hours (one week) would be accepted, but could then cause null-pointer dereference crashes in certain operations. There’s no use-case for such large UTC offsets, so reject them.
•
Fix bugs in tsquery @> tsquery operator (Heikki Linnakangas)
2232
Appendix E. Release Notes Two different terms would be considered to match if they had the same CRC. Also, if the second operand had more terms than the first, it would be assumed not to be contained in the first; which is wrong since it might contain duplicate terms. •
Improve ispell dictionary’s defenses against bad affix files (Tom Lane)
•
Allow more than 64K phrases in a thesaurus dictionary (David Boutin) The previous coding could crash on an oversize dictionary, so this was deemed a back-patchable bug fix rather than a feature addition.
•
Fix namespace handling in xpath() (Ali Akbar) Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation.
•
Fix planner problems with nested append relations, such as inherited tables within UNION ALL subqueries (Tom Lane)
•
Fail cleanly when a GiST index tuple doesn’t fit on a page, rather than going into infinite recursion (Andrew Gierth)
•
Exempt tables that have per-table cost_limit and/or cost_delay settings from autovacuum’s global cost balancing rules (Álvaro Herrera) The previous behavior resulted in basically ignoring these per-table settings, which was unintended. Now, a table having such settings will be vacuumed using those settings, independently of what is going on in other autovacuum workers. This may result in heavier total I/O load than before, so such settings should be re-examined for sanity.
•
Avoid wholesale autovacuuming when autovacuum is nominally off (Tom Lane) Even when autovacuum is nominally off, we will still launch autovacuum worker processes to vacuum tables that are at risk of XID wraparound. However, such a worker process then proceeded to vacuum all tables in the target database, if they met the usual thresholds for autovacuuming. This is at best pretty unexpected; at worst it delays response to the wraparound threat. Fix it so that if autovacuum is turned off, workers only do anti-wraparound vacuums and not any other work.
•
Fix race condition between hot standby queries and replaying a full-page image (Heikki Linnakangas) This mistake could result in transient errors in queries being executed in hot standby.
•
Fix several cases where recovery logic improperly ignored WAL records for COMMIT/ABORT PREPARED (Heikki Linnakangas) The most notable oversight was that recovery_target_xid could not be used to stop at a twophase commit.
•
Avoid creating unnecessary .ready marker files for timeline history files (Fujii Masao)
•
Fix possible null pointer dereference when an empty prepared statement is used and the log_statement setting is mod or ddl (Fujii Masao)
•
Change “pgstat wait timeout” warning message to be LOG level, and rephrase it to be more understandable (Tom Lane) This message was originally thought to be essentially a can’t-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads “using stale statistics instead of current ones because stats collector is not responding”.
2233
Appendix E. Release Notes •
Fix SPARC spinlock implementation to ensure correctness if the CPU is being run in a non-TSO coherency mode, as some non-Solaris kernels do (Andres Freund)
•
Warn if OS X’s setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch)
•
Fix processing of repeated dbname parameters in PQconnectdbParams() (Alex Shulgin) Unexpected behavior ensued if the first occurrence of dbname contained a connection string or URI to be expanded.
•
Ensure that libpq reports a suitable error message on unexpected socket EOF (Marko Tiikkaja, Tom Lane) Depending on kernel behavior, libpq might return an empty error string rather than something useful when the server unexpectedly closed the socket.
•
Clear any old error message during PQreset() (Heikki Linnakangas) If PQreset() is called repeatedly, and the connection cannot be re-established, error messages from the failed connection attempts kept accumulating in the PGconn’s error string.
•
Properly handle out-of-memory conditions while parsing connection options in libpq (Alex Shulgin, Heikki Linnakangas)
•
Fix array overrun in ecpg’s version of ParseDateTime() (Michael Paquier)
•
In initdb, give a clearer error message if a password file is specified but is empty (Mats Erik Andersson)
•
Fix psql’s \s command to work nicely with libedit, and add pager support (Stepan Rutz, Tom Lane) When using libedit rather than readline, \s printed the command history in a fairly unreadable encoded format, and on recent libedit versions might fail altogether. Fix that by printing the history ourselves rather than having the library do it. A pleasant side-effect is that the pager is used if appropriate. This patch also fixes a bug that caused newline encoding to be applied inconsistently when saving the command history with libedit. Multiline history entries written by older psql versions will be read cleanly with this patch, but perhaps not vice versa, depending on the exact libedit versions involved.
•
Improve consistency of parsing of psql’s special variables (Tom Lane) Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors.
•
Fix psql’s expanded-mode display to work consistently when using border = 3 and linestyle = ascii or unicode (Stephen Frost)
•
Fix possible deadlock during parallel restore of a schema-only dump (Robert Haas, Tom Lane)
•
Fix core dump in pg_dump --binary-upgrade on zero-column composite type (Rushabh Lathia)
•
Fix block number checking in contrib/pageinspect’s get_raw_page() (Tom Lane) The incorrect checking logic could prevent access to some pages in non-main relation forks.
•
Fix contrib/pgcrypto’s pgp_sym_decrypt() to not fail on messages whose length is 6 less than a power of 2 (Marko Tiikkaja)
2234
Appendix E. Release Notes •
Handle unexpected query results, especially NULLs, safely in contrib/tablefunc’s connectby() (Michael Paquier) connectby() previously crashed if it encountered a NULL key value. It now prints that row but
doesn’t recurse further. •
Avoid a possible crash in contrib/xml2’s xslt_process() (Mark Simonetti) libxslt seems to have an undocumented dependency on the order in which resources are freed; reorder our calls to avoid a crash.
•
Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier) These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues.
•
Detect incompatible OpenLDAP versions during build (Noah Misch) With OpenLDAP versions 2.4.24 through 2.4.31, inclusive, PostgreSQL backends can crash at exit. Raise a warning during configure based on the compile-time OpenLDAP version number, and test the crashing scenario in the contrib/dblink regression test.
•
In non-MSVC Windows builds, ensure libpq.dll is installed with execute permissions (Noah Misch)
•
Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane) This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations.
•
Support time zone abbreviations that change UTC offset from time to time (Tom Lane) Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date.
•
Update time zone abbreviations lists (Tom Lane) Add CST (China Standard Time) to our lists. Remove references to ADT as “Arabia Daylight Time”, an abbreviation that’s been out of use since 2007; therefore, claiming there is a conflict with “Atlantic Daylight Time” doesn’t seem especially helpful. Fix entirely incorrect GMT offsets for CKT (Cook Islands), FJT, and FJST (Fiji); we didn’t even have them on the proper side of the date line.
•
Update time zone data files to tzdata release 2015a. The IANA timezone database has adopted abbreviations of the form Ax ST/Ax DT for all Australian time zones, reflecting what they believe to be current majority practice Down Under. These names do not conflict with usage elsewhere (other than ACST for Acre Summer Time, which has been in disuse since 1994). Accordingly, adopt these names into our “Default” timezone abbreviation set. The “Australia” abbreviation set now contains only CST, EAST, EST, SAST, SAT, and WST, all of which are thought to be mostly historical usage. Note that SAST has also been changed to be South Africa Standard Time in the “Default” abbreviation set. Also, add zone abbreviations SRET (Asia/Srednekolymsk) and XJT (Asia/Urumqi), and use WSST/WSDT for western Samoa. Also, there were DST law changes in Chile, Mexico, the Turks
2235
Appendix E. Release Notes & Caicos Islands (America/Grand_Turk), and Fiji. There is a new zone Pacific/Bougainville for portions of Papua New Guinea. Also, numerous corrections for historical (pre-1970) time zone data.
E.75. Release 9.0.18 Release date: 2014-07-24 This release contains a variety of fixes from 9.0.17. For information about new features in the 9.0 major release, see Section E.93.
E.75.1. Migration to Version 9.0.18 A dump/restore is not required for those running 9.0.X. However, this release corrects an index corruption problem in some GiST indexes. See the first changelog entry below to find out whether your installation has been affected and what steps you should take if so. Also, if you are upgrading from a version earlier than 9.0.15, see Section E.78.
E.75.2. Changes •
Correctly initialize padding bytes in contrib/btree_gist indexes on bit columns (Heikki Linnakangas) This error could result in incorrect query results due to values that should compare equal not being seen as equal. Users with GiST indexes on bit or bit varying columns should REINDEX those indexes after installing this update.
•
Protect against torn pages when deleting GIN list pages (Heikki Linnakangas) This fix prevents possible index corruption if a system crash occurs while the page update is being written to disk.
•
Don’t clear the right-link of a GiST index page while replaying updates from WAL (Heikki Linnakangas) This error could lead to transiently wrong answers from GiST index scans performed in Hot Standby.
•
Fix
possibly-incorrect
cache
invalidation
during
nested
calls
to
ReceiveSharedInvalidMessages (Andres Freund) •
Don’t assume a subquery’s output is unique if there’s a set-returning function in its targetlist (David Rowley) This oversight could lead to misoptimization of constructs like WHERE x IN (SELECT y, generate_series(1,10) FROM t GROUP BY y).
•
Fix failure to detoast fields in composite elements of structured types (Tom Lane) This corrects cases where TOAST pointers could be copied into other tables without being dereferenced. If the original data is later deleted, it would lead to errors like “missing chunk number 0 for toast value ...” when the now-dangling pointer is used.
2236
Appendix E. Release Notes •
Fix “record type has not been registered” failures with whole-row references to the output of Append plan nodes (Tom Lane)
•
Fix possible crash when invoking a user-defined function while rewinding a cursor (Tom Lane)
•
Fix query-lifespan memory leak while evaluating the arguments for a function in FROM (Tom Lane)
•
Fix session-lifespan memory leaks in regular-expression processing (Tom Lane, Arthur O’Dwyer, Greg Stark)
•
Fix data encoding error in hungarian.stop (Tom Lane)
•
Fix liveness checks for rows that were inserted in the current transaction and then deleted by a now-rolled-back subtransaction (Andres Freund) This could cause problems (at least spurious warnings, and at worst an infinite loop) if CREATE INDEX or CLUSTER were done later in the same transaction.
•
Clear pg_stat_activity.xact_start during PREPARE TRANSACTION (Andres Freund) After the PREPARE, the originating session is no longer in a transaction, so it should not continue to display a transaction start time.
•
Fix REASSIGN OWNED to not fail for text search objects (Álvaro Herrera)
•
Block signals during postmaster startup (Tom Lane) This ensures that the postmaster will properly clean up after itself if, for example, it receives SIGINT while still starting up.
•
Secure Unix-domain sockets of temporary postmasters started during make check (Noah Misch) Any local user able to access the socket file could connect as the server’s bootstrap superuser, then proceed to execute arbitrary code as the operating-system user running the test, as we previously noted in CVE-2014-0067. This change defends against that risk by placing the server’s socket in a temporary, mode 0700 subdirectory of /tmp. The hazard remains however on platforms where Unix sockets are not supported, notably Windows, because then the temporary postmaster must accept local TCP connections. A useful side effect of this change is to simplify make check testing in builds that override DEFAULT_PGSOCKET_DIR. Popular non-default values like /var/run/postgresql are often not writable by the build user, requiring workarounds that will no longer be necessary.
•
Fix tablespace creation WAL replay to work on Windows (MauMau)
•
Fix detection of socket creation failures on Windows (Bruce Momjian)
•
On Windows, allow new sessions to absorb values of PGC_BACKEND parameters (such as log_connections) from the configuration file (Amit Kapila) Previously, if such a parameter were changed in the file post-startup, the change would have no effect.
•
Properly quote executable path names on Windows (Nikhil Deshpande) This oversight could cause initdb and pg_upgrade to fail on Windows, if the installation path contained both spaces and @ signs.
•
Fix linking of libpython on OS X (Tom Lane) The method we previously used can fail with the Python library supplied by Xcode 5.0 and later.
•
Avoid buffer bloat in libpq when the server consistently sends data faster than the client can absorb it (Shin-ichi Morita, Tom Lane)
2237
Appendix E. Release Notes libpq could be coerced into enlarging its input buffer until it runs out of memory (which would be reported misleadingly as “lost synchronization with server”). Under ordinary circumstances it’s quite far-fetched that data could be continuously transmitted more quickly than the recv() loop can absorb it, but this has been observed when the client is artificially slowed by scheduler constraints. •
Ensure that LDAP lookup attempts in libpq time out as intended (Laurenz Albe)
•
Fix ecpg to do the right thing when an array of char * is the target for a FETCH statement returning more than one row, as well as some other array-handling fixes (Ashutosh Bapat)
•
Fix pg_restore’s processing of old-style large object comments (Tom Lane) A direct-to-database restore from an archive file generated by a pre-9.0 version of pg_dump would usually fail if the archive contained more than a few comments for large objects.
•
In contrib/pgcrypto functions, ensure sensitive information is cleared from stack variables before returning (Marko Kreen)
•
In contrib/uuid-ossp, cache the state of the OSSP UUID library across calls (Tom Lane) This improves the efficiency of UUID generation and reduces the amount of entropy drawn from /dev/urandom, on platforms that have that.
•
Update time zone data files to tzdata release 2014e for DST law changes in Crimea, Egypt, and Morocco.
E.76. Release 9.0.17 Release date: 2014-03-20 This release contains a variety of fixes from 9.0.16. For information about new features in the 9.0 major release, see Section E.93.
E.76.1. Migration to Version 9.0.17 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.15, see Section E.78.
E.76.2. Changes •
Restore GIN metapages unconditionally to avoid torn-page risk (Heikki Linnakangas) Although this oversight could theoretically result in a corrupted index, it is unlikely to have caused any problems in practice, since the active part of a GIN metapage is smaller than a standard 512byte disk sector.
•
Avoid race condition in checking transaction commit status during receipt of a NOTIFY message (Marko Tiikkaja) This prevents a scenario wherein a sufficiently fast client might respond to a notification before database updates made by the notifier have become visible to the recipient.
•
Allow regular-expression operators to be terminated early by query cancel requests (Tom Lane)
2238
Appendix E. Release Notes This prevents scenarios wherein a pathological regular expression could lock up a server process uninterruptibly for a long time. •
Remove incorrect code that tried to allow OVERLAPS with single-element row arguments (Joshua Yanovski) This code never worked correctly, and since the case is neither specified by the SQL standard nor documented, it seemed better to remove it than fix it.
•
Avoid getting more than AccessShareLock when de-parsing a rule or view (Dean Rasheed) This oversight resulted in pg_dump unexpectedly acquiring RowExclusiveLock locks on tables mentioned as the targets of INSERT/UPDATE/DELETE commands in rules. While usually harmless, that could interfere with concurrent transactions that tried to acquire, for example, ShareLock on those tables.
•
Improve performance of index endpoint probes during planning (Tom Lane) This change fixes a significant performance problem that occurred when there were many not-yetcommitted rows at the end of the index, which is a common situation for indexes on sequentiallyassigned values such as timestamps or sequence-generated identifiers.
•
Fix test to see if hot standby connections can be allowed immediately after a crash (Heikki Linnakangas)
•
Prevent interrupts while reporting non-ERROR messages (Tom Lane) This guards against rare server-process freezeups due to recursive entry to syslog(), and perhaps other related problems.
•
Prevent intermittent “could not reserve shared memory region” failures on recent Windows versions (MauMau)
•
Update time zone data files to tzdata release 2014a for DST law changes in Fiji and Turkey, plus historical changes in Israel and Ukraine.
E.77. Release 9.0.16 Release date: 2014-02-20 This release contains a variety of fixes from 9.0.15. For information about new features in the 9.0 major release, see Section E.93.
E.77.1. Migration to Version 9.0.16 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.15, see Section E.78.
E.77.2. Changes •
Shore up GRANT ... WITH ADMIN OPTION restrictions (Noah Misch) Granting a role without ADMIN OPTION is supposed to prevent the grantee from adding or removing members from the granted role, but this restriction was easily bypassed by doing SET ROLE
2239
Appendix E. Release Notes first. The security impact is mostly that a role member can revoke the access of others, contrary to the wishes of his grantor. Unapproved role member additions are a lesser concern, since an uncooperative role member could provide most of his rights to others anyway by creating views or SECURITY DEFINER functions. (CVE-2014-0060) •
Prevent privilege escalation via manual calls to PL validator functions (Andres Freund) The primary role of PL validator functions is to be called implicitly during CREATE FUNCTION, but they are also normal SQL functions that a user can call explicitly. Calling a validator on a function actually written in some other language was not checked for and could be exploited for privilege-escalation purposes. The fix involves adding a call to a privilege-checking function in each validator function. Non-core procedural languages will also need to make this change to their own validator functions, if any. (CVE-2014-0061)
•
Avoid multiple name lookups during table and index DDL (Robert Haas, Andres Freund) If the name lookups come to different conclusions due to concurrent activity, we might perform some parts of the DDL on a different table than other parts. At least in the case of CREATE INDEX, this can be used to cause the permissions checks to be performed against a different table than the index creation, allowing for a privilege escalation attack. (CVE-2014-0062)
•
Prevent buffer overrun with long datetime strings (Noah Misch) The MAXDATELEN constant was too small for the longest possible value of type interval, allowing a buffer overrun in interval_out(). Although the datetime input functions were more careful about avoiding buffer overrun, the limit was short enough to cause them to reject some valid inputs, such as input containing a very long timezone name. The ecpg library contained these vulnerabilities along with some of its own. (CVE-2014-0063)
•
Prevent buffer overrun due to integer overflow in size calculations (Noah Misch, Heikki Linnakangas) Several functions, mostly type input functions, calculated an allocation size without checking for overflow. If overflow did occur, a too-small buffer would be allocated and then written past. (CVE2014-0064)
•
Prevent overruns of fixed-size buffers (Peter Eisentraut, Jozef Mlich) Use strlcpy() and related functions to provide a clear guarantee that fixed-size buffers are not overrun. Unlike the preceding items, it is unclear whether these cases really represent live issues, since in most cases there appear to be previous constraints on the size of the input string. Nonetheless it seems prudent to silence all Coverity warnings of this type. (CVE-2014-0065)
•
Avoid crashing if crypt() returns NULL (Honza Horak, Bruce Momjian) There are relatively few scenarios in which crypt() could return NULL, but contrib/chkpass would crash if it did. One practical case in which this could be an issue is if libc is configured to refuse to execute unapproved hashing algorithms (e.g., “FIPS mode”). (CVE-2014-0066)
•
Document risks of make check in the regression testing instructions (Noah Misch, Tom Lane) Since the temporary server started by make check uses “trust” authentication, another user on the same machine could connect to it as database superuser, and then potentially exploit the privileges of the operating-system user who started the tests. A future release will probably incorporate changes in the testing procedure to prevent this risk, but some public discussion is needed first. So for the moment, just warn people against using make check when there are untrusted users on the same machine. (CVE-2014-0067)
•
Fix possible mis-replay of WAL records when some segments of a relation aren’t full size (Greg Stark, Tom Lane)
2240
Appendix E. Release Notes The WAL update could be applied to the wrong page, potentially many pages past where it should have been. Aside from corrupting data, this error has been observed to result in significant “bloat” of standby servers compared to their masters, due to updates being applied far beyond where the end-of-file should have been. This failure mode does not appear to be a significant risk during crash recovery, only when initially synchronizing a standby created from a base backup taken from a quickly-changing master. •
Fix bug in determining when recovery has reached consistency (Tomonari Katsumata, Heikki Linnakangas) In some cases WAL replay would mistakenly conclude that the database was already consistent at the start of replay, thus possibly allowing hot-standby queries before the database was really consistent. Other symptoms such as “PANIC: WAL contains references to invalid pages” were also possible.
•
Fix improper locking of btree index pages while replaying a VACUUM operation in hot-standby mode (Andres Freund, Heikki Linnakangas, Tom Lane) This error could result in “PANIC: WAL contains references to invalid pages” failures.
•
Ensure that insertions into non-leaf GIN index pages write a full-page WAL record when appropriate (Heikki Linnakangas) The previous coding risked index corruption in the event of a partial-page write during a system crash.
•
Fix race conditions during server process exit (Robert Haas) Ensure that signal handlers don’t attempt to use the process’s MyProc pointer after it’s no longer valid.
•
Fix unsafe references to errno within error reporting logic (Christian Kruse) This would typically lead to odd behaviors such as missing or inappropriate HINT fields.
•
Fix possible crashes from using ereport() too early during server startup (Tom Lane) The principal case we’ve seen in the field is a crash if the server is started in a directory it doesn’t have permission to read.
•
Clear retry flags properly in OpenSSL socket write function (Alexander Kukushkin) This omission could result in a server lockup after unexpected loss of an SSL-encrypted connection.
•
Fix length checking for Unicode identifiers (U&"..." syntax) containing escapes (Tom Lane) A spurious truncation warning would be printed for such identifiers if the escaped form of the identifier was too long, but the identifier actually didn’t need truncation after de-escaping.
•
Allow keywords that are type names to be used in lists of roles (Stephen Frost) A previous patch allowed such keywords to be used without quoting in places such as role identifiers; but it missed cases where a list of role identifiers was permitted, such as DROP ROLE.
•
Fix possible crash due to invalid plan for nested sub-selects, such as WHERE (... x IN (SELECT ...) ...) IN (SELECT ...) (Tom Lane)
•
Ensure that ANALYZE creates statistics for a table column even when all the values in it are “too wide” (Tom Lane) ANALYZE intentionally omits very wide values from its histogram and most-common-values cal-
culations, but it neglected to do something sane in the case that all the sampled entries are too wide.
2241
Appendix E. Release Notes •
In ALTER TABLE ... SET TABLESPACE, allow the database’s default tablespace to be used without a permissions check (Stephen Frost) CREATE TABLE has always allowed such usage, but ALTER TABLE didn’t get the memo.
•
Fix “cannot accept a set” error when some arms of a CASE return a set and others don’t (Tom Lane)
•
Fix checks for all-zero client addresses in pgstat functions (Kevin Grittner)
•
Fix possible misclassification of multibyte characters by the text search parser (Tom Lane) Non-ASCII characters could be misclassified when using C locale with a multibyte encoding. On Cygwin, non-C locales could fail as well.
•
Fix possible misbehavior in plainto_tsquery() (Heikki Linnakangas) Use memmove() not memcpy() for copying overlapping memory regions. There have been no field reports of this actually causing trouble, but it’s certainly risky.
•
Accept SHIFT_JIS as an encoding name for locale checking purposes (Tatsuo Ishii)
•
Fix misbehavior of PQhost() on Windows (Fujii Masao) It should return localhost if no host has been specified.
•
Improve error handling in libpq and psql for failures during COPY TO STDOUT/FROM STDIN (Tom Lane) In particular this fixes an infinite loop that could occur in 9.2 and up if the server connection was lost during COPY FROM STDIN. Variants of that scenario might be possible in older versions, or with other client applications.
•
Fix misaligned descriptors in ecpg (MauMau)
•
In ecpg, handle lack of a hostname in the connection parameters properly (Michael Meskes)
•
Fix performance regression in contrib/dblink connection startup (Joe Conway) Avoid an unnecessary round trip when client and server encodings match.
•
In contrib/isn, fix incorrect calculation of the check digit for ISMN values (Fabien Coelho)
•
Ensure client-code-only installation procedure works as documented (Peter Eisentraut)
•
In Mingw and Cygwin builds, install the libpq DLL in the bin directory (Andrew Dunstan) This duplicates what the MSVC build has long done. It should fix problems with programs like psql failing to start because they can’t find the DLL.
•
Avoid using the deprecated dllwrap tool in Cygwin builds (Marco Atzeri)
•
Don’t generate plain-text HISTORY and src/test/regress/README files anymore (Tom Lane) These text files duplicated the main HTML and PDF documentation formats. The trouble involved in maintaining them greatly outweighs the likely audience for plain-text format. Distribution tarballs will still contain files by these names, but they’ll just be stubs directing the reader to consult the main documentation. The plain-text INSTALL file will still be maintained, as there is arguably a use-case for that.
•
Update time zone data files to tzdata release 2013i for DST law changes in Jordan and historical changes in Cuba. In addition, the zones Asia/Riyadh87, Asia/Riyadh88, and Asia/Riyadh89 have been removed, as they are no longer maintained by IANA, and never represented actual civil timekeeping practice.
2242
Appendix E. Release Notes
E.78. Release 9.0.15 Release date: 2013-12-05 This release contains a variety of fixes from 9.0.14. For information about new features in the 9.0 major release, see Section E.93.
E.78.1. Migration to Version 9.0.15 A dump/restore is not required for those running 9.0.X. However, this release corrects a number of potential data corruption issues. See the first two changelog entries below to find out whether your installation has been affected and what steps you can take if so. Also, if you are upgrading from a version earlier than 9.0.13, see Section E.80.
E.78.2. Changes •
Fix VACUUM’s tests to see whether it can update relfrozenxid (Andres Freund) In some cases VACUUM (either manual or autovacuum) could incorrectly advance a table’s relfrozenxid value, allowing tuples to escape freezing, causing those rows to become invisible once 2^31 transactions have elapsed. The probability of data loss is fairly low since multiple incorrect advancements would need to happen before actual loss occurs, but it’s not zero. Users upgrading from releases 9.0.4 or 8.4.8 or earlier are not affected, but all later versions contain the bug. The issue can be ameliorated by, after upgrading, vacuuming all tables in all databases while having vacuum_freeze_table_age set to zero. This will fix any latent corruption but will not be able to fix all pre-existing data errors. However, an installation can be presumed safe after performing this vacuuming if it has executed fewer than 2^31 update transactions in its lifetime (check this with SELECT txid_current() < 2^31).
•
Fix initialization of pg_clog and pg_subtrans during hot standby startup (Andres Freund, Heikki Linnakangas) This bug can cause data loss on standby servers at the moment they start to accept hot-standby queries, by marking committed transactions as uncommitted. The likelihood of such corruption is small unless, at the time of standby startup, the primary server has executed many updating transactions since its last checkpoint. Symptoms include missing rows, rows that should have been deleted being still visible, and obsolete versions of updated rows being still visible alongside their newer versions. This bug was introduced in versions 9.3.0, 9.2.5, 9.1.10, and 9.0.14. Standby servers that have only been running earlier releases are not at risk. It’s recommended that standby servers that have ever run any of the buggy releases be re-cloned from the primary (e.g., with a new base backup) after upgrading.
•
Truncate pg_multixact contents during WAL replay (Andres Freund) This avoids ever-increasing disk space consumption in standby servers.
•
Fix race condition in GIN index posting tree page deletion (Heikki Linnakangas) This could lead to transient wrong answers or query failures.
2243
Appendix E. Release Notes •
Avoid flattening a subquery whose SELECT list contains a volatile function wrapped inside a subSELECT (Tom Lane) This avoids unexpected results due to extra evaluations of the volatile function.
•
Fix planner’s processing of non-simple-variable subquery outputs nested within outer joins (Tom Lane) This error could lead to incorrect plans for queries involving multiple levels of subqueries within JOIN syntax.
•
Fix premature deletion of temporary files (Andres Freund)
•
Fix possible read past end of memory in rule printing (Peter Eisentraut)
•
Fix array slicing of int2vector and oidvector values (Tom Lane) Expressions of this kind are now implicitly promoted to regular int2 or oid arrays.
•
Fix incorrect behaviors when using a SQL-standard, simple GMT offset timezone (Tom Lane) In some cases, the system would use the simple GMT offset value when it should have used the regular timezone setting that had prevailed before the simple offset was selected. This change also causes the timeofday function to honor the simple GMT offset zone.
•
Prevent possible misbehavior when logging translations of Windows error codes (Tom Lane)
•
Properly quote generated command lines in pg_ctl (Naoya Anzai and Tom Lane) This fix applies only to Windows.
•
Fix pg_dumpall to work when a source database sets default_transaction_read_only via ALTER DATABASE SET (Kevin Grittner) Previously, the generated script would fail during restore.
•
Fix ecpg’s processing of lists of variables declared varchar (Zoltán Böszörményi)
•
Make contrib/lo defend against incorrect trigger definitions (Marc Cousin)
•
Update time zone data files to tzdata release 2013h for DST law changes in Argentina, Brazil, Jordan, Libya, Liechtenstein, Morocco, and Palestine. Also, new timezone abbreviations WIB, WIT, WITA for Indonesia.
E.79. Release 9.0.14 Release date: 2013-10-10 This release contains a variety of fixes from 9.0.13. For information about new features in the 9.0 major release, see Section E.93.
E.79.1. Migration to Version 9.0.14 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.13, see Section E.80.
2244
Appendix E. Release Notes
E.79.2. Changes •
Prevent corruption of multi-byte characters when attempting to case-fold identifiers (Andrew Dunstan) PostgreSQL case-folds non-ASCII characters only when using a single-byte server encoding.
•
Fix checkpoint memory leak in background writer when wal_level = hot_standby (Naoya Anzai)
•
Fix memory leak caused by lo_open() failure (Heikki Linnakangas)
•
Fix memory overcommit bug when work_mem is using more than 24GB of memory (Stephen Frost)
•
Fix deadlock bug in libpq when using SSL (Stephen Frost)
•
Fix possible SSL state corruption in threaded libpq applications (Nick Phillips, Stephen Frost)
•
Properly compute row estimates for boolean columns containing many NULL values (Andrew Gierth) Previously tests like col IS NOT TRUE and col IS NOT FALSE did not properly factor in NULL values when estimating plan costs.
•
Prevent pushing down WHERE clauses into unsafe UNION/INTERSECT subqueries (Tom Lane) Subqueries of a UNION or INTERSECT that contain set-returning functions or volatile functions in their SELECT lists could be improperly optimized, leading to run-time errors or incorrect query results.
•
Fix rare case of “failed to locate grouping columns” planner failure (Tom Lane)
•
Improve view dumping code’s handling of dropped columns in referenced tables (Tom Lane)
•
Properly record index comments created using UNIQUE and PRIMARY KEY syntax (Andres Freund) This fixes a parallel pg_restore failure.
•
Fix REINDEX TABLE and REINDEX DATABASE to properly revalidate constraints and mark invalidated indexes as valid (Noah Misch) REINDEX INDEX has always worked properly.
•
Fix possible deadlock during concurrent CREATE INDEX CONCURRENTLY operations (Tom Lane)
•
Fix regexp_matches() handling of zero-length matches (Jeevan Chalke) Previously, zero-length matches like ’^’ could return too many matches.
•
Fix crash for overly-complex regular expressions (Heikki Linnakangas)
•
Fix regular expression match failures for back references combined with non-greedy quantifiers (Jeevan Chalke)
•
Prevent CREATE FUNCTION from checking SET variables unless function body checking is enabled (Tom Lane)
•
Allow ALTER DEFAULT PRIVILEGES to operate on schemas without requiring CREATE permission (Tom Lane)
•
Loosen restriction on keywords used in queries (Tom Lane) Specifically, lessen keyword restrictions for role names, language names, EXPLAIN and COPY options, and SET values. This allows COPY ... (FORMAT BINARY) to work as expected; previously BINARY needed to be quoted.
•
Fix pgp_pub_decrypt() so it works for secret keys with passwords (Marko Kreen)
2245
Appendix E. Release Notes •
Remove rare inaccurate warning during vacuum of index-less tables (Heikki Linnakangas)
•
Ensure that VACUUM ANALYZE still runs the ANALYZE phase if its attempt to truncate the file is cancelled due to lock conflicts (Kevin Grittner)
•
Avoid possible failure when performing transaction control commands (e.g ROLLBACK) in prepared queries (Tom Lane)
•
Ensure that floating-point data input accepts standard spellings of “infinity” on all platforms (Tom Lane) The C99 standard says that allowable spellings are inf, +inf, -inf, infinity, +infinity, and -infinity. Make sure we recognize these even if the platform’s strtod function doesn’t.
•
Expand ability to compare rows to records and arrays (Rafal Rzepecki, Tom Lane)
•
Update time zone data files to tzdata release 2013d for DST law changes in Israel, Morocco, Palestine, and Paraguay. Also, historical zone data corrections for Macquarie Island.
E.80. Release 9.0.13 Release date: 2013-04-04 This release contains a variety of fixes from 9.0.12. For information about new features in the 9.0 major release, see Section E.93.
E.80.1. Migration to Version 9.0.13 A dump/restore is not required for those running 9.0.X. However, this release corrects several errors in management of GiST indexes. After installing this update, it is advisable to REINDEX any GiST indexes that meet one or more of the conditions described below. Also, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.80.2. Changes •
Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi) A connection request containing a database name that begins with “-” could be crafted to damage or destroy files within the server’s data directory, even if the request is eventually rejected. (CVE2013-1899)
•
Reset OpenSSL randomness state in each postmaster child process (Marko Kreen) This avoids a scenario wherein random numbers generated by contrib/pgcrypto functions might be relatively easy for another database user to guess. The risk is only significant when the postmaster is configured with ssl = on but most connections don’t use SSL encryption. (CVE2013-1900)
•
Fix GiST indexes to not use “fuzzy” geometric comparisons when it’s not appropriate to do so (Alexander Korotkov)
2246
Appendix E. Release Notes The core geometric types perform comparisons using “fuzzy” equality, but gist_box_same must do exact comparisons, else GiST indexes using it might become inconsistent. After installing this update, users should REINDEX any GiST indexes on box, polygon, circle, or point columns, since all of these use gist_box_same. •
Fix erroneous range-union and penalty logic in GiST indexes that use contrib/btree_gist for variable-width data types, that is text, bytea, bit, and numeric columns (Tom Lane) These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in useless index bloat. Users are advised to REINDEX such indexes after installing this update.
•
Fix bugs in GiST page splitting code for multi-column indexes (Tom Lane) These errors could result in inconsistent indexes in which some keys that are present would not be found by searches, and also in indexes that are unnecessarily inefficient to search. Users are advised to REINDEX multi-column GiST indexes after installing this update.
•
Fix gist_point_consistent to handle fuzziness consistently (Alexander Korotkov) Index scans on GiST indexes on point columns would sometimes yield results different from a sequential scan, because gist_point_consistent disagreed with the underlying operator code about whether to do comparisons exactly or fuzzily.
•
Fix buffer leak in WAL replay (Heikki Linnakangas) This bug could result in “incorrect local pin count” errors during replay, making recovery impossible.
•
Fix race condition in DELETE RETURNING (Tom Lane) Under the right circumstances, DELETE RETURNING could attempt to fetch data from a shared buffer that the current process no longer has any pin on. If some other process changed the buffer meanwhile, this would lead to garbage RETURNING output, or even a crash.
•
Fix infinite-loop risk in regular expression compilation (Tom Lane, Don Porter)
•
Fix potential null-pointer dereference in regular expression compilation (Tom Lane)
•
Fix to_char() to use ASCII-only case-folding rules where appropriate (Tom Lane) This fixes misbehavior of some template patterns that should be locale-independent, but mishandled “I” and “i” in Turkish locales.
•
Fix unwanted rejection of timestamp 1999-12-31 24:00:00 (Tom Lane)
•
Fix logic error when a single transaction does UNLISTEN then LISTEN (Tom Lane) The session wound up not listening for notify events at all, though it surely should listen in this case.
•
Remove useless “picksplit doesn’t support secondary split” log messages (Josh Hansen, Tom Lane) This message seems to have been added in expectation of code that was never written, and probably never will be, since GiST’s default handling of secondary splits is actually pretty good. So stop nagging end users about it.
•
Fix possible failure to send a session’s last few transaction commit/abort counts to the statistics collector (Tom Lane)
•
Eliminate memory leaks in PL/Perl’s spi_prepare() function (Alex Hunsaker, Tom Lane)
Avoid crash in pg_dump when an incorrect connection string is given (Heikki Linnakangas)
2247
Appendix E. Release Notes •
Ignore invalid indexes in pg_dump and pg_upgrade (Michael Paquier, Bruce Momjian) Dumping invalid indexes can cause problems at restore time, for example if the reason the index creation failed was because it tried to enforce a uniqueness condition not satisfied by the table’s data. Also, if the index creation is in fact still in progress, it seems reasonable to consider it to be an uncommitted DDL change, which pg_dump wouldn’t be expected to dump anyway. pg_upgrade now also skips invalid indexes rather than failing.
•
Fix contrib/pg_trgm’s similarity() function to return zero for trigram-less strings (Tom Lane) Previously it returned NaN due to internal division by zero.
•
Update time zone data files to tzdata release 2013b for DST law changes in Chile, Haiti, Morocco, Paraguay, and some Russian areas. Also, historical zone data corrections for numerous places. Also, update the time zone abbreviation files for recent changes in Russia and elsewhere: CHOT, GET, IRKT, KGT, KRAT, MAGT, MAWT, MSK, NOVT, OMST, TKT, VLAT, WST, YAKT, YEKT now follow their current meanings, and VOLT (Europe/Volgograd) and MIST (Antarctica/Macquarie) are added to the default abbreviations list.
E.81. Release 9.0.12 Release date: 2013-02-07 This release contains a variety of fixes from 9.0.11. For information about new features in the 9.0 major release, see Section E.93.
E.81.1. Migration to Version 9.0.12 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.81.2. Changes •
Prevent execution of enum_recv from SQL (Tom Lane) The function was misdeclared, allowing a simple SQL command to crash the server. In principle an attacker might be able to use it to examine the contents of server memory. Our thanks to Sumit Soni (via Secunia SVCRP) for reporting this issue. (CVE-2013-0255)
•
Fix multiple problems in detection of when a consistent database state has been reached during WAL replay (Fujii Masao, Heikki Linnakangas, Simon Riggs, Andres Freund)
•
Update minimum recovery point when truncating a relation file (Heikki Linnakangas) Once data has been discarded, it’s no longer safe to stop recovery at an earlier point in the timeline.
•
Fix missing cancellations in hot standby mode (Noah Misch, Simon Riggs) The need to cancel conflicting hot-standby queries would sometimes be missed, allowing those queries to see inconsistent data.
•
Fix SQL grammar to allow subscripting or field selection from a sub-SELECT result (Tom Lane)
2248
Appendix E. Release Notes •
Fix performance problems with autovacuum truncation in busy workloads (Jan Wieck) Truncation of empty pages at the end of a table requires exclusive lock, but autovacuum was coded to fail (and release the table lock) when there are conflicting lock requests. Under load, it is easily possible that truncation would never occur, resulting in table bloat. Fix by performing a partial truncation, releasing the lock, then attempting to re-acquire the lock and continue. This fix also greatly reduces the average time before autovacuum releases the lock after a conflicting request arrives.
•
Protect against race conditions when scanning pg_tablespace (Stephen Frost, Tom Lane) CREATE DATABASE and DROP DATABASE could misbehave if there were concurrent updates of pg_tablespace entries.
•
Prevent DROP OWNED from trying to drop whole databases or tablespaces (Álvaro Herrera) For safety, ownership of these objects must be reassigned, not dropped.
•
Fix error in vacuum_freeze_table_age implementation (Andres Freund) In installations that have existed for more than vacuum_freeze_min_age transactions, this mistake prevented autovacuum from using partial-table scans, so that a full-table scan would always happen instead.
•
Prevent misbehavior when a RowExpr or XmlExpr is parse-analyzed twice (Andres Freund, Tom Lane) This mistake could be user-visible in contexts such as CREATE TABLE LIKE INCLUDING INDEXES.
•
Improve defenses against integer overflow in hashtable sizing calculations (Jeff Davis)
•
Reject out-of-range dates in to_date() (Hitoshi Harada)
•
Ensure that non-ASCII prompt strings are translated to the correct code page on Windows (Alexander Law, Noah Misch) This bug affected psql and some other client programs.
•
Fix possible crash in psql’s \? command when not connected to a database (Meng Qingzhong)
•
Fix pg_upgrade to deal with invalid indexes safely (Bruce Momjian)
•
Fix one-byte buffer overrun in libpq’s PQprintTuples (Xi Wang) This ancient function is not used anywhere by PostgreSQL itself, but it might still be used by some client code.
•
Make ecpglib use translated messages properly (Chen Huajun)
•
Properly install ecpg_compat and pgtypes libraries on MSVC (Jiang Guiqing)
•
Include our version of isinf() in libecpg if it’s not provided by the system (Jiang Guiqing)
•
Rearrange configure’s tests for supplied functions so it is not fooled by bogus exports from libedit/libreadline (Christoph Berg)
•
Ensure Windows build number increases over time (Magnus Hagander)
•
Make pgxs build executables with the right .exe suffix when cross-compiling for Windows (Zoltan Boszormenyi)
•
Add new timezone abbreviation FET (Tom Lane) This is now used in some eastern-European time zones.
2249
Appendix E. Release Notes
E.82. Release 9.0.11 Release date: 2012-12-06 This release contains a variety of fixes from 9.0.10. For information about new features in the 9.0 major release, see Section E.93.
E.82.1. Migration to Version 9.0.11 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.82.2. Changes •
Fix multiple bugs associated with CREATE INDEX CONCURRENTLY (Andres Freund, Tom Lane) Fix CREATE INDEX CONCURRENTLY to use in-place updates when changing the state of an index’s pg_index row. This prevents race conditions that could cause concurrent sessions to miss updating the target index, thus resulting in corrupt concurrently-created indexes. Also, fix various other operations to ensure that they ignore invalid indexes resulting from a failed CREATE INDEX CONCURRENTLY command. The most important of these is VACUUM, because an auto-vacuum could easily be launched on the table before corrective action can be taken to fix or remove the invalid index.
•
Fix buffer locking during WAL replay (Tom Lane) The WAL replay code was insufficiently careful about locking buffers when replaying WAL records that affect more than one page. This could result in hot standby queries transiently seeing inconsistent states, resulting in wrong answers or unexpected failures.
•
Fix an error in WAL generation logic for GIN indexes (Tom Lane) This could result in index corruption, if a torn-page failure occurred.
•
Properly remove startup process’s virtual XID lock when promoting a hot standby server to normal running (Simon Riggs) This oversight could prevent subsequent execution of certain operations such as CREATE INDEX CONCURRENTLY.
Prevent the postmaster from launching new child processes after it’s received a shutdown signal (Tom Lane) This mistake could result in shutdown taking longer than it should, or even never completing at all without additional user action.
•
Avoid corruption of internal hash tables when out of memory (Hitoshi Harada)
•
Fix planning of non-strict equivalence clauses above outer joins (Tom Lane) The planner could derive incorrect constraints from a clause equating a non-strict construct to something else, for example WHERE COALESCE(foo, 0) = 0 when foo is coming from the nullable side of an outer join.
•
Improve planner’s ability to prove exclusion constraints from equivalence classes (Tom Lane)
2250
Appendix E. Release Notes •
Fix partial-row matching in hashed subplans to handle cross-type cases correctly (Tom Lane) This affects multicolumn NOT IN subplans, such as WHERE (a, b) NOT IN (SELECT x, y FROM ...) when for instance b and y are int4 and int8 respectively. This mistake led to wrong answers or crashes depending on the specific datatypes involved.
•
Acquire buffer lock when re-fetching the old tuple for an AFTER ROW UPDATE/DELETE trigger (Andres Freund) In very unusual circumstances, this oversight could result in passing incorrect data to the precheck logic for a foreign-key enforcement trigger. That could result in a crash, or in an incorrect decision about whether to fire the trigger.
•
Fix ALTER COLUMN TYPE to handle inherited check constraints properly (Pavan Deolasee) This worked correctly in pre-8.4 releases, and now works correctly in 8.4 and later.
•
Fix REASSIGN OWNED to handle grants on tablespaces (Álvaro Herrera)
•
Ignore incorrect pg_attribute entries for system columns for views (Tom Lane) Views do not have any system columns. However, we forgot to remove such entries when converting a table to a view. That’s fixed properly for 9.3 and later, but in previous branches we need to defend against existing mis-converted views.
•
Fix rule printing to dump INSERT INTO table DEFAULT VALUES correctly (Tom Lane)
•
Guard against stack overflow when there are too many UNION/INTERSECT/EXCEPT clauses in a query (Tom Lane)
•
Prevent platform-dependent failures when dividing the minimum possible integer value by -1 (Xi Wang, Tom Lane)
•
Fix possible access past end of string in date parsing (Hitoshi Harada)
•
Fix failure to advance XID epoch if XID wraparound happens during a checkpoint and wal_level is hot_standby (Tom Lane, Andres Freund) While this mistake had no particular impact on PostgreSQL itself, it was bad for applications that rely on txid_current() and related functions: the TXID value would appear to go backwards.
•
Produce an understandable error message if the length of the path name for a Unix-domain socket exceeds the platform-specific limit (Tom Lane, Andrew Dunstan) Formerly, this would result in something quite unhelpful, such as “Non-recoverable failure in name resolution”.
•
Fix memory leaks when sending composite column values to the client (Tom Lane)
•
Make pg_ctl more robust about reading the postmaster.pid file (Heikki Linnakangas) Fix race conditions and possible file descriptor leakage.
•
Fix possible crash in psql if incorrectly-encoded data is presented and the client_encoding setting is a client-only encoding, such as SJIS (Jiang Guiqing)
•
Fix bugs in the restore.sql script emitted by pg_dump in tar output format (Tom Lane) The script would fail outright on tables whose names include upper-case characters. Also, make the script capable of restoring data in --inserts mode as well as the regular COPY mode.
•
Fix pg_restore to accept POSIX-conformant tar files (Brian Weaver, Tom Lane) The original coding of pg_dump’s tar output mode produced files that are not fully conformant with the POSIX standard. This has been corrected for version 9.3. This patch updates previous
2251
Appendix E. Release Notes branches so that they will accept both the incorrect and the corrected formats, in hopes of avoiding compatibility problems when 9.3 comes out. •
Fix pg_resetxlog to locate postmaster.pid correctly when given a relative path to the data directory (Tom Lane) This mistake could lead to pg_resetxlog not noticing that there is an active postmaster using the data directory.
•
Fix libpq’s lo_import() and lo_export() functions to report file I/O errors properly (Tom Lane)
•
Fix ecpg’s processing of nested structure pointer variables (Muhammad Usama)
•
Fix ecpg’s ecpg_get_data function to handle arrays properly (Michael Meskes)
•
Make contrib/pageinspect’s btree page inspection functions take buffer locks while examining pages (Tom Lane)
•
Fix pgxs support for building loadable modules on AIX (Tom Lane) Building modules outside the original source tree didn’t work on AIX.
•
Update time zone data files to tzdata release 2012j for DST law changes in Cuba, Israel, Jordan, Libya, Palestine, Western Samoa, and portions of Brazil.
E.83. Release 9.0.10 Release date: 2012-09-24 This release contains a variety of fixes from 9.0.9. For information about new features in the 9.0 major release, see Section E.93.
E.83.1. Migration to Version 9.0.10 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.83.2. Changes •
Fix planner’s assignment of executor parameters, and fix executor’s rescan logic for CTE plan nodes (Tom Lane) These errors could result in wrong answers from queries that scan the same WITH subquery multiple times.
•
Improve page-splitting decisions in GiST indexes (Alexander Korotkov, Robert Haas, Tom Lane) Multi-column GiST indexes might suffer unexpected bloat due to this error.
•
Fix cascading privilege revoke to stop if privileges are still held (Tom Lane) If we revoke a grant option from some role X , but X still holds that option via a grant from someone else, we should not recursively revoke the corresponding privilege from role(s) Y that X had granted it to.
2252
Appendix E. Release Notes •
Improve error messages for Hot Standby misconfiguration errors (Gurjeet Singh)
•
Fix handling of SIGFPE when PL/Perl is in use (Andres Freund) Perl resets the process’s SIGFPE handler to SIG_IGN, which could result in crashes later on. Restore the normal Postgres signal handler after initializing PL/Perl.
•
Prevent PL/Perl from crashing if a recursive PL/Perl function is redefined while being executed (Tom Lane)
•
Work around possible misoptimization in PL/Perl (Tom Lane) Some Linux distributions contain an incorrect version of pthread.h that results in incorrect compiled code in PL/Perl, leading to crashes if a PL/Perl function calls another one that throws an error.
•
Fix pg_upgrade’s handling of line endings on Windows (Andrew Dunstan) Previously, pg_upgrade might add or remove carriage returns in places such as function bodies.
•
On Windows, make pg_upgrade use backslash path separators in the scripts it emits (Andrew Dunstan)
•
Update time zone data files to tzdata release 2012f for DST law changes in Fiji
E.84. Release 9.0.9 Release date: 2012-08-17 This release contains a variety of fixes from 9.0.8. For information about new features in the 9.0 major release, see Section E.93.
E.84.1. Migration to Version 9.0.9 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.84.2. Changes •
Prevent access to external files/URLs via XML entity references (Noah Misch, Tom Lane) xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity
references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn’t get returned directly to the user, portions of it could be exposed in error messages if the data didn’t parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. (CVE-20123489) •
Prevent access to external files/URLs via contrib/xml2’s xslt_process() (Peter Eisentraut) libxslt offers the ability to read and write both files and URLs through stylesheet commands, thus allowing unprivileged database users to both read and write data with the privileges of the database server. Disable that through proper use of libxslt’s security options. (CVE-2012-3488)
2253
Appendix E. Release Notes Also, remove xslt_process()’s ability to fetch documents and stylesheets from external files/URLs. While this was a documented “feature”, it was long regarded as a bad idea. The fix for CVE-2012-3489 broke that capability, and rather than expend effort on trying to fix it, we’re just going to summarily remove it. •
Prevent too-early recycling of btree index pages (Noah Misch) When we allowed read-only transactions to skip assigning XIDs, we introduced the possibility that a deleted btree page could be recycled while a read-only transaction was still in flight to it. This would result in incorrect index search results. The probability of such an error occurring in the field seems very low because of the timing requirements, but nonetheless it should be fixed.
•
Fix crash-safety bug with newly-created-or-reset sequences (Tom Lane) If ALTER SEQUENCE was executed on a freshly created or reset sequence, and then precisely one nextval() call was made on it, and then the server crashed, WAL replay would restore the sequence to a state in which it appeared that no nextval() had been done, thus allowing the first sequence value to be returned again by the next nextval() call. In particular this could manifest for serial columns, since creation of a serial column’s sequence includes an ALTER SEQUENCE OWNED BY step.
•
Fix txid_current() to report the correct epoch when not in hot standby (Heikki Linnakangas) This fixes a regression introduced in the previous minor release.
•
Fix bug in startup of Hot Standby when a master transaction has many subtransactions (Andres Freund) This mistake led to failures reported as “out-of-order XID insertion in KnownAssignedXids”.
•
Ensure the backup_label file is fsync’d after pg_start_backup() (Dave Kerr)
•
Fix timeout handling in walsender processes (Tom Lane) WAL sender background processes neglected to establish a SIGALRM handler, meaning they would wait forever in some corner cases where a timeout ought to happen.
•
Back-patch 9.1 improvement to compress the fsync request queue (Robert Haas) This improves performance during checkpoints. The 9.1 change has now seen enough field testing to seem safe to back-patch.
•
Fix LISTEN/NOTIFY to cope better with I/O problems, such as out of disk space (Tom Lane) After a write failure, all subsequent attempts to send more NOTIFY messages would fail with messages like “Could not read from file "pg_notify/nnnn" at offset nnnnn: Success”.
•
Only allow autovacuum to be auto-canceled by a directly blocked process (Tom Lane) The original coding could allow inconsistent behavior in some cases; in particular, an autovacuum could get canceled after less than deadlock_timeout grace period.
•
Improve logging of autovacuum cancels (Robert Haas)
•
Fix log collector so that log_truncate_on_rotation works during the very first log rotation after server start (Tom Lane)
•
Fix WITH attached to a nested set operation (UNION/INTERSECT/EXCEPT) (Tom Lane)
•
Ensure that a whole-row reference to a subquery doesn’t include any extra GROUP BY or ORDER BY columns (Tom Lane)
•
Disallow copying whole-row references in CHECK constraints and index definitions during CREATE TABLE (Tom Lane)
2254
Appendix E. Release Notes This situation can arise in CREATE TABLE with LIKE or INHERITS. The copied whole-row variable was incorrectly labeled with the row type of the original table not the new one. Rejecting the case seems reasonable for LIKE, since the row types might well diverge later. For INHERITS we should ideally allow it, with an implicit coercion to the parent table’s row type; but that will require more work than seems safe to back-patch. •
Fix memory leak in ARRAY(SELECT ...) subqueries (Heikki Linnakangas, Tom Lane)
•
Fix extraction of common prefixes from regular expressions (Tom Lane) The code could get confused by quantified parenthesized subexpressions, such as ^(foo)?bar. This would lead to incorrect index optimization of searches for such patterns.
•
Fix bugs with parsing signed hh:mm and hh:mm:ss fields in interval constants (Amit Kapila, Tom Lane)
•
Use Postgres’ encoding conversion functions, not Python’s, when converting a Python Unicode string to the server encoding in PL/Python (Jan Urbanski) This avoids some corner-case problems, notably that Python doesn’t support all the encodings Postgres does. A notable functional change is that if the server encoding is SQL_ASCII, you will get the UTF-8 representation of the string; formerly, any non-ASCII characters in the string would result in an error.
•
Fix mapping of PostgreSQL encodings to Python encodings in PL/Python (Jan Urbanski)
•
Report errors properly in contrib/xml2’s xslt_process() (Tom Lane)
•
Update time zone data files to tzdata release 2012e for DST law changes in Morocco and Tokelau
E.85. Release 9.0.8 Release date: 2012-06-04 This release contains a variety of fixes from 9.0.7. For information about new features in the 9.0 major release, see Section E.93.
E.85.1. Migration to Version 9.0.8 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.85.2. Changes •
Fix incorrect password transformation in contrib/pgcrypto’s DES crypt() function (Solar Designer) If a password string contained the byte value 0x80, the remainder of the password was ignored, causing the password to be much weaker than it appeared. With this fix, the rest of the string is properly included in the DES hash. Any stored password values that are affected by this bug will thus no longer match, so the stored values may need to be updated. (CVE-2012-2143)
•
Ignore SECURITY DEFINER and SET attributes for a procedural language’s call handler (Tom Lane)
2255
Appendix E. Release Notes Applying such attributes to a call handler could crash the server. (CVE-2012-2655) •
Allow numeric timezone offsets in timestamp input to be up to 16 hours away from UTC (Tom Lane) Some historical time zones have offsets larger than 15 hours, the previous limit. This could result in dumped data values being rejected during reload.
•
Fix timestamp conversion to cope when the given time is exactly the last DST transition time for the current timezone (Tom Lane) This oversight has been there a long time, but was not noticed previously because most DST-using zones are presumed to have an indefinite sequence of future DST transitions.
•
Fix text to name and char to name casts to perform string truncation correctly in multibyte encodings (Karl Schnaitter)
•
Fix memory copying bug in to_tsquery() (Heikki Linnakangas)
•
Ensure txid_current() reports the correct epoch when executed in hot standby (Simon Riggs)
•
Fix planner’s handling of outer PlaceHolderVars within subqueries (Tom Lane) This bug concerns sub-SELECTs that reference variables coming from the nullable side of an outer join of the surrounding query. In 9.1, queries affected by this bug would fail with “ERROR: Upper-level PlaceHolderVar found where not expected”. But in 9.0 and 8.4, you’d silently get possibly-wrong answers, since the value transmitted into the subquery wouldn’t go to null when it should.
•
Fix slow session startup when pg_attribute is very large (Tom Lane) If pg_attribute exceeds one-fourth of shared_buffers, cache rebuilding code that is sometimes needed during session start would trigger the synchronized-scan logic, causing it to take many times longer than normal. The problem was particularly acute if many new sessions were starting at once.
•
Ensure sequential scans check for query cancel reasonably often (Merlin Moncure) A scan encountering many consecutive pages that contain no live tuples would not respond to interrupts meanwhile.
•
Ensure the Windows implementation of PGSemaphoreLock() clears ImmediateInterruptOK before returning (Tom Lane) This oversight meant that a query-cancel interrupt received later in the same query could be accepted at an unsafe time, with unpredictable but not good consequences.
•
Show whole-row variables safely when printing views or rules (Abbas Butt, Tom Lane) Corner cases involving ambiguous names (that is, the name could be either a table or column name of the query) were printed in an ambiguous way, risking that the view or rule would be interpreted differently after dump and reload. Avoid the ambiguous case by attaching a no-op cast.
•
Fix COPY FROM to properly handle null marker strings that correspond to invalid encoding (Tom Lane) A null marker string such as E’\\0’ should work, and did work in the past, but the case got broken in 8.4.
•
Ensure autovacuum worker processes perform stack depth checking properly (Heikki Linnakangas) Previously, infinite recursion in a function invoked by auto-ANALYZE could crash worker processes.
•
Fix logging collector to not lose log coherency under high load (Andrew Dunstan) The collector previously could fail to reassemble large messages if it got too busy.
2256
Appendix E. Release Notes •
Fix logging collector to ensure it will restart file rotation after receiving SIGHUP (Tom Lane)
•
Fix WAL replay logic for GIN indexes to not fail if the index was subsequently dropped (Tom Lane)
•
Fix memory leak in PL/pgSQL’s RETURN NEXT command (Joe Conway)
•
Fix PL/pgSQL’s GET DIAGNOSTICS command when the target is the function’s first variable (Tom Lane)
•
Fix potential access off the end of memory in psql’s expanded display (\x) mode (Peter Eisentraut)
•
Fix several performance problems in pg_dump when the database contains many objects (Jeff Janes, Tom Lane) pg_dump could get very slow if the database contained many schemas, or if many objects are in dependency loops, or if there are many owned sequences.
•
Fix pg_upgrade for the case that a database stored in a non-default tablespace contains a table in the cluster’s default tablespace (Bruce Momjian)
•
In ecpg, fix rare memory leaks and possible overwrite of one byte after the sqlca_t structure (Peter Eisentraut)
•
Fix contrib/dblink’s dblink_exec() to not leak temporary database connections upon error (Tom Lane)
•
Fix contrib/dblink to report the correct connection name in error messages (Kyotaro Horiguchi)
•
Fix contrib/vacuumlo to use multiple transactions when dropping many large objects (Tim Lewis, Robert Haas, Tom Lane) This change avoids exceeding max_locks_per_transaction when many objects need to be dropped. The behavior can be adjusted with the new -l (limit) option.
•
Update time zone data files to tzdata release 2012c for DST law changes in Antarctica, Armenia, Chile, Cuba, Falkland Islands, Gaza, Haiti, Hebron, Morocco, Syria, and Tokelau Islands; also historical corrections for Canada.
E.86. Release 9.0.7 Release date: 2012-02-27 This release contains a variety of fixes from 9.0.6. For information about new features in the 9.0 major release, see Section E.93.
E.86.1. Migration to Version 9.0.7 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.6, see Section E.87.
E.86.2. Changes •
Require execute permission on the trigger function for CREATE TRIGGER (Robert Haas)
2257
Appendix E. Release Notes This missing check could allow another user to execute a trigger function with forged input data, by installing it on a table he owns. This is only of significance for trigger functions marked SECURITY DEFINER, since otherwise trigger functions run as the table owner anyway. (CVE-2012-0866) •
Remove arbitrary limitation on length of common name in SSL certificates (Heikki Linnakangas) Both libpq and the server truncated the common name extracted from an SSL certificate at 32 bytes. Normally this would cause nothing worse than an unexpected verification failure, but there are some rather-implausible scenarios in which it might allow one certificate holder to impersonate another. The victim would have to have a common name exactly 32 bytes long, and the attacker would have to persuade a trusted CA to issue a certificate in which the common name has that string as a prefix. Impersonating a server would also require some additional exploit to redirect client connections. (CVE-2012-0867)
•
Convert newlines to spaces in names written in pg_dump comments (Robert Haas) pg_dump was incautious about sanitizing object names that are emitted within SQL comments in its output script. A name containing a newline would at least render the script syntactically incorrect. Maliciously crafted object names could present a SQL injection risk when the script is reloaded. (CVE-2012-0868)
•
Fix btree index corruption from insertions concurrent with vacuuming (Tom Lane) An index page split caused by an insertion could sometimes cause a concurrently-running VACUUM to miss removing index entries that it should remove. After the corresponding table rows are removed, the dangling index entries would cause errors (such as “could not read block N in file ...”) or worse, silently wrong query results after unrelated rows are re-inserted at the now-free table locations. This bug has been present since release 8.2, but occurs so infrequently that it was not diagnosed until now. If you have reason to suspect that it has happened in your database, reindexing the affected index will fix things.
•
Fix transient zeroing of shared buffers during WAL replay (Tom Lane) The replay logic would sometimes zero and refill a shared buffer, so that the contents were transiently invalid. In hot standby mode this can result in a query that’s executing in parallel seeing garbage data. Various symptoms could result from that, but the most common one seems to be “invalid memory alloc request size”.
•
Fix postmaster to attempt restart after a hot-standby crash (Tom Lane) A logic error caused the postmaster to terminate, rather than attempt to restart the cluster, if any backend process crashed while operating in hot standby mode.
•
Fix CLUSTER/VACUUM FULL handling of toast values owned by recently-updated rows (Tom Lane) This oversight could lead to “duplicate key value violates unique constraint” errors being reported against the toast table’s index during one of these commands.
•
Update per-column permissions, not only per-table permissions, when changing table owner (Tom Lane) Failure to do this meant that any previously granted column permissions were still shown as having been granted by the old owner. This meant that neither the new owner nor a superuser could revoke the now-untraceable-to-table-owner permissions.
•
Support foreign data wrappers and foreign servers in REASSIGN OWNED (Alvaro Herrera) This command failed with “unexpected classid” errors if it needed to change the ownership of any such objects.
•
Allow non-existent values for some settings in ALTER USER/DATABASE SET (Heikki Linnakangas)
2258
Appendix E. Release Notes Allow default_text_search_config, default_tablespace, and temp_tablespaces to be set to names that are not known. This is because they might be known in another database where the setting is intended to be used, or for the tablespace cases because the tablespace might not be created yet. The same issue was previously recognized for search_path, and these settings now act like that one. •
Avoid crashing when we have problems deleting table files post-commit (Tom Lane) Dropping a table should lead to deleting the underlying disk files only after the transaction commits. In event of failure then (for instance, because of wrong file permissions) the code is supposed to just emit a warning message and go on, since it’s too late to abort the transaction. This logic got broken as of release 8.4, causing such situations to result in a PANIC and an unrestartable database.
•
Recover from errors occurring during WAL replay of DROP TABLESPACE (Tom Lane) Replay will attempt to remove the tablespace’s directories, but there are various reasons why this might fail (for example, incorrect ownership or permissions on those directories). Formerly the replay code would panic, rendering the database unrestartable without manual intervention. It seems better to log the problem and continue, since the only consequence of failure to remove the directories is some wasted disk space.
•
Fix race condition in logging AccessExclusiveLocks for hot standby (Simon Riggs) Sometimes a lock would be logged as being held by “transaction zero”. This is at least known to produce assertion failures on slave servers, and might be the cause of more serious problems.
•
Track the OID counter correctly during WAL replay, even when it wraps around (Tom Lane) Previously the OID counter would remain stuck at a high value until the system exited replay mode. The practical consequences of that are usually nil, but there are scenarios wherein a standby server that’s been promoted to master might take a long time to advance the OID counter to a reasonable value once values are needed.
•
Prevent emitting misleading “consistent recovery state reached” log message at the beginning of crash recovery (Heikki Linnakangas)
•
Fix initial value of pg_stat_replication.replay_location (Fujii Masao) Previously, the value shown would be wrong until at least one WAL record had been replayed.
•
Fix regular expression back-references with * attached (Tom Lane) Rather than enforcing an exact string match, the code would effectively accept any string that satisfies the pattern sub-expression referenced by the back-reference symbol. A similar problem still afflicts back-references that are embedded in a larger quantified expression, rather than being the immediate subject of the quantifier. This will be addressed in a future PostgreSQL release.
•
Fix recently-introduced memory leak in processing of inet/cidr values (Heikki Linnakangas) A patch in the December 2011 releases of PostgreSQL caused memory leakage in these operations, which could be significant in scenarios such as building a btree index on such a column.
•
Fix dangling pointer after CREATE TABLE AS/SELECT INTO in a SQL-language function (Tom Lane) In most cases this only led to an assertion failure in assert-enabled builds, but worse consequences seem possible.
•
Avoid double close of file handle in syslogger on Windows (MauMau) Ordinarily this error was invisible, but it would cause an exception when running on a debug version of Windows.
2259
Appendix E. Release Notes •
Fix I/O-conversion-related memory leaks in plpgsql (Andres Freund, Jan Urbanski, Tom Lane) Certain operations would leak memory until the end of the current function.
•
Improve pg_dump’s handling of inherited table columns (Tom Lane) pg_dump mishandled situations where a child column has a different default expression than its parent column. If the default is textually identical to the parent’s default, but not actually the same (for instance, because of schema search path differences) it would not be recognized as different, so that after dump and restore the child would be allowed to inherit the parent’s default. Child columns that are NOT NULL where their parent is not could also be restored subtly incorrectly.
•
Fix pg_restore’s direct-to-database mode for INSERT-style table data (Tom Lane) Direct-to-database restores from archive files made with --inserts or --column-inserts options fail when using pg_restore from a release dated September or December 2011, as a result of an oversight in a fix for another problem. The archive file itself is not at fault, and text-mode output is okay.
•
Allow pg_upgrade to process tables containing regclass columns (Bruce Momjian) Since pg_upgrade now takes care to preserve pg_class OIDs, there was no longer any reason for this restriction.
•
Make libpq ignore ENOTDIR errors when looking for an SSL client certificate file (Magnus Hagander) This allows SSL connections to be established, though without a certificate, even when the user’s home directory is set to something like /dev/null.
•
Fix some more field alignment issues in ecpg’s SQLDA area (Zoltan Boszormenyi)
•
Allow AT option in ecpg DEALLOCATE statements (Michael Meskes) The infrastructure to support this has been there for awhile, but through an oversight there was still an error check rejecting the case.
•
Do not use the variable name when defining a varchar structure in ecpg (Michael Meskes)
•
Fix contrib/auto_explain’s JSON output mode to produce valid JSON (Andrew Dunstan) The output used brackets at the top level, when it should have used braces.
•
Fix error in contrib/intarray’s int[] & int[] operator (Guillaume Lelarge) If the smallest integer the two input arrays have in common is 1, and there are smaller values in either array, then 1 would be incorrectly omitted from the result.
•
Fix error detection in contrib/pgcrypto’s encrypt_iv() and decrypt_iv() (Marko Kreen) These functions failed to report certain types of invalid-input errors, and would instead return random garbage values for incorrect input.
•
Fix one-byte buffer overrun in contrib/test_parser (Paul Guyot) The code would try to read one more byte than it should, which would crash in corner cases. Since contrib/test_parser is only example code, this is not a security issue in itself, but bad example code is still bad.
•
Use __sync_lock_test_and_set() for spinlocks on ARM, if available (Martin Pitt) This function replaces our previous use of the SWPB instruction, which is deprecated and not available on ARMv6 and later. Reports suggest that the old code doesn’t fail in an obvious way on recent ARM boards, but simply doesn’t interlock concurrent accesses, leading to bizarre failures in multiprocess operation.
2260
Appendix E. Release Notes •
Use -fexcess-precision=standard option when building with gcc versions that accept it (Andrew Dunstan) This prevents assorted scenarios wherein recent versions of gcc will produce creative results.
•
Allow use of threaded Python on FreeBSD (Chris Rees) Our configure script previously believed that this combination wouldn’t work; but FreeBSD fixed the problem, so remove that error check.
E.87. Release 9.0.6 Release date: 2011-12-05 This release contains a variety of fixes from 9.0.5. For information about new features in the 9.0 major release, see Section E.93.
E.87.1. Migration to Version 9.0.6 A dump/restore is not required for those running 9.0.X. However,
a
longstanding
error
was
discovered
in
the
definition
of
the
information_schema.referential_constraints view. If you rely on correct results from
that view, you should replace its definition as explained in the first changelog item below. Also, if you are upgrading from a version earlier than 9.0.4, see Section E.89.
E.87.2. Changes •
Fix bugs in information_schema.referential_constraints view (Tom Lane) This view was being insufficiently careful about matching the foreign-key constraint to the depended-on primary or unique key constraint. That could result in failure to show a foreign key constraint at all, or showing it multiple times, or claiming that it depends on a different constraint than the one it really does. Since the view definition is installed by initdb, merely upgrading will not fix the problem. If you need to fix this in an existing installation, you can (as a superuser) drop the information_schema schema then re-create it by sourcing SHAREDIR/information_schema.sql. (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) This must be repeated in each database to be fixed.
•
Fix possible crash during UPDATE or DELETE that joins to the output of a scalar-returning function (Tom Lane) A crash could only occur if the target row had been concurrently updated, so this problem surfaced only intermittently.
•
Fix incorrect replay of WAL records for GIN index updates (Tom Lane) This could result in transiently failing to find index entries after a crash, or on a hot-standby server. The problem would be repaired by the next VACUUM of the index, however.
•
Fix TOAST-related data corruption during CREATE TABLE dest AS SELECT * FROM src or INSERT INTO dest SELECT * FROM src (Tom Lane)
2261
Appendix E. Release Notes If a table has been modified by ALTER TABLE ADD COLUMN, attempts to copy its data verbatim to another table could produce corrupt results in certain corner cases. The problem can only manifest in this precise form in 8.4 and later, but we patched earlier versions as well in case there are other code paths that could trigger the same bug. •
Fix possible failures during hot standby startup (Simon Riggs)
•
Start hot standby faster when initial snapshot is incomplete (Simon Riggs)
•
Fix race condition during toast table access from stale syscache entries (Tom Lane) The typical symptom was transient errors like “missing chunk number 0 for toast value NNNNN in pg_toast_2619”, where the cited toast table would always belong to a system catalog.
•
Track dependencies of functions on items used in parameter default expressions (Tom Lane) Previously, a referenced object could be dropped without having dropped or modified the function, leading to misbehavior when the function was used. Note that merely installing this update will not fix the missing dependency entries; to do that, you’d need to CREATE OR REPLACE each such function afterwards. If you have functions whose defaults depend on non-built-in objects, doing so is recommended.
•
Allow inlining of set-returning SQL functions with multiple OUT parameters (Tom Lane)
•
Don’t trust deferred-unique indexes for join removal (Tom Lane and Marti Raudsepp) A deferred uniqueness constraint might not hold intra-transaction, so assuming that it does could give incorrect query results.
•
Make DatumGetInetP() unpack inet datums that have a 1-byte header, and add a new macro, DatumGetInetPP(), that does not (Heikki Linnakangas) This change affects no core code, but might prevent crashes in add-on code that expects DatumGetInetP() to produce an unpacked datum as per usual convention.
•
Improve locale support in money type’s input and output (Tom Lane) Aside from not supporting all standard lc_monetary formatting options, the input and output functions were inconsistent, meaning there were locales in which dumped money values could not be re-read.
•
Don’t let transform_null_equals affect CASE foo WHEN NULL ... constructs (Heikki Linnakangas) transform_null_equals is only supposed to affect foo = NULL expressions written directly by the user, not equality checks generated internally by this form of CASE.
•
Change foreign-key trigger creation order to better support self-referential foreign keys (Tom Lane) For a cascading foreign key that references its own table, a row update will fire both the ON UPDATE trigger and the CHECK trigger as one event. The ON UPDATE trigger must execute first, else the CHECK will check a non-final state of the row and possibly throw an inappropriate error. However, the firing order of these triggers is determined by their names, which generally sort in creation order since the triggers have auto-generated names following the convention “RI_ConstraintTrigger_NNNN”. A proper fix would require modifying that convention, which we will do in 9.2, but it seems risky to change it in existing releases. So this patch just changes the creation order of the triggers. Users encountering this type of error should drop and re-create the foreign key constraint to get its triggers into the right order.
•
Avoid floating-point underflow while tracking buffer allocation rate (Greg Matthews) While harmless in itself, on certain platforms this would result in annoying kernel log messages.
2262
Appendix E. Release Notes •
Preserve configuration file name and line number values when starting child processes under Windows (Tom Lane) Formerly, these would not be displayed correctly in the pg_settings view.
•
Fix incorrect field alignment in ecpg’s SQLDA area (Zoltan Boszormenyi)
•
Preserve blank lines within commands in psql’s command history (Robert Haas) The former behavior could cause problems if an empty line was removed from within a string literal, for example.
•
Fix pg_dump to dump user-defined casts between auto-generated types, such as table rowtypes (Tom Lane)
•
Assorted fixes for pg_upgrade (Bruce Momjian) Handle exclusion constraints correctly, avoid failures on Windows, don’t complain about mismatched toast table names in 8.4 databases.
•
Use the preferred version of xsubpp to build PL/Perl, not necessarily the operating system’s main copy (David Wheeler and Alex Hunsaker)
•
Fix incorrect coding in contrib/dict_int and contrib/dict_xsyn (Tom Lane) Some functions incorrectly assumed that memory returned by palloc() is guaranteed zeroed.
•
Fix assorted errors in contrib/unaccent’s configuration file parsing (Tom Lane)
•
Honor query cancel interrupts promptly in pgstatindex() (Robert Haas)
•
Fix incorrect quoting of log file name in Mac OS X start script (Sidar Lopez)
•
Ensure VPATH builds properly install all server header files (Peter Eisentraut)
•
Shorten file names reported in verbose error messages (Peter Eisentraut) Regular builds have always reported just the name of the C file containing the error message call, but VPATH builds formerly reported an absolute path name.
•
Fix interpretation of Windows timezone names for Central America (Tom Lane) Map “Central America Standard Time” to CST6, not CST6CDT, because DST is generally not observed anywhere in Central America.
•
Update time zone data files to tzdata release 2011n for DST law changes in Brazil, Cuba, Fiji, Palestine, Russia, and Samoa; also historical corrections for Alaska and British East Africa.
E.88. Release 9.0.5 Release date: 2011-09-26 This release contains a variety of fixes from 9.0.4. For information about new features in the 9.0 major release, see Section E.93.
E.88.1. Migration to Version 9.0.5 A dump/restore is not required for those running 9.0.X. However, if you are upgrading from a version earlier than 9.0.4, see Section E.89.
2263
Appendix E. Release Notes
E.88.2. Changes •
Fix catalog cache invalidation after a VACUUM FULL or CLUSTER on a system catalog (Tom Lane) In some cases the relocation of a system catalog row to another place would not be recognized by concurrent server processes, allowing catalog corruption to occur if they then tried to update that row. The worst-case outcome could be as bad as complete loss of a table.
•
Fix incorrect order of operations during sinval reset processing, and ensure that TOAST OIDs are preserved in system catalogs (Tom Lane) These mistakes could lead to transient failures after a VACUUM FULL or CLUSTER on a system catalog.
•
Fix bugs in indexing of in-doubt HOT-updated tuples (Tom Lane) These bugs could result in index corruption after reindexing a system catalog. They are not believed to affect user indexes.
•
Fix multiple bugs in GiST index page split processing (Heikki Linnakangas) The probability of occurrence was low, but these could lead to index corruption.
•
Fix possible buffer overrun in tsvector_concat() (Tom Lane) The function could underestimate the amount of memory needed for its result, leading to server crashes.
•
Fix crash in xml_recv when processing a “standalone” parameter (Tom Lane)
•
Make pg_options_to_table return NULL for an option with no value (Tom Lane) Previously such cases would result in a server crash.
•
Avoid possibly accessing off the end of memory in ANALYZE and in SJIS-2004 encoding conversion (Noah Misch) This fixes some very-low-probability server crash scenarios.
•
Protect pg_stat_reset_shared() against NULL input (Magnus Hagander)
•
Fix possible failure when a recovery conflict deadlock is detected within a sub-transaction (Tom Lane)
•
Avoid spurious conflicts while recycling btree index pages during hot standby (Noah Misch, Simon Riggs)
•
Shut down WAL receiver if it’s still running at end of recovery (Heikki Linnakangas) The postmaster formerly panicked in this situation, but it’s actually a legitimate case.
•
Fix race condition in relcache init file invalidation (Tom Lane) There was a window wherein a new backend process could read a stale init file but miss the inval messages that would tell it the data is stale. The result would be bizarre failures in catalog accesses, typically “could not read block 0 in file ...” later during startup.
•
Fix memory leak at end of a GiST index scan (Tom Lane) Commands that perform many separate GiST index scans, such as verification of a new GiSTbased exclusion constraint on a table already containing many rows, could transiently require large amounts of memory due to this leak.
•
Fix memory leak when encoding conversion has to be done on incoming command strings and LISTEN is active (Tom Lane)
2264
Appendix E. Release Notes •
Fix incorrect memory accounting (leading to possible memory bloat) in tuplestores supporting holdable cursors and plpgsql’s RETURN NEXT command (Tom Lane)
•
Fix trigger WHEN conditions when both BEFORE and AFTER triggers exist (Tom Lane) Evaluation of WHEN conditions for AFTER ROW UPDATE triggers could crash if there had been a BEFORE ROW trigger fired for the same update.
•
Fix performance problem when constructing a large, lossy bitmap (Tom Lane)
•
Fix join selectivity estimation for unique columns (Tom Lane) This fixes an erroneous planner heuristic that could lead to poor estimates of the result size of a join.
•
Fix nested PlaceHolderVar expressions that appear only in sub-select target lists (Tom Lane) This mistake could result in outputs of an outer join incorrectly appearing as NULL.
•
Allow the planner to assume that empty parent tables really are empty (Tom Lane) Normally an empty table is assumed to have a certain minimum size for planning purposes; but this heuristic seems to do more harm than good for the parent table of an inheritance hierarchy, which often is permanently empty.
•
Allow nested EXISTS queries to be optimized properly (Tom Lane)
•
Fix array- and path-creating functions to ensure padding bytes are zeroes (Tom Lane) This avoids some situations where the planner will think that semantically-equal constants are not equal, resulting in poor optimization.
•
Fix EXPLAIN to handle gating Result nodes within inner-indexscan subplans (Tom Lane) The usual symptom of this oversight was “bogus varno” errors.
•
Fix btree preprocessing of indexedcol IS NULL conditions (Dean Rasheed) Such a condition is unsatisfiable if combined with any other type of btree-indexable condition on the same index column. The case was handled incorrectly in 9.0.0 and later, leading to query output where there should be none.
•
Work around gcc 4.6.0 bug that breaks WAL replay (Tom Lane) This could lead to loss of committed transactions after a server crash.
•
Fix dump bug for VALUES in a view (Tom Lane)
•
Disallow SELECT FOR UPDATE/SHARE on sequences (Tom Lane) This operation doesn’t work as expected and can lead to failures.
•
Fix VACUUM so that it always updates pg_class.reltuples/relpages (Tom Lane) This fixes some scenarios where autovacuum could make increasingly poor decisions about when to vacuum tables.
•
Defend against integer overflow when computing size of a hash table (Tom Lane)
•
Fix cases where CLUSTER might attempt to access already-removed TOAST data (Tom Lane)
•
Fix premature timeout failures during initial authentication transaction (Tom Lane)
•
Fix portability bugs in use of credentials control messages for “peer” authentication (Tom Lane)
•
Fix SSPI login when multiple roundtrips are required (Ahmed Shinwari, Magnus Hagander) The typical symptom of this problem was “The function requested is not supported” errors during SSPI login.
2265
Appendix E. Release Notes •
Fix failure when adding a new variable of a custom variable class to postgresql.conf (Tom Lane)
•
Throw an error if pg_hba.conf contains hostssl but SSL is disabled (Tom Lane) This was concluded to be more user-friendly than the previous behavior of silently ignoring such lines.
•
Fix failure when DROP OWNED BY attempts to remove default privileges on sequences (Shigeru Hanada)
•
Fix typo in pg_srand48 seed initialization (Andres Freund) This led to failure to use all bits of the provided seed. This function is not used on most platforms (only those without srandom), and the potential security exposure from a less-random-thanexpected seed seems minimal in any case.
•
Avoid integer overflow when the sum of LIMIT and OFFSET values exceeds 2^63 (Heikki Linnakangas)
•
Add overflow checks to int4 and int8 versions of generate_series() (Robert Haas)
•
Fix trailing-zero removal in to_char() (Marti Raudsepp) In a format with FM and no digit positions after the decimal point, zeroes to the left of the decimal point could be removed incorrectly.
•
Fix pg_size_pretty() to avoid overflow for inputs close to 2^63 (Tom Lane)
•
Weaken plpgsql’s check for typmod matching in record values (Tom Lane) An overly enthusiastic check could lead to discarding length modifiers that should have been kept.
•
Correctly handle quotes in locale names during initdb (Heikki Linnakangas) The case can arise with some Windows locales, such as “People’s Republic of China”.
•
In pg_upgrade, avoid dumping orphaned temporary tables (Bruce Momjian) This prevents situations wherein table OID assignments could get out of sync between old and new installations.
•
Fix pg_upgrade to preserve toast tables’ relfrozenxids during an upgrade from 8.3 (Bruce Momjian) Failure to do this could lead to pg_clog files being removed too soon after the upgrade.
•
In pg_upgrade, fix the -l (log) option to work on Windows (Bruce Momjian)
•
In pg_ctl, support silent mode for service registrations on Windows (MauMau)
•
Fix psql’s counting of script file line numbers during COPY from a different file (Tom Lane)
•
Fix pg_restore’s direct-to-database mode for standard_conforming_strings (Tom Lane) pg_restore could emit incorrect commands when restoring directly to a database server from an archive file that had been made with standard_conforming_strings set to on.
•
Be more user-friendly about unsupported cases for parallel pg_restore (Tom Lane) This change ensures that such cases are detected and reported before any restore actions have been taken.
•
Fix write-past-buffer-end and memory leak in libpq’s LDAP service lookup code (Albe Laurenz)
•
In libpq, avoid failures when using nonblocking I/O and an SSL connection (Martin Pihlak, Tom Lane)
•
Improve libpq’s handling of failures during connection startup (Tom Lane)
2266
Appendix E. Release Notes In particular, the response to a server report of fork() failure during SSL connection startup is now saner. •
Improve libpq’s error reporting for SSL failures (Tom Lane)
•
Fix PQsetvalue() to avoid possible crash when adding a new tuple to a PGresult originally obtained from a server query (Andrew Chernow)
•
Make ecpglib write double values with 15 digits precision (Akira Kurosawa)
•
In ecpglib, be sure LC_NUMERIC setting is restored after an error (Michael Meskes)
•
Apply upstream fix for blowfish signed-character bug (CVE-2011-2483) (Tom Lane) contrib/pg_crypto’s blowfish encryption code could give wrong results on platforms where
char is signed (which is most), leading to encrypted passwords being weaker than they should be. •
Fix memory leak in contrib/seg (Heikki Linnakangas)
•
Fix pgstatindex() to give consistent results for empty indexes (Tom Lane)
•
Allow building with perl 5.14 (Alex Hunsaker)
•
Fix assorted issues with build and install file paths containing spaces (Tom Lane)
•
Update time zone data files to tzdata release 2011i for DST law changes in Canada, Egypt, Russia, Samoa, and South Sudan.
E.89. Release 9.0.4 Release date: 2011-04-18 This release contains a variety of fixes from 9.0.3. For information about new features in the 9.0 major release, see Section E.93.
E.89.1. Migration to Version 9.0.4 A dump/restore is not required for those running 9.0.X. However, if your installation was upgraded from a previous major release by running pg_upgrade, you should take action to prevent possible data loss due to a now-fixed bug in pg_upgrade. The recommended solution is to run VACUUM FREEZE on all TOAST tables. More information is available at http://wiki.postgresql.org/wiki/20110408pg_upgrade_fix3.
E.89.2. Changes •
Fix pg_upgrade’s handling of TOAST tables (Bruce Momjian) The pg_class.relfrozenxid value for TOAST tables was not correctly copied into the new installation during pg_upgrade. This could later result in pg_clog files being discarded while they were still needed to validate tuples in the TOAST tables, leading to “could not access status of transaction” failures.
Appendix E. Release Notes This error poses a significant risk of data loss for installations that have been upgraded with pg_upgrade. This patch corrects the problem for future uses of pg_upgrade, but does not in itself cure the issue in installations that have been processed with a buggy version of pg_upgrade. •
Suppress incorrect “PD_ALL_VISIBLE flag was incorrectly set” warning (Heikki Linnakangas) VACUUM would sometimes issue this warning in cases that are actually valid.
•
Use better SQLSTATE error codes for hot standby conflict cases (Tatsuo Ishii and Simon Riggs) All retryable conflict errors now have an error code that indicates that a retry is possible. Also, session closure due to the database being dropped on the master is now reported as ERRCODE_DATABASE_DROPPED, rather than ERRCODE_ADMIN_SHUTDOWN, so that connection poolers can handle the situation correctly.
•
Prevent intermittent hang in interactions of startup process with bgwriter process (Simon Riggs) This affected recovery in non-hot-standby cases.
•
Disallow including a composite type in itself (Tom Lane) This prevents scenarios wherein the server could recurse infinitely while processing the composite type. While there are some possible uses for such a structure, they don’t seem compelling enough to justify the effort required to make sure it always works safely.
•
Avoid potential deadlock during catalog cache initialization (Nikhil Sontakke) In some cases the cache loading code would acquire share lock on a system index before locking the index’s catalog. This could deadlock against processes trying to acquire exclusive locks in the other, more standard order.
•
Fix dangling-pointer problem in BEFORE ROW UPDATE trigger handling when there was a concurrent update to the target tuple (Tom Lane) This bug has been observed to result in intermittent “cannot extract system attribute from virtual tuple” failures while trying to do UPDATE RETURNING ctid. There is a very small probability of more serious errors, such as generating incorrect index entries for the updated tuple.
•
Disallow DROP TABLE when there are pending deferred trigger events for the table (Tom Lane) Formerly the DROP would go through, leading to “could not open relation with OID nnn” errors when the triggers were eventually fired.
•
Allow “replication” as a user name in pg_hba.conf (Andrew Dunstan) “replication” is special in the database name column, but it was mistakenly also treated as special in the user name column.
•
Prevent crash triggered by constant-false WHERE conditions during GEQO optimization (Tom Lane)
•
Improve planner’s handling of semi-join and anti-join cases (Tom Lane)
•
Fix handling of SELECT FOR UPDATE in a sub-SELECT (Tom Lane) This bug typically led to “cannot extract system attribute from virtual tuple” errors.
•
Fix selectivity estimation for text search to account for NULLs (Jesper Krogh)
•
Fix get_actual_variable_range() to support hypothetical indexes injected by an index adviser plugin (Gurjeet Singh)
Allow libpq’s SSL initialization to succeed when user’s home directory is unavailable (Tom Lane)
2268
Appendix E. Release Notes If the SSL mode is such that a root certificate file is not required, there is no need to fail. This change restores the behavior to what it was in pre-9.0 releases. •
Fix libpq to return a useful error message for errors detected in conninfo_array_parse (Joseph Adams) A typo caused the library to return NULL, rather than the PGconn structure containing the error message, to the application.
•
Fix ecpg preprocessor’s handling of float constants (Heikki Linnakangas)
•
Fix parallel pg_restore to handle comments on POST_DATA items correctly (Arnd Hannemann)
•
Fix pg_restore to cope with long lines (over 1KB) in TOC files (Tom Lane)
•
Put in more safeguards against crashing due to division-by-zero with overly enthusiastic compiler optimization (Aurelien Jarno)
•
Support use of dlopen() in FreeBSD and OpenBSD on MIPS (Tom Lane) There was a hard-wired assumption that this system function was not available on MIPS hardware on these systems. Use a compile-time test instead, since more recent versions have it.
•
Fix compilation failures on HP-UX (Heikki Linnakangas)
•
Avoid crash when trying to write to the Windows console very early in process startup (Rushabh Lathia)
•
Support building with MinGW 64 bit compiler for Windows (Andrew Dunstan)
•
Fix version-incompatibility problem with libintl on Windows (Hiroshi Inoue)
•
Fix usage of xcopy in Windows build scripts to work correctly under Windows 7 (Andrew Dunstan) This affects the build scripts only, not installation or usage.
•
Fix path separator used by pg_regress on Cygwin (Andrew Dunstan)
•
Update time zone data files to tzdata release 2011f for DST law changes in Chile, Cuba, Falkland Islands, Morocco, Samoa, and Turkey; also historical corrections for South Australia, Alaska, and Hawaii.
E.90. Release 9.0.3 Release date: 2011-01-31 This release contains a variety of fixes from 9.0.2. For information about new features in the 9.0 major release, see Section E.93.
E.90.1. Migration to Version 9.0.3 A dump/restore is not required for those running 9.0.X.
E.90.2. Changes •
Before exiting walreceiver, ensure all the received WAL is fsync’d to disk (Heikki Linnakangas)
2269
Appendix E. Release Notes Otherwise the standby server could replay some un-synced WAL, conceivably leading to data corruption if the system crashes just at that point. •
Avoid excess fsync activity in walreceiver (Heikki Linnakangas)
•
Make ALTER TABLE revalidate uniqueness and exclusion constraints when needed (Noah Misch) This was broken in 9.0 by a change that was intended to suppress revalidation during VACUUM FULL and CLUSTER, but unintentionally affected ALTER TABLE as well.
•
Fix EvalPlanQual for UPDATE of an inheritance tree in which the tables are not all alike (Tom Lane) Any variation in the table row types (including dropped columns present in only some child tables) would confuse the EvalPlanQual code, leading to misbehavior or even crashes. Since EvalPlanQual is only executed during concurrent updates to the same row, the problem was only seen intermittently.
•
Avoid failures when EXPLAIN tries to display a simple-form CASE expression (Tom Lane) If the CASE’s test expression was a constant, the planner could simplify the CASE into a form that confused the expression-display code, resulting in “unexpected CASE WHEN clause” errors.
•
Fix assignment to an array slice that is before the existing range of subscripts (Tom Lane) If there was a gap between the newly added subscripts and the first pre-existing subscript, the code miscalculated how many entries needed to be copied from the old array’s null bitmap, potentially leading to data corruption or crash.
•
Avoid unexpected conversion overflow in planner for very distant date values (Tom Lane) The date type supports a wider range of dates than can be represented by the timestamp types, but the planner assumed it could always convert a date to timestamp with impunity.
•
Fix PL/Python crash when an array contains null entries (Alex Hunsaker)
•
Remove ecpg’s fixed length limit for constants defining an array dimension (Michael Meskes)
•
Fix erroneous parsing of tsquery values containing ... & !(subexpression) | ... (Tom Lane) Queries containing this combination of operators were not executed correctly. The same error existed in contrib/intarray’s query_int type and contrib/ltree’s ltxtquery type.
•
Fix buffer overrun in contrib/intarray’s input function for the query_int type (Apple) This bug is a security risk since the function’s return address could be overwritten. Thanks to Apple Inc’s security team for reporting this issue and supplying the fix. (CVE-2010-4015)
•
Fix bug in contrib/seg’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a seg column. If you have such an index, consider REINDEXing it after installing this update. (This is identical to the bug that was fixed in contrib/cube in the previous update.)
E.91. Release 9.0.2 Release date: 2010-12-16 This release contains a variety of fixes from 9.0.1. For information about new features in the 9.0 major release, see Section E.93.
2270
Appendix E. Release Notes
E.91.1. Migration to Version 9.0.2 A dump/restore is not required for those running 9.0.X.
E.91.2. Changes •
Force the default wal_sync_method to be fdatasync on Linux (Tom Lane, Marti Raudsepp) The default on Linux has actually been fdatasync for many years, but recent kernel changes caused PostgreSQL to choose open_datasync instead. This choice did not result in any performance improvement, and caused outright failures on certain filesystems, notably ext4 with the data=journal mount option.
•
Fix “too many KnownAssignedXids” error during Hot Standby replay (Heikki Linnakangas)
•
Fix race condition in lock acquisition during Hot Standby (Simon Riggs)
•
Avoid unnecessary conflicts during Hot Standby (Simon Riggs) This fixes some cases where replay was considered to conflict with standby queries (causing delay of replay or possibly cancellation of the queries), but there was no real conflict.
•
Fix assorted bugs in WAL replay logic for GIN indexes (Tom Lane) This could result in “bad buffer id: 0” failures or corruption of index contents during replication.
•
Fix recovery from base backup when the starting checkpoint WAL record is not in the same WAL segment as its redo point (Jeff Davis)
•
Fix corner-case bug when streaming replication is enabled immediately after creating the master database cluster (Heikki Linnakangas)
•
Fix persistent slowdown of autovacuum workers when multiple workers remain active for a long time (Tom Lane) The effective vacuum_cost_limit for an autovacuum worker could drop to nearly zero if it processed enough tables, causing it to run extremely slowly.
•
Fix long-term memory leak in autovacuum launcher (Alvaro Herrera)
•
Avoid failure when trying to report an impending transaction wraparound condition from outside a transaction (Tom Lane) This oversight prevented recovery after transaction wraparound got too close, because database startup processing would fail.
•
Add support for detecting register-stack overrun on IA64 (Tom Lane) The IA64 architecture has two hardware stacks. Full prevention of stack-overrun failures requires checking both.
•
Add a check for stack overflow in copyObject() (Tom Lane) Certain code paths could crash due to stack overflow given a sufficiently complex query.
•
Fix detection of page splits in temporary GiST indexes (Heikki Linnakangas) It is possible to have a “concurrent” page split in a temporary index, if for example there is an open cursor scanning the index when an insertion is done. GiST failed to detect this case and hence could deliver wrong results when execution of the cursor continued.
•
Fix error checking during early connection processing (Tom Lane)
2271
Appendix E. Release Notes The check for too many child processes was skipped in some cases, possibly leading to postmaster crash when attempting to add the new child process to fixed-size arrays. •
Improve efficiency of window functions (Tom Lane) Certain cases where a large number of tuples needed to be read in advance, but work_mem was large enough to allow them all to be held in memory, were unexpectedly slow. percent_rank(), cume_dist() and ntile() in particular were subject to this problem.
•
Avoid memory leakage while ANALYZE’ing complex index expressions (Tom Lane)
•
Ensure an index that uses a whole-row Var still depends on its table (Tom Lane) An index declared like create index i on t (foo(t.*)) would not automatically get dropped when its table was dropped.
•
Add missing support in DROP OWNED BY for removing foreign data wrapper/server privileges belonging to a user (Heikki Linnakangas)
•
Do not “inline” a SQL function with multiple OUT parameters (Tom Lane) This avoids a possible crash due to loss of information about the expected result rowtype.
•
Fix crash when inline-ing a set-returning function whose argument list contains a reference to an inline-able user function (Tom Lane)
•
Behave correctly if ORDER BY, LIMIT, FOR UPDATE, or WITH is attached to the VALUES part of INSERT ... VALUES (Tom Lane)
•
Make the OFF keyword unreserved (Heikki Linnakangas) This prevents problems with using off as a variable name in PL/pgSQL. That worked before 9.0, but was now broken because PL/pgSQL now treats all core reserved words as reserved.
•
Fix constant-folding of COALESCE() expressions (Tom Lane) The planner would sometimes attempt to evaluate sub-expressions that in fact could never be reached, possibly leading to unexpected errors.
•
Fix “could not find pathkey item to sort” planner failure with comparison of whole-row Vars (Tom Lane)
•
Fix postmaster crash when connection acceptance (accept() or one of the calls made immediately after it) fails, and the postmaster was compiled with GSSAPI support (Alexander Chernikov)
•
Retry after receiving an invalid response packet from a RADIUS authentication server (Magnus Hagander) This fixes a low-risk potential denial of service condition.
•
Fix missed unlink of temporary files when log_temp_files is active (Tom Lane) If an error occurred while attempting to emit the log message, the unlink was not done, resulting in accumulation of temp files.
•
Add print functionality for InhRelation nodes (Tom Lane) This avoids a failure when debug_print_parse is enabled and certain types of query are executed.
•
Fix incorrect calculation of distance from a point to a horizontal line segment (Tom Lane) This bug affected several different geometric distance-measurement operators.
•
Fix incorrect calculation of transaction status in ecpg (Itagaki Takahiro)
•
Fix errors in psql’s Unicode-escape support (Tom Lane)
2272
Appendix E. Release Notes •
Speed up parallel pg_restore when the archive contains many large objects (blobs) (Tom Lane)
•
Fix PL/pgSQL’s handling of “simple” expressions to not fail in recursion or error-recovery cases (Tom Lane)
•
Fix PL/pgSQL’s error reporting for no-such-column cases (Tom Lane) As of 9.0, it would sometimes report “missing FROM-clause entry for table foo” when “record foo has no field bar” would be more appropriate.
•
Fix PL/Python to honor typmod (i.e., length or precision restrictions) when assigning to tuple fields (Tom Lane) This fixes a regression from 8.4.
•
Fix PL/Python’s handling of set-returning functions (Jan Urbanski) Attempts to call SPI functions within the iterator generating a set result would fail.
•
Fix bug in contrib/cube’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a cube column. If you have such an index, consider REINDEXing it after installing this update.
•
Don’t emit “identifier will be truncated” notices in contrib/dblink except when creating new connections (Itagaki Takahiro)
•
Fix potential coredump on missing public key in contrib/pgcrypto (Marti Raudsepp)
•
Fix buffer overrun in contrib/pg_upgrade (Hernan Gonzalez)
•
Fix memory leak in contrib/xml2’s XPath query functions (Tom Lane)
•
Update time zone data files to tzdata release 2010o for DST law changes in Fiji and Samoa; also historical corrections for Hong Kong.
E.92. Release 9.0.1 Release date: 2010-10-04 This release contains a variety of fixes from 9.0.0. For information about new features in the 9.0 major release, see Section E.93.
E.92.1. Migration to Version 9.0.1 A dump/restore is not required for those running 9.0.X.
E.92.2. Changes •
Use a separate interpreter for each calling SQL userid in PL/Perl and PL/Tcl (Tom Lane) This change prevents security problems that can be caused by subverting Perl or Tcl code that will be executed later in the same session under another SQL user identity (for example, within a SECURITY DEFINER function). Most scripting languages offer numerous ways that that might be done, such as redefining standard functions or operators called by the target function. Without this
2273
Appendix E. Release Notes change, any SQL user with Perl or Tcl language usage rights can do essentially anything with the SQL privileges of the target function’s owner. The cost of this change is that intentional communication among Perl and Tcl functions becomes more difficult. To provide an escape hatch, PL/PerlU and PL/TclU functions continue to use only one interpreter per session. This is not considered a security issue since all such functions execute at the trust level of a database superuser already. It is likely that third-party procedural languages that claim to offer trusted execution have similar security issues. We advise contacting the authors of any PL you are depending on for securitycritical purposes. Our thanks to Tim Bunce for pointing out this issue (CVE-2010-3433). •
Improve pg_get_expr() security fix so that the function can still be used on the output of a sub-select (Tom Lane)
•
Fix incorrect placement of placeholder evaluation (Tom Lane) This bug could result in query outputs being non-null when they should be null, in cases where the inner side of an outer join is a sub-select with non-strict expressions in its output list.
•
Fix join removal’s handling of placeholder expressions (Tom Lane)
•
Fix possible duplicate scans of UNION ALL member relations (Tom Lane)
•
Prevent infinite loop in ProcessIncomingNotify() after unlistening (Jeff Davis)
•
Prevent show_session_authorization() from crashing within autovacuum processes (Tom Lane)
•
Re-allow input of Julian dates prior to 0001-01-01 AD (Tom Lane) Input such as ’J100000’::date worked before 8.4, but was unintentionally broken by added error-checking.
•
Make psql recognize DISCARD ALL as a command that should not be encased in a transaction block in autocommit-off mode (Itagaki Takahiro)
•
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
E.93. Release 9.0 Release date: 2010-09-20
E.93.1. Overview This release of PostgreSQL adds features that have been requested for years, such as easy-to-use replication, a mass permission-changing facility, and anonymous code blocks. While past major releases have been conservative in their scope, this release shows a bold new desire to provide facilities that new and existing users of PostgreSQL will embrace. This has all been done with few incompatibilities. Major enhancements include: •
Built-in replication based on log shipping. This advance consists of two features: Streaming Replication, allowing continuous archive (WAL) files to be streamed over a network connection to a standby server, and Hot Standby, allowing continuous archive standby servers to execute read-only queries. The net effect is to support a single master with multiple read-only slave servers.
2274
Appendix E. Release Notes •
Easier database object permissions management. GRANT/REVOKE IN SCHEMA supports mass permissions changes on existing objects, while ALTER DEFAULT PRIVILEGES allows control of privileges for objects created in the future. Large objects (BLOBs) now support permissions management as well.
•
Broadly enhanced stored procedure support. The DO statement supports ad-hoc or “anonymous” code blocks. Functions can now be called using named parameters. PL/pgSQL is now installed by default, and PL/Perl and PL/Python have been enhanced in several ways, including support for Python3.
•
Full support for 64-bit Windows.
•
More advanced reporting queries, including additional windowing options (PRECEDING and FOLLOWING) and the ability to control the order in which values are fed to aggregate functions.
•
New trigger features, including SQL-standard-compliant per-column triggers and conditional trigger execution.
•
Deferrable unique constraints. Mass updates to unique keys are now possible without trickery.
•
Exclusion constraints. These provide a generalized version of unique constraints, allowing enforcement of complex conditions.
•
New and enhanced security features, including RADIUS authentication, LDAP authentication improvements, and a new contrib module passwordcheck for testing password strength.
•
New high-performance implementation of the LISTEN/NOTIFY feature. Pending events are now stored in a memory-based queue rather than a table. Also, a “payload” string can be sent with each event, rather than transmitting just an event name as before.
•
New implementation of VACUUM FULL. This command now rewrites the entire table and indexes, rather than moving individual rows to compact space. It is substantially faster in most cases, and no longer results in index bloat.
•
New contrib module pg_upgrade to support in-place upgrades from 8.3 or 8.4 to 9.0.
•
Multiple performance enhancements for specific types of queries, including elimination of unnecessary joins. This helps optimize some automatically-generated queries, such as those produced by object-relational mappers (ORMs). enhancements. The output is now available in JSON, XML, or YAML format, and includes buffer utilization and other data not previously available.
• EXPLAIN
• hstore
improvements, including new functions and greater data capacity.
The above items are explained in more detail in the sections below.
E.93.2. Migration to Version 9.0 A dump/restore using pg_dump, or use of pg_upgrade, is required for those wishing to migrate data from any previous release. Version 9.0 contains a number of changes that selectively break backwards compatibility in order to support new features and code quality improvements. In particular, users who make extensive use of PL/pgSQL, Point-In-Time Recovery (PITR), or Warm Standby should test their applications because of slight user-visible changes in those areas. Observe the following incompatibilities:
2275
Appendix E. Release Notes
E.93.2.1. Server Settings •
Remove server parameter add_missing_from, which was defaulted to off for many years (Tom Lane)
•
Remove server parameter regex_flavor, which was defaulted to advanced for many years (Tom Lane)
• archive_mode now only affects archive_command; a new setting, wal_level, affects the con-
tents of the write-ahead log (Heikki Linnakangas) • log_temp_files
now uses default file size units of kilobytes (Robert Haas)
E.93.2.2. Queries •
When querying a parent table, do not do any separate permission checks on child tables scanned as part of the query (Peter Eisentraut) The SQL standard specifies this behavior, and it is also much more convenient in practice than the former behavior of checking permissions on each child as well as the parent.
E.93.2.3. Data Types • bytea
output now appears in hex format by default (Peter Eisentraut)
The server parameter bytea_output can be used to select the traditional output format if needed for compatibility. •
Array input now considers only plain ASCII whitespace characters to be potentially ignorable; it will never ignore non-ASCII characters, even if they are whitespace according to some locales (Tom Lane) This avoids some corner cases where array values could be interpreted differently depending on the server’s locale settings.
•
Improve standards compliance of SIMILAR TO patterns and SQL-style substring() patterns (Tom Lane) This includes treating ? and {...} as pattern metacharacters, while they were simple literal characters before; that corresponds to new features added in SQL:2008. Also, ^ and $ are now treated as simple literal characters; formerly they were treated as metacharacters, as if the pattern were following POSIX rather than SQL rules. Also, in SQL-standard substring(), use of parentheses for nesting no longer interferes with capturing of a substring. Also, processing of bracket expressions (character classes) is now more standards-compliant.
•
Reject negative length values in 3-parameter substring() for bit strings, per the SQL standard (Tom Lane)
•
Make date_trunc truncate rather than round when reducing precision of fractional seconds (Tom Lane) The code always acted this way for integer-based dates/times. Now float-based dates/times behave similarly.
2276
Appendix E. Release Notes
E.93.2.4. Object Renaming •
Tighten enforcement of column name consistency during RENAME when a child table inherits the same column from multiple unrelated parents (KaiGai Kohei)
•
No longer automatically rename indexes and index columns when the underlying table columns are renamed (Tom Lane) Administrators can still rename such indexes and columns manually. This change will require an update of the JDBC driver, and possibly other drivers, so that unique indexes are correctly recognized after a rename.
• CREATE OR REPLACE FUNCTION
can no longer change the declared names of function parame-
ters (Pavel Stehule) In order to avoid creating ambiguity in named-parameter calls, it is no longer allowed to change the aliases for input parameters in the declaration of an existing function (although names can still be assigned to previously unnamed parameters). You now have to DROP and recreate the function to do that.
E.93.2.5. PL/pgSQL •
PL/pgSQL now throws an error if a variable name conflicts with a column name used in a query (Tom Lane) The former behavior was to bind ambiguous names to PL/pgSQL variables in preference to query columns, which often resulted in surprising misbehavior. Throwing an error allows easy detection of ambiguous situations. Although it’s recommended that functions encountering this type of error be modified to remove the conflict, the old behavior can be restored if necessary via the configuration parameter plpgsql.variable_conflict, or via the per-function option #variable_conflict.
•
PL/pgSQL no longer allows variable names that match certain SQL reserved words (Tom Lane) This is a consequence of aligning the PL/pgSQL parser to match the core SQL parser more closely. If necessary, variable names can be double-quoted to avoid this restriction.
•
PL/pgSQL now requires columns of composite results to match the expected type modifier as well as base type (Pavel Stehule, Tom Lane) For example, if a column of the result type is declared as NUMERIC(30,2), it is no longer acceptable to return a NUMERIC of some other precision in that column. Previous versions neglected to check the type modifier and would thus allow result rows that didn’t actually conform to the declared restrictions.
•
PL/pgSQL now treats selection into composite fields more consistently (Tom Lane) Formerly, a statement like SELECT ... INTO rec.fld FROM ... was treated as a scalar assignment even if the record field fld was of composite type. Now it is treated as a record assignment, the same as when the INTO target is a regular variable of composite type. So the values to be assigned to the field’s subfields should be written as separate columns of the SELECT list, not as a ROW(...) construct as in previous versions. If you need to do this in a way that will work in both 9.0 and previous releases, you can write something like rec.fld := ROW(...) FROM ....
•
Remove PL/pgSQL’s RENAME declaration (Tom Lane)
2277
Appendix E. Release Notes Instead of RENAME, use ALIAS, which can now create an alias for any variable, not only dollar sign parameter names (such as $1) as before.
E.93.2.6. Other Incompatibilities •
Deprecate use of => as an operator name (Robert Haas) Future versions of PostgreSQL will probably reject this operator name entirely, in order to support the SQL-standard notation for named function parameters. For the moment, it is still allowed, but a warning is emitted when such an operator is defined.
•
Remove support for platforms that don’t have a working 64-bit integer data type (Tom Lane) It is believed all still-supported platforms have working 64-bit integer data types.
E.93.3. Changes Version 9.0 has an unprecedented number of new major features, and over 200 enhancements, improvements, new commands, new functions, and other changes.
E.93.3.1. Server E.93.3.1.1. Continuous Archiving and Streaming Replication PostgreSQL’s existing standby-server capability has been expanded both to support read-only queries on standby servers and to greatly reduce the lag between master and standby servers. For many users, this will be a useful and low-administration form of replication, either for high availability or for horizontal scalability. •
Allow a standby server to accept read-only queries (Simon Riggs, Heikki Linnakangas) This feature is called Hot Standby. There are new postgresql.conf and recovery.conf settings to control this feature, as well as extensive documentation.
•
Allow write-ahead log (WAL) data to be streamed to a standby server (Fujii Masao, Heikki Linnakangas) This feature is called Streaming Replication. Previously WAL data could be sent to standby servers only in units of entire WAL files (normally 16 megabytes each). Streaming Replication eliminates this inefficiency and allows updates on the master to be propagated to standby servers with very little delay. There are new postgresql.conf and recovery.conf settings to control this feature, as well as extensive documentation.
•
Add pg_last_xlog_receive_location() and pg_last_xlog_replay_location(), which can be used to monitor standby server WAL activity (Simon Riggs, Fujii Masao, Heikki Linnakangas)
2278
Appendix E. Release Notes E.93.3.1.2. Performance •
Allow per-tablespace values to be set for sequential and random page cost estimates (seq_page_cost/random_page_cost) via ALTER TABLESPACE ... SET/RESET (Robert Haas)
•
Improve performance and reliability of EvalPlanQual rechecks in join queries (Tom Lane) UPDATE, DELETE, and SELECT FOR UPDATE/SHARE queries that involve joins will now behave much better when encountering freshly-updated rows.
•
Improve performance of TRUNCATE when the table was created or truncated earlier in the same transaction (Tom Lane)
•
Improve performance of finding inheritance child tables (Tom Lane)
E.93.3.1.3. Optimizer •
Remove unnecessary outer joins (Robert Haas) Outer joins where the inner side is unique and not referenced above the join are unnecessary and are therefore now removed. This will accelerate many automatically generated queries, such as those created by object-relational mappers (ORMs).
•
Allow IS NOT NULL restrictions to use indexes (Tom Lane) This is particularly useful for finding MAX()/MIN() values in indexes that contain many null values.
•
Improve the optimizer’s choices about when to use materialize nodes, and when to use sorting versus hashing for DISTINCT (Tom Lane)
•
Improve the optimizer’s equivalence detection for expressions involving boolean operators (Tom Lane)
E.93.3.1.4. GEQO •
Use the same random seed every time GEQO plans a query (Andres Freund) While the Genetic Query Optimizer (GEQO) still selects random plans, it now always selects the same random plans for identical queries, thus giving more consistent performance. You can modify geqo_seed to experiment with alternative plans.
•
Improve GEQO plan selection (Tom Lane) This avoids the rare error “failed to make a valid plan”, and should also improve planning speed.
E.93.3.1.5. Optimizer Statistics •
Improve ANALYZE to support inheritance-tree statistics (Tom Lane) This is particularly useful for partitioned tables. However, autovacuum does not yet automatically re-analyze parent tables when child tables change.
•
Improve autovacuum’s detection of when re-analyze is necessary (Tom Lane)
•
Improve optimizer’s estimation for greater/less-than comparisons (Tom Lane)
2279
Appendix E. Release Notes When looking up statistics for greater/less-than comparisons, if the comparison value is in the first or last histogram bucket, use an index (if available) to fetch the current actual column minimum or maximum. This greatly improves the accuracy of estimates for comparison values near the ends of the data range, particularly if the range is constantly changing due to addition of new data. •
Allow setting of number-of-distinct-values statistics using ALTER TABLE (Robert Haas) This allows users to override the estimated number or percentage of distinct values for a column. This statistic is normally computed by ANALYZE, but the estimate can be poor, especially on tables with very large numbers of rows.
E.93.3.1.6. Authentication •
Add support for RADIUS (Remote Authentication Dial In User Service) authentication (Magnus Hagander)
•
Allow LDAP (Lightweight Directory Access Protocol) authentication to operate in “search/bind” mode (Robert Fleming, Magnus Hagander) This allows the user to be looked up first, then the system uses the DN (Distinguished Name) returned for that user.
•
Add samehost and samenet designations to pg_hba.conf (Stef Walter) These match the server’s IP address and subnet address respectively.
•
Pass trusted SSL root certificate names to the client so the client can return an appropriate client certificate (Craig Ringer)
E.93.3.1.7. Monitoring •
Add the ability for clients to set an application name, which is displayed in pg_stat_activity (Dave Page) This allows administrators to characterize database traffic and troubleshoot problems by source application.
•
Add a SQLSTATE option (%e) to log_line_prefix (Guillaume Smet) This allows users to compile statistics on errors and messages by error code number.
•
Write to the Windows event log in UTF16 encoding (Itagaki Takahiro) Now there is true multilingual support for PostgreSQL log messages on Windows.
E.93.3.1.8. Statistics Counters •
Add pg_stat_reset_shared(’bgwriter’) to reset the cluster-wide shared statistics for the background writer (Greg Smith)
•
Add
pg_stat_reset_single_table_counters() pg_stat_reset_single_function_counters() to allow
resetting
and the
statistics counters for individual tables and functions (Magnus Hagander)
2280
Appendix E. Release Notes E.93.3.1.9. Server Settings •
Allow setting of configuration parameters based on database/role combinations (Alvaro Herrera) Previously only per-database and per-role settings were possible, not combinations. All role and database settings are now stored in the new pg_db_role_setting system catalog. A new psql command \drds shows these settings. The legacy system views pg_roles, pg_shadow, and pg_user do not show combination settings, and therefore no longer completely represent the configuration for a user or database.
•
Add server parameter bonjour, which controls whether a Bonjour-enabled server advertises itself via Bonjour (Tom Lane) The default is off, meaning it does not advertise. This allows packagers to distribute Bonjourenabled builds without worrying that individual users might not want the feature.
•
Add server parameter enable_material, which controls the use of materialize nodes in the optimizer (Robert Haas) The default is on. When off, the optimizer will not add materialize nodes purely for performance reasons, though they will still be used when necessary for correctness.
•
Change server parameter log_temp_files to use default file size units of kilobytes (Robert Haas) Previously this setting was interpreted in bytes if no units were specified.
•
Log changes of parameter values when postgresql.conf is reloaded (Peter Eisentraut) This lets administrators and security staff audit changes of database settings, and is also very convenient for checking the effects of postgresql.conf edits.
•
Properly enforce superuser permissions for custom server parameters (Tom Lane) Non-superusers can no longer issue ALTER ROLE/DATABASE SET for parameters that are not currently known to the server. This allows the server to correctly check that superuser-only parameters are only set by superusers. Previously, the SET would be allowed and then ignored at session start, making superuser-only custom parameters much less useful than they should be.
E.93.3.2. Queries •
Perform SELECT FOR UPDATE/SHARE processing after applying LIMIT, so the number of rows returned is always predictable (Tom Lane) Previously, changes made by concurrent transactions could cause a SELECT FOR UPDATE to unexpectedly return fewer rows than specified by its LIMIT. FOR UPDATE in combination with ORDER BY can still produce surprising results, but that can be corrected by placing FOR UPDATE in a subquery.
•
Allow mixing of traditional and SQL-standard LIMIT/OFFSET syntax (Tom Lane)
•
Extend the supported frame options in window functions (Hitoshi Harada) Frames can now start with CURRENT ROW, and the ROWS n PRECEDING/FOLLOWING options are now supported.
•
Make SELECT INTO and CREATE TABLE AS return row counts to the client in their command tags (Boszormenyi Zoltan) This can save an entire round-trip to the client, allowing result counts and pagination to be calculated without an additional COUNT query.
2281
Appendix E. Release Notes E.93.3.2.1. Unicode Strings •
Support Unicode surrogate pairs (dual 16-bit representation) in U& strings and identifiers (Peter Eisentraut)
•
Support Unicode escapes in E’...’ strings (Marko Kreen)
E.93.3.3. Object Manipulation •
Speed up CREATE DATABASE by deferring flushes to disk (Andres Freund, Greg Stark)
•
Allow comments on columns of tables, views, and composite types only, not other relation types such as indexes and TOAST tables (Tom Lane)
•
Allow the creation of enumerated types containing no values (Bruce Momjian)
•
Let values of columns having storage type MAIN remain on the main heap page unless the row cannot fit on a page (Kevin Grittner) Previously MAIN values were forced out to TOAST tables until the row size was less than onequarter of the page size.
E.93.3.3.1. ALTER TABLE •
Implement
IF EXISTS for CONSTRAINT (Andres Freund)
•
Allow ALTER TABLE commands that rewrite tables to skip WAL logging (Itagaki Takahiro)
ALTER TABLE DROP COLUMN
and
ALTER TABLE DROP
Such operations either produce a new copy of the table or are rolled back, so WAL archiving can be skipped, unless running in continuous archiving mode. This reduces I/O overhead and improves performance. •
Fix failure of ALTER TABLE table ADD COLUMN col serial when done by non-owner of table (Tom Lane)
E.93.3.3.2. CREATE TABLE •
Add support for copying COMMENTS and STORAGE settings in CREATE TABLE ... LIKE commands (Itagaki Takahiro)
•
Add a shortcut for copying all properties in CREATE TABLE ... LIKE commands (Itagaki Takahiro)
•
Add the SQL-standard CREATE TABLE ... OF type command (Peter Eisentraut) This allows creation of a table that matches an existing composite type. Additional constraints and defaults can be specified in the command.
E.93.3.3.3. Constraints •
Add deferrable unique constraints (Dean Rasheed)
2282
Appendix E. Release Notes This allows mass updates, such as UPDATE tab SET col = col + 1, to work reliably on columns that have unique indexes or are marked as primary keys. If the constraint is specified as DEFERRABLE it will be checked at the end of the statement, rather than after each row is updated. The constraint check can also be deferred until the end of the current transaction, allowing such updates to be spread over multiple SQL commands. •
Add exclusion constraints (Jeff Davis) Exclusion constraints generalize uniqueness constraints by allowing arbitrary comparison operators, not just equality. They are created with the CREATE TABLE CONSTRAINT ... EXCLUDE clause. The most common use of exclusion constraints is to specify that column entries must not overlap, rather than simply not be equal. This is useful for time periods and other ranges, as well as arrays. This feature enhances checking of data integrity for many calendaring, time-management, and scientific applications.
•
Improve uniqueness-constraint violation error messages to report the values causing the failure (Itagaki Takahiro) For example, a uniqueness constraint violation might now report Key (x)=(2) already exists.
E.93.3.3.4. Object Permissions •
Add the ability to make mass permission changes across a whole schema using the new GRANT/REVOKE IN SCHEMA clause (Petr Jelinek) This simplifies management of object permissions and makes it easier to utilize database roles for application data security.
•
Add ALTER DEFAULT PRIVILEGES command to control privileges of objects created later (Petr Jelinek) This greatly simplifies the assignment of object privileges in a complex database application. Default privileges can be set for tables, views, sequences, and functions. Defaults may be assigned on a per-schema basis, or database-wide.
•
Add the ability to control large object (BLOB) permissions with GRANT/REVOKE (KaiGai Kohei) Formerly, any database user could read or modify any large object. Read and write permissions can now be granted and revoked per large object, and the ownership of large objects is tracked.
E.93.3.4. Utility Operations •
Make LISTEN/NOTIFY store pending events in a memory queue, rather than in a system table (Joachim Wieland) This substantially improves performance, while retaining the existing features of transactional support and guaranteed delivery.
•
Allow NOTIFY to pass an optional “payload” string to listeners (Joachim Wieland) This greatly improves the usefulness of LISTEN/NOTIFY as a general-purpose event queue system.
•
Allow CLUSTER on all per-database system catalogs (Tom Lane) Shared catalogs still cannot be clustered.
2283
Appendix E. Release Notes E.93.3.4.1. COPY •
Accept COPY ... CSV FORCE QUOTE * (Itagaki Takahiro) Now * can be used as shorthand for “all columns” in the FORCE QUOTE clause.
•
Add new COPY syntax that allows options to be specified inside parentheses (Robert Haas, Emmanuel Cecchet) This allows greater flexibility for future COPY options. The old syntax is still supported, but only for pre-existing options.
E.93.3.4.2. EXPLAIN •
Allow EXPLAIN to output in XML, JSON, or YAML format (Robert Haas, Greg Sabino Mullane) The new output formats are easily machine-readable, supporting the development of new tools for analysis of EXPLAIN output.
•
Add new BUFFERS option to report query buffer usage during EXPLAIN ANALYZE (Itagaki Takahiro) This allows better query profiling for individual queries. Buffer usage is no longer reported in the output for log_statement_stats and related settings.
•
Add hash usage information to EXPLAIN output (Robert Haas)
•
Add new EXPLAIN syntax that allows options to be specified inside parentheses (Robert Haas) This allows greater flexibility for future EXPLAIN options. The old syntax is still supported, but only for pre-existing options.
E.93.3.4.3. VACUUM •
Change VACUUM FULL to rewrite the entire table and rebuild its indexes, rather than moving individual rows around to compact space (Itagaki Takahiro, Tom Lane) The previous method was usually slower and caused index bloat. Note that the new method will use more disk space transiently during VACUUM FULL; potentially as much as twice the space normally occupied by the table and its indexes.
•
Add new VACUUM syntax that allows options to be specified inside parentheses (Itagaki Takahiro) This allows greater flexibility for future VACUUM options. The old syntax is still supported, but only for pre-existing options.
E.93.3.4.4. Indexes •
Allow an index to be named automatically by omitting the index name in CREATE INDEX (Tom Lane)
•
By default, multicolumn indexes are now named after all their columns; and index expression columns are now named based on their expressions (Tom Lane)
•
Reindexing shared system catalogs is now fully transactional and crash-safe (Tom Lane)
2284
Appendix E. Release Notes Formerly, reindexing a shared index was only allowed in standalone mode, and a crash during the operation could leave the index in worse condition than it was before. •
Add point_ops operator class for GiST (Teodor Sigaev) This feature permits GiST indexing of point columns. The index can be used for several types of queries such as point and operator. It was backpatched so that future-proofed code can be used with older server versions. Note that the patch will be effective only after contrib/hstore is installed or reinstalled in a particular database. Users might prefer to execute the CREATE FUNCTION command by hand, instead.
•
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
•
Update time zone data files to tzdata release 2010l for DST law changes in Egypt and Palestine; also historical corrections for Finland. This change also adds new names for two Micronesian timezones: Pacific/Chuuk is now preferred over Pacific/Truk (and the preferred abbreviation is CHUT not TRUT) and Pacific/Pohnpei is preferred over Pacific/Ponape.
•
Make Windows’ “N. Central Asia Standard Time” timezone map to Asia/Novosibirsk, not Asia/Almaty (Magnus Hagander) Microsoft changed the DST behavior of this zone in the timezone update from KB976098. Asia/Novosibirsk is a better match to its new behavior.
E.112. Release 8.4.4 Release date: 2010-05-17 This release contains a variety of fixes from 8.4.3. For information about new features in the 8.4 major release, see Section E.116.
2329
Appendix E. Release Notes
E.112.1. Migration to Version 8.4.4 A dump/restore is not required for those running 8.4.X. However, if you are upgrading from a version earlier than 8.4.2, see Section E.114.
E.112.2. Changes •
Enforce restrictions in plperl using an opmask applied to the whole interpreter, instead of using Safe.pm (Tim Bunce, Andrew Dunstan) Recent developments have convinced us that Safe.pm is too insecure to rely on for making plperl trustable. This change removes use of Safe.pm altogether, in favor of using a separate interpreter with an opcode mask that is always applied. Pleasant side effects of the change include that it is now possible to use Perl’s strict pragma in a natural way in plperl, and that Perl’s $a and $b variables work as expected in sort routines, and that function compilation is significantly faster. (CVE-2010-1169)
•
Prevent PL/Tcl from executing untrustworthy code from pltcl_modules (Tom) PL/Tcl’s feature for autoloading Tcl code from a database table could be exploited for trojan-horse attacks, because there was no restriction on who could create or insert into that table. This change disables the feature unless pltcl_modules is owned by a superuser. (However, the permissions on the table are not checked, so installations that really need a less-than-secure modules table can still grant suitable privileges to trusted non-superusers.) Also, prevent loading code into the unrestricted “normal” Tcl interpreter unless we are really going to execute a pltclu function. (CVE-2010-1170)
•
Fix data corruption during WAL replay of ALTER ... SET TABLESPACE (Tom) When archive_mode is on, ALTER ... SET TABLESPACE generates a WAL record whose replay logic was incorrect. It could write the data to the wrong place, leading to possibly-unrecoverable data corruption. Data corruption would be observed on standby slaves, and could occur on the master as well if a database crash and recovery occurred after committing the ALTER and before the next checkpoint.
•
Fix possible crash if a cache reset message is received during rebuild of a relcache entry (Heikki) This error was introduced in 8.4.3 while fixing a related failure.
•
Apply per-function GUC settings while running the language validator for the function (Itagaki Takahiro) This avoids failures if the function’s code is invalid without the setting; an example is that SQL functions may not parse if the search_path is not correct.
•
Do
constraint exclusion for inherited UPDATE constraint_exclusion = partition (Tom)
and
DELETE
target
tables
when
Due to an oversight, this setting previously only caused constraint exclusion to be checked in SELECT commands. •
Do not allow an unprivileged user to reset superuser-only parameter settings (Alvaro) Previously, if an unprivileged user ran ALTER USER ... RESET ALL for himself, or ALTER DATABASE ... RESET ALL for a database he owns, this would remove all special parameter settings for the user or database, even ones that are only supposed to be changeable by a superuser. Now, the ALTER will only remove the parameters that the user has permission to change.
2330
Appendix E. Release Notes •
Avoid possible crash during backend shutdown if shutdown occurs when a CONTEXT addition would be made to log entries (Tom) In some cases the context-printing function would fail because the current transaction had already been rolled back when it came time to print a log message.
•
Fix erroneous handling of %r parameter in recovery_end_command (Heikki) The value always came out zero.
•
Ensure the archiver process responds to changes in archive_command as soon as possible (Tom)
•
Fix pl/pgsql’s CASE statement to not fail when the case expression is a query that returns no rows (Tom)
•
Update pl/perl’s ppport.h for modern Perl versions (Andrew)
•
Fix assorted memory leaks in pl/python (Andreas Freund, Tom)
•
Handle empty-string connect parameters properly in ecpg (Michael)
•
Prevent infinite recursion in psql when expanding a variable that refers to itself (Tom)
•
Fix psql’s \copy to not add spaces around a dot within \copy (select ...) (Tom) Addition of spaces around the decimal point in a numeric literal would result in a syntax error.
•
Avoid formatting failure in psql when running in a locale context that doesn’t match the client_encoding (Tom)
•
Fix unnecessary “GIN indexes do not support whole-index scans” errors for unsatisfiable queries using contrib/intarray operators (Tom)
•
Ensure that contrib/pgstattuple functions respond to cancel interrupts promptly (Tatsuhito Kasahara)
•
Make server startup deal properly with the case that shmget() returns EINVAL for an existing shared memory segment (Tom) This behavior has been observed on BSD-derived kernels including OS X. It resulted in an entirelymisleading startup failure complaining that the shared memory request size was too large.
•
Avoid possible crashes in syslogger process on Windows (Heikki)
•
Deal more robustly with incomplete time zone information in the Windows registry (Magnus)
•
Update the set of known Windows time zone names (Magnus)
•
Update time zone data files to tzdata release 2010j for DST law changes in Argentina, Australian Antarctic, Bangladesh, Mexico, Morocco, Pakistan, Palestine, Russia, Syria, Tunisia; also historical corrections for Taiwan. Also, add PKST (Pakistan Summer Time) to the default set of timezone abbreviations.
E.113. Release 8.4.3 Release date: 2010-03-15 This release contains a variety of fixes from 8.4.2. For information about new features in the 8.4 major release, see Section E.116.
2331
Appendix E. Release Notes
E.113.1. Migration to Version 8.4.3 A dump/restore is not required for those running 8.4.X. However, if you are upgrading from a version earlier than 8.4.2, see Section E.114.
E.113.2. Changes •
Add new configuration parameter ssl_renegotiation_limit to control how often we do session key renegotiation for an SSL connection (Magnus) This can be set to zero to disable renegotiation completely, which may be required if a broken SSL library is used. In particular, some vendors are shipping stopgap patches for CVE-2009-3555 that cause renegotiation attempts to fail.
•
Fix possible deadlock during backend startup (Tom)
•
Fix possible crashes due to not handling errors during relcache reload cleanly (Tom)
•
Fix possible crash due to use of dangling pointer to a cached plan (Tatsuo)
•
Fix possible crash due to overenthusiastic invalidation of cached plan for ROLLBACK (Tom)
•
Fix possible crashes when trying to recover from a failure in subtransaction start (Tom)
•
Fix server memory leak associated with use of savepoints and a client encoding different from server’s encoding (Tom)
•
Fix incorrect WAL data emitted during end-of-recovery cleanup of a GIST index page split (Yoichi Hirai) This would result in index corruption, or even more likely an error during WAL replay, if we were unlucky enough to crash during end-of-recovery cleanup after having completed an incomplete GIST insertion.
•
Fix bug in WAL redo cleanup method for GIN indexes (Heikki)
•
Fix incorrect comparison of scan key in GIN index search (Teodor)
•
Make substring() for bit types treat any negative length as meaning “all the rest of the string” (Tom) The previous coding treated only -1 that way, and would produce an invalid result value for other negative values, possibly leading to a crash (CVE-2010-0442).
•
Fix integer-to-bit-string conversions to handle the first fractional byte correctly when the output bit width is wider than the given integer by something other than a multiple of 8 bits (Tom)
•
Fix some cases of pathologically slow regular expression matching (Tom)
•
Fix bug occurring when trying to inline a SQL function that returns a set of a composite type that contains dropped columns (Tom)
•
Fix bug with trying to update a field of an element of a composite-type array column (Tom)
•
Avoid failure when EXPLAIN has to print a FieldStore or assignment ArrayRef expression (Tom) These cases can arise now that EXPLAIN VERBOSE tries to print plan node target lists.
•
Avoid an unnecessary coercion failure in some cases where an undecorated literal string appears in a subquery within UNION/INTERSECT/EXCEPT (Tom) This fixes a regression for some cases that worked before 8.4.
2332
Appendix E. Release Notes •
Avoid undesirable rowtype compatibility check failures in some cases where a whole-row Var has a rowtype that contains dropped columns (Tom)
•
Fix the STOP WAL LOCATION entry in backup history files to report the next WAL segment’s name when the end location is exactly at a segment boundary (Itagaki Takahiro)
•
Always pass the catalog ID to an option validator function specified in CREATE FOREIGN DATA WRAPPER (Martin Pihlak)
•
Fix some more cases of temporary-file leakage (Heikki) This corrects a problem introduced in the previous minor release. One case that failed is when a plpgsql function returning set is called within another function’s exception handler.
•
Add support for doing FULL JOIN ON FALSE (Tom) This prevents a regression from pre-8.4 releases for some queries that can now be simplified to a constant-false join condition.
•
Improve constraint exclusion processing of boolean-variable cases, in particular make it possible to exclude a partition that has a “bool_column = false” constraint (Tom)
•
Prevent treating an INOUT cast as representing binary compatibility (Heikki)
•
Include column name in the message when warning about inability to grant or revoke column-level privileges (Stephen Frost) This is more useful than before and helps to prevent confusion when a REVOKE generates multiple messages, which formerly appeared to be duplicates.
•
When reading pg_hba.conf and related files, do not treat @something as a file inclusion request if the @ appears inside quote marks; also, never treat @ by itself as a file inclusion request (Tom) This prevents erratic behavior if a role or database name starts with @. If you need to include a file whose path name contains spaces, you can still do so, but you must write @"/path to/file" rather than putting the quotes around the whole construct.
•
Prevent infinite loop on some platforms if a directory is named as an inclusion target in pg_hba.conf and related files (Tom)
•
Fix possible infinite loop if SSL_read or SSL_write fails without setting errno (Tom) This is reportedly possible with some Windows versions of openssl.
•
Disallow GSSAPI authentication on local connections, since it requires a hostname to function correctly (Magnus)
•
Protect ecpg against applications freeing strings unexpectedly (Michael)
•
Make ecpg report the proper SQLSTATE if the connection disappears (Michael)
•
Fix translation of cell contents in psql \d output (Heikki)
•
Fix psql’s numericlocale option to not format strings it shouldn’t in latex and troff output formats (Heikki)
•
Fix a small per-query memory leak in psql (Tom)
•
Make psql return the correct exit status (3) when ON_ERROR_STOP and --single-transaction are both specified and an error occurs during the implied COMMIT (Bruce)
•
Fix pg_dump’s output of permissions for foreign servers (Heikki)
•
Fix possible crash in parallel pg_restore due to out-of-range dependency IDs (Tom)
•
Fix plpgsql failure in one case where a composite column is set to NULL (Tom)
•
Fix possible failure when calling PL/Perl functions from PL/PerlU or vice versa (Tim Bunce)
2333
Appendix E. Release Notes •
Add volatile markings in PL/Python to avoid possible compiler-specific misbehavior (Zdenek Kotala)
•
Ensure PL/Tcl initializes the Tcl interpreter fully (Tom) The only known symptom of this oversight is that the Tcl clock command misbehaves if using Tcl 8.5 or later.
•
Prevent ExecutorEnd from being run on portals created within a failed transaction or subtransaction (Tom) This is known to cause issues when using contrib/auto_explain.
•
Prevent crash in contrib/dblink when too many key columns are specified to a dblink_build_sql_* function (Rushabh Lathia, Joe Conway)
•
Allow zero-dimensional arrays in contrib/ltree operations (Tom) This case was formerly rejected as an error, but it’s more convenient to treat it the same as a zeroelement array. In particular this avoids unnecessary failures when an ltree operation is applied to the result of ARRAY(SELECT ...) and the sub-select returns no rows.
•
Fix assorted crashes in contrib/xml2 caused by sloppy memory management (Tom)
•
Make building of contrib/xml2 more robust on Windows (Andrew)
•
Fix race condition in Windows signal handling (Radu Ilie) One known symptom of this bug is that rows in pg_listener could be dropped under heavy load.
•
Make the configure script report failure if the C compiler does not provide a working 64-bit integer datatype (Tom) This case has been broken for some time, and no longer seems worth supporting, so just reject it at configure time instead.
•
Update time zone data files to tzdata release 2010e for DST law changes in Bangladesh, Chile, Fiji, Mexico, Paraguay, Samoa.
E.114. Release 8.4.2 Release date: 2009-12-14 This release contains a variety of fixes from 8.4.1. For information about new features in the 8.4 major release, see Section E.116.
E.114.1. Migration to Version 8.4.2 A dump/restore is not required for those running 8.4.X. However, if you have any hash indexes, you should REINDEX them after updating to 8.4.2, to repair possible damage.
E.114.2. Changes •
Protect against indirect security threats caused by index functions changing session-local state (Gurjeet Singh, Tom)
2334
Appendix E. Release Notes This change prevents allegedly-immutable index functions from possibly subverting a superuser’s session (CVE-2009-4136). •
Reject SSL certificates containing an embedded null byte in the common name (CN) field (Magnus) This prevents unintended matching of a certificate to a server or client name during SSL validation (CVE-2009-4034).
•
Fix hash index corruption (Tom) The 8.4 change that made hash indexes keep entries sorted by hash value failed to update the bucket splitting and compaction routines to preserve the ordering. So application of either of those operations could lead to permanent corruption of an index, in the sense that searches might fail to find entries that are present. To deal with this, it is recommended to REINDEX any hash indexes you may have after installing this update.
•
Fix possible crash during backend-startup-time cache initialization (Tom)
•
Avoid crash on empty thesaurus dictionary (Tom)
•
Prevent signals from interrupting VACUUM at unsafe times (Alvaro) This fix prevents a PANIC if a VACUUM FULL is canceled after it’s already committed its tuple movements, as well as transient errors if a plain VACUUM is interrupted after having truncated the table.
•
Fix possible crash due to integer overflow in hash table size calculation (Tom) This could occur with extremely large planner estimates for the size of a hashjoin’s result.
•
Fix crash if a DROP is attempted on an internally-dependent object (Tom)
•
Fix very rare crash in inet/cidr comparisons (Chris Mikkelson)
•
Ensure that shared tuple-level locks held by prepared transactions are not ignored (Heikki)
•
Fix premature drop of temporary files used for a cursor that is accessed within a subtransaction (Heikki)
•
Fix memory leak in syslogger process when rotating to a new CSV logfile (Tom)
•
Fix memory leak in postmaster when re-parsing pg_hba.conf (Tom)
•
Fix Windows permission-downgrade logic (Jesse Morris) This fixes some cases where the database failed to start on Windows, often with misleading error messages such as “could not locate matching postgres executable”.
•
Make FOR UPDATE/SHARE in the primary query not propagate into WITH queries (Tom) For example, in WITH w AS (SELECT * FROM foo) SELECT * FROM w, bar ... FOR UPDATE the FOR UPDATE will now affect bar but not foo. This is more useful and consistent than the original 8.4 behavior, which tried to propagate FOR UPDATE into the WITH query but always failed due to assorted implementation restrictions. It also follows the design rule that WITH queries are
executed as if independent of the main query. •
Fix bug with a WITH RECURSIVE query immediately inside another one (Tom)
•
Fix concurrency bug in hash indexes (Tom) Concurrent insertions could cause index scans to transiently report wrong results.
•
Fix incorrect logic for GiST index page splits, when the split depends on a non-first column of the index (Paul Ramsey)
•
Fix wrong search results for a multi-column GIN index with fastupdate enabled (Teodor)
2335
Appendix E. Release Notes •
Fix bugs in WAL entry creation for GIN indexes (Tom) These bugs were masked when full_page_writes was on, but with it off a WAL replay failure was certain if a crash occurred before the next checkpoint.
•
Don’t error out if recycling or removing an old WAL file fails at the end of checkpoint (Heikki) It’s better to treat the problem as non-fatal and allow the checkpoint to complete. Future checkpoints will retry the removal. Such problems are not expected in normal operation, but have been seen to be caused by misdesigned Windows anti-virus and backup software.
•
Ensure WAL files aren’t repeatedly archived on Windows (Heikki) This is another symptom that could happen if some other process interfered with deletion of a no-longer-needed file.
•
Fix PAM password processing to be more robust (Tom) The previous code is known to fail with the combination of the Linux pam_krb5 PAM module with Microsoft Active Directory as the domain controller. It might have problems elsewhere too, since it was making unjustified assumptions about what arguments the PAM stack would pass to it.
•
Raise the maximum authentication token (Kerberos ticket) size in GSSAPI and SSPI authentication methods (Ian Turner) While the old 2000-byte limit was more than enough for Unix Kerberos implementations, tickets issued by Windows Domain Controllers can be much larger.
•
Ensure that domain constraints are enforced in constructs like ARRAY[...]::domain, where the domain is over an array type (Heikki)
•
Fix foreign-key logic for some cases involving composite-type columns as foreign keys (Tom)
•
Ensure that a cursor’s snapshot is not modified after it is created (Alvaro) This could lead to a cursor delivering wrong results if later operations in the same transaction modify the data the cursor is supposed to return.
•
Fix CREATE TABLE to properly merge default expressions coming from different inheritance parent tables (Tom) This used to work but was broken in 8.4.
•
Re-enable collection of access statistics for sequences (Akira Kurosawa) This used to work but was broken in 8.3.
•
Fix processing of ownership dependencies during CREATE OR REPLACE FUNCTION (Tom)
•
Fix incorrect handling of WHERE x =x conditions (Tom) In some cases these could get ignored as redundant, but they aren’t — they’re equivalent to x IS NOT NULL.
•
Fix incorrect plan construction when using hash aggregation to implement DISTINCT for textually identical volatile expressions (Tom)
•
Fix Assert failure for a volatile SELECT DISTINCT ON expression (Tom)
•
Fix ts_stat() to not fail on an empty tsvector value (Tom)
•
Make text search parser accept underscores in XML attributes (Peter)
•
Fix encoding handling in xml binary input (Heikki) If the XML header doesn’t specify an encoding, we now assume UTF-8 by default; the previous handling was inconsistent.
2336
Appendix E. Release Notes •
Fix bug with calling plperl from plperlu or vice versa (Tom) An error exit from the inner function could result in crashes due to failure to re-select the correct Perl interpreter for the outer function.
•
Fix session-lifespan memory leak when a PL/Perl function is redefined (Tom)
•
Ensure that Perl arrays are properly converted to PostgreSQL arrays when returned by a setreturning PL/Perl function (Andrew Dunstan, Abhijit Menon-Sen) This worked correctly already for non-set-returning functions.
•
Fix rare crash in exception processing in PL/Python (Peter)
•
Fix ecpg problem with comments in DECLARE CURSOR statements (Michael)
•
Fix ecpg to not treat recently-added keywords as reserved words (Tom) This affected the keywords CALLED, CATALOG, DEFINER, ENUM, FOLLOWING, INVOKER, OPTIONS, PARTITION, PRECEDING, RANGE, SECURITY, SERVER, UNBOUNDED, and WRAPPER.
•
Re-allow regular expression special characters in psql’s \df function name parameter (Tom)
•
In contrib/fuzzystrmatch, correct the calculation of levenshtein distances with non-default costs (Marcin Mank)
•
In contrib/pg_standby, disable triggering failover with a signal on Windows (Fujii Masao) This never did anything useful, because Windows doesn’t have Unix-style signals, but recent changes made it actually crash.
•
Put FREEZE and VERBOSE options in the right order in the VACUUM command that contrib/vacuumdb produces (Heikki)
•
Fix possible leak of connections when contrib/dblink encounters an error (Tatsuhito Kasahara)
•
Ensure psql’s flex module is compiled with the correct system header definitions (Tom) This fixes build failures on platforms where --enable-largefile causes incompatible changes in the generated code.
•
Make the postmaster ignore any application_name parameter in connection request packets, to improve compatibility with future libpq versions (Tom)
•
Update the timezone abbreviation files to match current reality (Joachim Wieland) This includes adding IDT to the default timezone abbreviation set.
•
Update time zone data files to tzdata release 2009s for DST law changes in Antarctica, Argentina, Bangladesh, Fiji, Novokuznetsk, Pakistan, Palestine, Samoa, Syria; also historical corrections for Hong Kong.
E.115. Release 8.4.1 Release date: 2009-09-09 This release contains a variety of fixes from 8.4. For information about new features in the 8.4 major release, see Section E.116.
E.115.1. Migration to Version 8.4.1 A dump/restore is not required for those running 8.4.X.
2337
Appendix E. Release Notes
E.115.2. Changes •
Fix WAL page header initialization at the end of archive recovery (Heikki) This could lead to failure to process the WAL in a subsequent archive recovery.
•
Fix “cannot make new WAL entries during recovery” error (Tom)
•
Fix problem that could make expired rows visible after a crash (Tom) This bug involved a page status bit potentially not being set correctly after a server crash.
•
Disallow RESET ROLE and RESET SESSION AUTHORIZATION inside security-definer functions (Tom, Heikki) This covers a case that was missed in the previous patch that disallowed SET ROLE and SET SESSION AUTHORIZATION inside security-definer functions. (See CVE-2007-6600)
•
Make LOAD of an already-loaded loadable module into a no-op (Tom) Formerly, LOAD would attempt to unload and re-load the module, but this is unsafe and not all that useful.
•
Make window function PARTITION BY and ORDER BY items always be interpreted as simple expressions (Tom) In 8.4.0 these lists were parsed following the rules used for top-level GROUP BY and ORDER BY lists. But this was not correct per the SQL standard, and it led to possible circularity.
•
Fix several errors in planning of semi-joins (Tom) These led to wrong query results in some cases where IN or EXISTS was used together with another join.
•
Fix handling of whole-row references to subqueries that are within an outer join (Tom) An example is SELECT COUNT(ss.*) FROM ... LEFT JOIN (SELECT ...) ss ON .... Here, ss.* would be treated as ROW(NULL,NULL,...) for null-extended join rows, which is not the same as a simple NULL. Now it is treated as a simple NULL.
•
Fix Windows shared-memory allocation code (Tsutomu Yamada, Magnus) This bug led to the often-reported “could not reattach to shared memory” error message.
•
Fix locale handling with plperl (Heikki) This bug could cause the server’s locale setting to change when a plperl function is called, leading to data corruption.
•
Fix handling of reloptions to ensure setting one option doesn’t force default values for others (Itagaki Takahiro)
•
Ensure that a “fast shutdown” request will forcibly terminate open sessions, even if a “smart shutdown” was already in progress (Fujii Masao)
•
Avoid memory leak for array_agg() in GROUP BY queries (Tom)
•
Treat to_char(..., ’TH’) as an uppercase ordinal suffix with ’HH’/’HH12’ (Heikki) It was previously handled as ’th’ (lowercase).
•
Include the fractional part in the result of EXTRACT(second) and EXTRACT(milliseconds) for time and time with time zone inputs (Tom) This has always worked for floating-point datetime configurations, but was broken in the integer datetime code.
2338
Appendix E. Release Notes •
Fix overflow for INTERVAL ’x ms’ when x is more than 2 million and integer datetimes are in use (Alex Hunsaker)
•
Improve performance when processing toasted values in index scans (Tom) This is particularly useful for PostGIS5.
•
Fix a typo that disabled commit_delay (Jeff Janes)
•
Output early-startup messages to postmaster.log if the server is started in silent mode (Tom) Previously such error messages were discarded, leading to difficulty in debugging.
•
Remove translated FAQs (Peter) They are now on the wiki6. The main FAQ was moved to the wiki some time ago.
•
Fix pg_ctl to not go into an infinite loop if postgresql.conf is empty (Jeff Davis)
•
Fix several errors in pg_dump’s --binary-upgrade mode (Bruce, Tom) pg_dump --binary-upgrade is used by pg_migrator.
•
Fix contrib/xml2’s xslt_process() to properly handle the maximum number of parameters (twenty) (Tom)
•
Improve robustness of libpq’s code to recover from errors during COPY FROM STDIN (Tom)
•
Avoid including conflicting readline and editline header files when both libraries are installed (Zdenek Kotala)
•
Work around gcc bug that causes “floating-point exception” instead of “division by zero” on some platforms (Tom)
•
Update time zone data files to tzdata release 2009l for DST law changes in Bangladesh, Egypt, Mauritius.
E.116. Release 8.4 Release date: 2009-07-01
E.116.1. Overview After many years of development, PostgreSQL has become feature-complete in many areas. This release shows a targeted approach to adding features (e.g., authentication, monitoring, space reuse), and adds capabilities defined in the later SQL standards. The major areas of enhancement are:
Improved join performance for EXISTS and NOT EXISTS queries
•
Easier-to-use Warm Standby
•
Automatic sizing of the Free Space Map
•
Visibility Map (greatly reduces vacuum overhead for slowly-changing tables)
•
Version-aware psql (backslash commands work against older servers)
•
Support SSL certificates for user authentication
•
Per-function runtime statistics
•
Easy editing of functions in psql
•
New contrib modules: pg_stat_statements, auto_explain, citext, btree_gin
The above items are explained in more detail in the sections below.
E.116.2. Migration to Version 8.4 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities:
E.116.2.1. General •
Use 64-bit integer datetimes by default (Neil Conway) Previously this was selected by configure’s --enable-integer-datetimes option. To retain the old behavior, build with --disable-integer-datetimes.
•
Remove ipcclean utility command (Bruce) The utility only worked on a few platforms. Users should use their operating system tools instead.
E.116.2.2. Server Settings •
Change default setting for log_min_messages to warning (previously it was notice) to reduce log file volume (Tom)
•
Change default setting for max_prepared_transactions to zero (previously it was 5) (Tom)
•
Make debug_print_parse, debug_print_rewritten, and debug_print_plan output appear at LOG message level, not DEBUG1 as formerly (Tom)
•
Make debug_pretty_print default to on (Tom)
•
Remove explain_pretty_print parameter (no longer needed) (Tom)
•
Make log_temp_files settable by superusers only, like other logging options (Simon Riggs)
•
Remove automatic appending of the epoch timestamp when no % escapes are present in log_filename (Robert Haas) This change was made because some users wanted a fixed log filename, for use with an external log rotation tool.
2340
Appendix E. Release Notes •
Remove log_restartpoints from recovery.conf; instead use log_checkpoints (Simon)
•
Remove krb_realm and krb_server_hostname; these are now set in pg_hba.conf instead (Magnus)
•
There are also significant changes in pg_hba.conf, as described below.
E.116.2.3. Queries •
Change TRUNCATE and LOCK to apply to child tables of the specified table(s) (Peter) These commands now accept an ONLY option that prevents processing child tables; this option must be used if the old behavior is needed.
• SELECT DISTINCT
and UNION/INTERSECT/EXCEPT no longer always produce sorted output
(Tom) Previously, these types of queries always removed duplicate rows by means of Sort/Unique processing (i.e., sort then remove adjacent duplicates). Now they can be implemented by hashing, which will not produce sorted output. If an application relied on the output being in sorted order, the recommended fix is to add an ORDER BY clause. As a short-term workaround, the previous behavior can be restored by disabling enable_hashagg, but that is a very performance-expensive fix. SELECT DISTINCT ON never uses hashing, however, so its behavior is unchanged. •
Force child tables to inherit CHECK constraints from parents (Alex Hunsaker, Nikhil Sontakke, Tom) Formerly it was possible to drop such a constraint from a child table, allowing rows that violate the constraint to be visible when scanning the parent table. This was deemed inconsistent, as well as contrary to SQL standard.
•
Disallow negative LIMIT or OFFSET values, rather than treating them as zero (Simon)
•
Disallow LOCK TABLE outside a transaction block (Tom) Such an operation is useless because the lock would be released immediately.
•
Sequences now contain an additional start_value column (Zoltan Boszormenyi) This supports ALTER SEQUENCE ... RESTART.
E.116.2.4. Functions and Operators •
Make numeric zero raised to a fractional power return 0, rather than throwing an error, and make numeric zero raised to the zero power return 1, rather than error (Bruce) This matches the longstanding float8 behavior.
•
Allow unary minus of floating-point values to produce minus zero (Tom) The changed behavior is more IEEE-standard compliant.
•
Throw an error if an escape character is the last character in a LIKE pattern (i.e., it has nothing to escape) (Tom) Previously, such an escape character was silently ignored, thus possibly masking application logic errors.
•
Remove ~=~ and ~~ operators formerly used for LIKE index comparisons (Tom)
2341
Appendix E. Release Notes Pattern indexes now use the regular equality operator. • xpath()
now passes its arguments to libxml without any changes (Andrew)
This means that the XML argument must be a well-formed XML document. The previous coding attempted to allow XML fragments, but it did not work well. •
Make xmlelement() format attribute values just like content values (Peter) Previously, attribute values were formatted according to the normal SQL output behavior, which is sometimes at odds with XML rules.
•
Rewrite memory management for libxml-using functions (Tom) This change should avoid some compatibility problems with use of libxml in PL/Perl and other add-on code.
•
Adopt a faster algorithm for hash functions (Kenneth Marshall, based on work of Bob Jenkins) Many of the built-in hash functions now deliver different results on little-endian and big-endian platforms.
E.116.2.4.1. Temporal Functions and Operators • DateStyle
no longer controls interval output formatting; instead there is a new variable
IntervalStyle (Ron Mayer) •
Improve consistency of handling of fractional seconds in timestamp and interval output (Ron Mayer) This may result in displaying a different number of fractional digits than before, or rounding instead of truncating.
•
Make to_char()’s localized month/day names depend on LC_TIME, not LC_MESSAGES (Euler Taveira de Oliveira)
•
Cause to_date() and to_timestamp() to more consistently report errors for invalid input (Brendan Jurd) Previous versions would often ignore or silently misread input that did not match the format string. Such cases will now result in an error.
•
Fix to_timestamp() to not require upper/lower case matching for meridian (AM/PM) and era (BC/AD) format designations (Brendan Jurd) For example, input value ad now matches the format string AD.
E.116.3. Changes Below you will find a detailed account of the changes between PostgreSQL 8.4 and the previous major release.
E.116.3.1. Performance •
Improve optimizer statistics calculations (Jan Urbanski, Tom) In particular, estimates for full-text-search operators are greatly improved.
2342
Appendix E. Release Notes •
Allow SELECT DISTINCT and UNION/INTERSECT/EXCEPT to use hashing (Tom) This means that these types of queries no longer automatically produce sorted output.
•
Create explicit concepts of semi-joins and anti-joins (Tom) This work formalizes our previous ad-hoc treatment of IN (SELECT ...) clauses, and extends it to EXISTS and NOT EXISTS clauses. It should result in significantly better planning of EXISTS and NOT EXISTS queries. In general, logically equivalent IN and EXISTS clauses should now have similar performance, whereas previously IN often won.
•
Improve optimization of sub-selects beneath outer joins (Tom) Formerly, a sub-select or view could not be optimized very well if it appeared within the nullable side of an outer join and contained non-strict expressions (for instance, constants) in its result list.
•
Improve the performance of text_position() and related functions by using Boyer-MooreHorspool searching (David Rowley) This is particularly helpful for long search patterns.
•
Reduce I/O load of writing the statistics collection file by writing the file only when requested (Martin Pihlak)
•
Improve performance for bulk inserts (Robert Haas, Simon)
•
Increase the default value of default_statistics_target from 10 to 100 (Greg Sabino Mullane, Tom) The maximum value was also increased from 1000 to 10000.
•
Perform constraint_exclusion checking by default in queries involving inheritance or UNION ALL (Tom) A new constraint_exclusion setting, partition, was added to specify this behavior.
•
Allow I/O read-ahead for bitmap index scans (Greg Stark) The amount of read-ahead is controlled by effective_io_concurrency. This feature is available only if the kernel has posix_fadvise() support.
•
Inline simple set-returning SQL functions in FROM clauses (Richard Rowell)
•
Improve performance of multi-batch hash joins by providing a special case for join key values that are especially common in the outer relation (Bryce Cutt, Ramon Lawrence)
•
Reduce volume of temporary data in multi-batch hash joins by suppressing “physical tlist” optimization (Michael Henderson, Ramon Lawrence)
•
Avoid waiting for idle-in-transaction sessions during CREATE INDEX CONCURRENTLY (Simon)
•
Improve performance of shared cache invalidation (Tom)
E.116.3.2. Server E.116.3.2.1. Settings •
Convert many postgresql.conf settings to enumerated values so that pg_settings can display the valid values (Magnus)
•
Add cursor_tuple_fraction parameter to control the fraction of a cursor’s rows that the planner assumes will be fetched (Robert Hell)
2343
Appendix E. Release Notes •
Allow underscores in the names of custom variable classes in postgresql.conf (Tom)
E.116.3.2.2. Authentication and security •
Remove support for the (insecure) crypt authentication method (Magnus) This effectively obsoletes pre-PostgreSQL 7.2 client libraries, as there is no longer any non-plaintext password method that they can use.
•
Support regular expressions in pg_ident.conf (Magnus)
•
Allow Kerberos/GSSAPI parameters to be changed without restarting the postmaster (Magnus)
•
Support SSL certificate chains in server certificate file (Andrew Gierth) Including the full certificate chain makes the client able to verify the certificate without having all intermediate CA certificates present in the local store, which is often the case for commercial CAs.
•
Report
appropriate
error message db_user_namespace enabled (Bruce)
for
combination
of
MD5
authentication
and
E.116.3.2.3. pg_hba.conf •
Change all authentication options to use name=value syntax (Magnus) This makes incompatible changes to the ldap, pam and ident authentication methods. All pg_hba.conf entries with these methods need to be rewritten using the new format.
•
Remove the ident sameuser option, instead making that behavior the default if no usermap is specified (Magnus)
•
Allow a usermap parameter for all external authentication methods (Magnus) Previously a usermap was only supported for ident authentication.
•
Add clientcert option to control requesting of a client certificate (Magnus) Previously this was controlled by the presence of a root certificate file in the server’s data directory.
•
Add cert authentication method to allow user authentication via SSL certificates (Magnus) Previously SSL certificates could only verify that the client had access to a certificate, not authenticate a user.
•
Allow krb5, gssapi and sspi realm and krb5 host settings to be specified in pg_hba.conf (Magnus) These override the settings in postgresql.conf.
•
Add include_realm parameter for krb5, gssapi, and sspi methods (Magnus) This allows identical usernames from different realms to be authenticated as different database users using usermaps.
•
Parse pg_hba.conf fully when it is loaded, so that errors are reported immediately (Magnus) Previously, most errors in the file wouldn’t be detected until clients tried to connect, so an erroneous file could render the system unusable. With the new behavior, if an error is detected during reload then the bad file is rejected and the postmaster continues to use its old copy.
•
Show all parsing errors in pg_hba.conf instead of aborting after the first one (Selena Deckelmann)
2344
Appendix E. Release Notes •
Support ident authentication over Unix-domain sockets on Solaris (Garick Hamlin)
E.116.3.2.4. Continuous Archiving •
Provide an option to pg_start_backup() to force its implied checkpoint to finish as quickly as possible (Tom) The default behavior avoids excess I/O consumption, but that is pointless if no concurrent query activity is going on.
•
Make pg_stop_backup() wait for modified WAL files to be archived (Simon) This guarantees that the backup is valid at the time pg_stop_backup() completes.
•
When archiving is enabled, rotate the last WAL segment at shutdown so that all transactions can be archived immediately (Guillaume Smet, Heikki)
•
Delay “smart” shutdown while a continuous archiving base backup is in progress (Laurenz Albe)
•
Cancel a continuous archiving base backup if “fast” shutdown is requested (Laurenz Albe)
•
Allow recovery.conf boolean variables to take the same range of string values as postgresql.conf boolean variables (Bruce)
E.116.3.2.5. Monitoring •
Add pg_conf_load_time() to report when the PostgreSQL configuration files were last loaded (George Gensure)
•
Add pg_terminate_backend() to safely terminate a backend (the SIGTERM signal works also) (Tom, Bruce) While it’s always been possible to SIGTERM a single backend, this was previously considered unsupported; and testing of the case found some bugs that are now fixed.
•
Add ability to track user-defined functions’ call counts and runtimes (Martin Pihlak) Function statistics appear in a new system view, pg_stat_user_functions. Tracking is controlled by the new parameter track_functions.
•
Allow specification of the maximum query string size in pg_stat_activity via new track_activity_query_size parameter (Thomas Lee)
•
Increase the maximum line length sent to syslog, in hopes of improving performance (Tom)
When reporting a deadlock, report the text of all queries involved in the deadlock to the server log (Itagaki Takahiro)
•
Add pg_stat_get_activity(pid) function to return information about a specific process id (Magnus)
•
Allow the location of the server’s statistics file to be specified via stats_temp_directory (Magnus)
read-only
segment_size,
wal_block_size,
and
This allows the statistics file to be placed in a RAM-resident directory to reduce I/O requirements. On startup/shutdown, the file is copied to its traditional location ($PGDATA/global/) so it is preserved across restarts.
2345
Appendix E. Release Notes
E.116.3.3. Queries •
Add support for WINDOW functions (Hitoshi Harada)
•
Add support for WITH clauses (CTEs), including WITH RECURSIVE (Yoshiyuki Asaba, Tatsuo Ishii, Tom)
•
Add TABLE command (Peter) TABLE tablename is a SQL standard short-hand for SELECT * FROM tablename.
•
Allow AS to be optional when specifying a SELECT (or RETURNING) column output label (Hiroshi Saito) This works so long as the column label is not any PostgreSQL keyword; otherwise AS is still needed.
•
Support set-returning functions in SELECT result lists even for functions that return their result via a tuplestore (Tom) In particular, this means that functions written in PL/pgSQL and other PL languages can now be called this way.
•
Support set-returning functions in the output of aggregation and grouping queries (Tom)
•
Allow SELECT FOR UPDATE/SHARE to work on inheritance trees (Tom)
•
Add infrastructure for SQL/MED (Martin Pihlak, Peter) There are no remote or external SQL/MED capabilities yet, but this change provides a standardized and future-proof system for managing connection information for modules like dblink and plproxy.
•
Invalidate cached plans when referenced schemas, functions, operators, or operator classes are modified (Martin Pihlak, Tom) This improves the system’s ability to respond to on-the-fly DDL changes.
•
Allow comparison of composite types and allow arrays of anonymous composite types (Tom) This allows constructs such as row(1, 1.1) = any (array[row(7, 7.7), row(1, 1.0)]). This is particularly useful in recursive queries.
•
Add support for Unicode string literal and identifier specifications using code points, e.g. U&’d\0061t\+000061’ (Peter)
•
Reject \000 in string literals and COPY data (Tom) Previously, this was accepted but had the effect of terminating the string contents.
•
Improve the parser’s ability to report error locations (Tom) An error location is now reported for many semantic errors, such as mismatched datatypes, that previously could not be localized.
E.116.3.3.1. TRUNCATE •
Support statement-level ON TRUNCATE triggers (Simon)
•
Add RESTART/CONTINUE IDENTITY options for TRUNCATE TABLE (Zoltan Boszormenyi) The start value of a sequence can be changed by ALTER SEQUENCE START WITH.
•
Allow TRUNCATE tab1, tab1 to succeed (Bruce)
2346
Appendix E. Release Notes •
Add a separate TRUNCATE permission (Robert Haas)
E.116.3.3.2. EXPLAIN •
Make EXPLAIN VERBOSE show the output columns of each plan node (Tom) Previously EXPLAIN VERBOSE output an internal representation of the query plan. (That behavior is now available via debug_print_plan.)
•
Make EXPLAIN identify subplans and initplans with individual labels (Tom)
•
Make EXPLAIN honor debug_print_plan (Tom)
•
Allow EXPLAIN on CREATE TABLE AS (Peter)
E.116.3.3.3. LIMIT /OFFSET •
Allow sub-selects in LIMIT and OFFSET (Tom)
•
Add SQL-standard syntax for LIMIT/OFFSET capabilities (Peter) To wit, OFFSET num {ROW|ROWS} FETCH {FIRST|NEXT} [num] {ROW|ROWS} ONLY.
E.116.3.4. Object Manipulation •
Add support for column-level privileges (Stephen Frost, KaiGai Kohei)
•
Refactor multi-object DROP operations to reduce the need for CASCADE (Alex Hunsaker) For example, if table B has a dependency on table A, the command DROP TABLE A, B no longer requires the CASCADE option.
•
Fix various problems with concurrent DROP commands by ensuring that locks are taken before we begin to drop dependencies of an object (Tom)
•
Improve reporting of dependencies during DROP commands (Tom)
•
Add WITH [NO] DATA clause to CREATE TABLE AS, per the SQL standard (Peter, Tom)
•
Add support for user-defined I/O conversion casts (Heikki)
•
Allow CREATE AGGREGATE to use an internal transition datatype (Tom)
•
Add LIKE clause to CREATE TYPE (Tom) This simplifies creation of data types that use the same internal representation as an existing type.
•
Allow specification of the type category and “preferred” status for user-defined base types (Tom) This allows more control over the coercion behavior of user-defined types.
•
Allow CREATE OR REPLACE VIEW to add columns to the end of a view (Robert Haas)
E.116.3.4.1. ALTER •
Add ALTER TYPE RENAME (Petr Jelinek)
2347
Appendix E. Release Notes •
Add ALTER SEQUENCE ... RESTART (with no parameter) to reset a sequence to its initial value (Zoltan Boszormenyi)
•
Modify the ALTER TABLE syntax to allow all reasonable combinations for tables, indexes, sequences, and views (Tom) This change allows the following new syntaxes: •
ALTER SEQUENCE OWNER TO
•
ALTER VIEW ALTER COLUMN SET/DROP DEFAULT
•
ALTER VIEW OWNER TO
•
ALTER VIEW SET SCHEMA
There is no actual new functionality here, but formerly you had to say ALTER TABLE to do these things, which was confusing. •
Add support for the syntax ALTER TABLE ... ALTER COLUMN ... SET DATA TYPE (Peter) This is SQL-standard syntax for functionality that was already supported.
•
Make ALTER TABLE SET WITHOUT OIDS rewrite the table to physically remove OID values (Tom) Also, add ALTER TABLE SET WITH OIDS to rewrite the table to add OIDs.
E.116.3.4.2. Database Manipulation •
Improve reporting of CREATE/DROP/RENAME DATABASE failure when uncommitted prepared transactions are the cause (Tom)
•
Make LC_COLLATE and LC_CTYPE into per-database settings (Radek Strnad, Heikki) This makes collation similar to encoding, which was always configurable per database.
•
Improve checks that the database encoding, collation (LC_COLLATE), and character classes (LC_CTYPE) match (Heikki, Tom) Note in particular that a new database’s encoding and locale settings can be changed only when copying from template0. This prevents possibly copying data that doesn’t match the settings.
•
Add ALTER DATABASE SET TABLESPACE to move a database to a new tablespace (Guillaume Lelarge, Bernd Helmle)
E.116.3.5. Utility Operations •
Add a VERBOSE option to the CLUSTER command and clusterdb (Jim Cox)
•
Decrease memory requirements for recording pending trigger events (Tom)
E.116.3.5.1. Indexes •
Dramatically improve the speed of building and accessing hash indexes (Tom Raney, Shreya Bhargava) This allows hash indexes to be sometimes faster than btree indexes. However, hash indexes are still not crash-safe.
2348
Appendix E. Release Notes •
Make hash indexes store only the hash code, not the full value of the indexed column (Xiao Meng) This greatly reduces the size of hash indexes for long indexed values, improving performance.
•
Implement fast update option for GIN indexes (Teodor, Oleg) This option greatly improves update speed at a small penalty in search speed.
• xxx_pattern_ops
indexes can now be used for simple equality comparisons, not only for LIKE
(Tom)
E.116.3.5.2. Full Text Indexes •
Remove the requirement to use @@@ when doing GIN weighted lookups on full text indexes (Tom, Teodor) The normal @@ text search operator can be used instead.
•
Add an optimizer selectivity function for @@ text search operations (Jan Urbanski)
•
Allow prefix matching in full text searches (Teodor Sigaev, Oleg Bartunov)
•
Support multi-column GIN indexes (Teodor Sigaev)
•
Improve support for Nepali language and Devanagari alphabet (Teodor)
E.116.3.5.3. VACUUM •
Track free space in separate per-relation “fork” files (Heikki) Free space discovered by VACUUM is now recorded in *_fsm files, rather than in a fixed-sized shared memory area. The max_fsm_pages and max_fsm_relations settings have been removed, greatly simplifying administration of free space management.
•
Add a visibility map to track pages that do not require vacuuming (Heikki) This allows VACUUM to avoid scanning all of a table when only a portion of the table needs vacuuming. The visibility map is stored in per-relation “fork” files.
•
Add vacuum_freeze_table_age parameter to control when VACUUM should ignore the visibility map and do a full table scan to freeze tuples (Heikki)
•
Track transaction snapshots more carefully (Alvaro) This improves VACUUM’s ability to reclaim space in the presence of long-running transactions.
•
Add ability to specify per-relation autovacuum and TOAST parameters in CREATE TABLE (Alvaro, Euler Taveira de Oliveira) Autovacuum options used to be stored in a system table.
•
Add --freeze option to vacuumdb (Bruce)
E.116.3.6. Data Types •
Add a CaseSensitive option for text search synonym dictionaries (Simon)
•
Improve the precision of NUMERIC division (Tom)
2349
Appendix E. Release Notes •
Add basic arithmetic operators for int2 with int8 (Tom) This eliminates the need for explicit casting in some situations.
•
Allow UUID input to accept an optional hyphen after every fourth digit (Robert Haas)
•
Allow on/off as input for the boolean data type (Itagaki Takahiro)
•
Allow spaces around NaN in the input string for type numeric (Sam Mason)
E.116.3.6.1. Temporal Data Types •
Reject year 0 BC and years 000 and 0000 (Tom) Previously these were interpreted as 1 BC. (Note: years 0 and 00 are still assumed to be the year 2000.)
•
Include SGT (Singapore time) in the default list of known time zone abbreviations (Tom)
•
Support infinity and -infinity as values of type date (Tom)
•
Make parsing of interval literals more standard-compliant (Tom, Ron Mayer) For example, INTERVAL ’1’ YEAR now does what it’s supposed to.
•
Allow interval fractional-seconds precision to be specified after the second keyword, for SQL standard compliance (Tom) Formerly the precision had to be specified after the keyword interval. (For backwards compatibility, this syntax is still supported, though deprecated.) Data type definitions will now be output using the standard format.
•
Support the IS0 8601 interval syntax (Ron Mayer, Kevin Grittner) For example, INTERVAL ’P1Y2M3DT4H5M6.7S’ is now supported.
•
Add IntervalStyle parameter which controls how interval values are output (Ron Mayer) Valid values are: postgres, postgres_verbose, sql_standard, iso_8601. This setting also controls the handling of negative interval input when only some fields have positive/negative designations.
•
Improve consistency of handling of fractional seconds in timestamp and interval output (Ron Mayer)
E.116.3.6.2. Arrays •
Improve the handling of casts applied to ARRAY[] constructs, such as ARRAY[...]::integer[] (Brendan Jurd) Formerly PostgreSQL attempted to determine a data type for the ARRAY[] construct without reference to the ensuing cast. This could fail unnecessarily in many cases, in particular when the ARRAY[] construct was empty or contained only ambiguous entries such as NULL. Now the cast is consulted to determine the type that the array elements must be.
•
Make SQL-syntax ARRAY dimensions optional to match the SQL standard (Peter)
•
Add array_ndims() to return the number of dimensions of an array (Robert Haas)
•
Add array_length() to return the length of an array for a specified dimension (Jim Nasby, Robert Haas, Peter Eisentraut)
2350
Appendix E. Release Notes •
Add aggregate function array_agg(), which returns all aggregated values as a single array (Robert Haas, Jeff Davis, Peter)
•
Add unnest(), which converts an array to individual row values (Tom) This is the opposite of array_agg().
•
Add array_fill() to create arrays initialized with a value (Pavel Stehule)
•
Add generate_subscripts() to simplify generating the range of an array’s subscripts (Pavel Stehule)
E.116.3.6.3. Wide-Value Storage (TOAST ) •
Consider TOAST compression on values as short as 32 bytes (previously 256 bytes) (Greg Stark)
•
Require 25% minimum space savings before using TOAST compression (previously 20% for small values and any-savings-at-all for large values) (Greg)
•
Improve TOAST heuristics for rows that have a mix of large and small toastable fields, so that we prefer to push large values out of line and don’t compress small values unnecessarily (Greg, Tom)
E.116.3.7. Functions •
Document that setseed() allows values from -1 to 1 (not just 0 to 1), and enforce the valid range (Kris Jurka)
•
Add server-side function lo_import(filename, oid) (Tatsuo)
•
Add quote_nullable(), which behaves like quote_literal() but returns the string NULL for a null argument (Brendan Jurd)
•
Improve full text search headline() function to allow extracting several fragments of text (Sushant Sinha)
•
Add suppress_redundant_updates_trigger() trigger function to avoid overhead for nondata-changing updates (Andrew)
•
Add div(numeric, numeric) to perform numeric division without rounding (Tom)
•
Add timestamp and timestamptz versions of generate_series() (Hitoshi Harada)
E.116.3.7.1. Object Information Functions •
Implement current_query() for use by functions that need to know the currently running query (Tomas Doran)
•
Add pg_get_keywords() to return a list of the parser keywords (Dave Page)
•
Add pg_get_functiondef() to see a function’s definition (Abhijit Menon-Sen)
•
Allow the second argument of pg_get_expr() to be zero when deparsing an expression that does not contain variables (Tom)
•
Modify pg_relation_size() to use regclass (Heikki) pg_relation_size(data_type_name) no longer works.
•
Add boot_val and reset_val columns to pg_settings output (Greg Smith)
2351
Appendix E. Release Notes •
Add source file name and line number columns to pg_settings output for variables set in a configuration file (Magnus, Alvaro) For security reasons, these columns are only visible to superusers.
•
Add support for CURRENT_CATALOG, CURRENT_SCHEMA, SET CATALOG, SET SCHEMA (Peter) These provide SQL-standard syntax for existing features.
•
Add pg_typeof() which returns the data type of any value (Brendan Jurd)
•
Make version() return information about whether the server is a 32- or 64-bit binary (Bruce)
•
Fix the behavior of information schema columns is_insertable_into and is_updatable to be consistent (Peter)
•
Improve the behavior of information schema datetime_precision columns (Peter) These columns now show zero for date columns, and 6 (the default precision) for time, timestamp, and interval without a declared precision, rather than showing null as formerly.
•
Convert remaining builtin set-returning functions to use OUT parameters (Jaime Casanova) This makes it possible to call these functions without specifying a column list: pg_show_all_settings(), pg_lock_status(), pg_prepared_xact(), pg_prepared_statement(), pg_cursor()
•
Make pg_*_is_visible() and has_*_privilege() functions return NULL for invalid OIDs, rather than reporting an error (Tom)
•
Extend has_*_privilege() functions to allow inquiring about the OR of multiple privileges in one call (Stephen Frost, Tom)
•
Add has_column_privilege() and has_any_column_privilege() functions (Stephen Frost, Tom)
E.116.3.7.2. Function Creation •
Support variadic functions (functions with a variable number of arguments) (Pavel Stehule) Only trailing arguments can be optional, and they all must be of the same data type.
•
Support default values for function arguments (Pavel Stehule)
•
Add CREATE FUNCTION ... RETURNS TABLE clause (Pavel Stehule)
•
Allow SQL-language functions to return the output of an INSERT/UPDATE/DELETE RETURNING clause (Tom)
E.116.3.7.3. PL/pgSQL Server-Side Language •
Support EXECUTE USING for easier insertion of data values into a dynamic query string (Pavel Stehule)
•
Allow looping over the results of a cursor using a FOR loop (Pavel Stehule)
•
Support RETURN QUERY EXECUTE (Pavel Stehule)
•
Improve the RAISE command (Pavel Stehule) •
Support DETAIL and HINT fields
•
Support specification of the SQLSTATE error code
2352
Appendix E. Release Notes
•
•
Support an exception name parameter
•
Allow RAISE without parameters in an exception block to re-throw the current error
Allow specification of SQLSTATE codes in EXCEPTION lists (Pavel Stehule) This is useful for handling custom SQLSTATE codes.
•
Support the CASE statement (Pavel Stehule)
•
Make RETURN QUERY set the special FOUND and GET DIAGNOSTICS ROW_COUNT variables (Pavel Stehule)
•
Make FETCH and MOVE set the GET DIAGNOSTICS ROW_COUNT variable (Andrew Gierth)
•
Make EXIT without a label always exit the innermost loop (Tom) Formerly, if there were a BEGIN block more closely nested than any loop, it would exit that block instead. The new behavior matches Oracle(TM) and is also what was previously stated by our own documentation.
•
Make processing of string literals and nested block comments match the main SQL parser’s processing (Tom) In particular, the format string in RAISE now works the same as any other string literal, including being subject to standard_conforming_strings. This change also fixes other cases in which valid commands would fail when standard_conforming_strings is on.
•
Avoid memory leakage when the same function is called at varying exception-block nesting depths (Tom)
E.116.3.8. Client Applications •
Fix pg_ctl restart to preserve command-line arguments (Bruce)
•
Add -w/--no-password option that prevents password prompting in all utilities that have a -W/--password option (Peter)
•
Remove -q (quiet) option of createdb, createuser, dropdb, dropuser (Peter) These options have had no effect since PostgreSQL 8.3.
E.116.3.8.1. psql •
Remove verbose startup banner; now just suggest help (Joshua Drake)
•
Make help show common backslash commands (Greg Sabino Mullane)
•
Add \pset format wrapped mode to wrap output to the screen width, or file/pipe output too if \pset columns is set (Bryce Nesbitt)
•
Allow all supported spellings of boolean values in \pset, rather than just on and off (Bruce) Formerly, any string other than “off” was silently taken to mean true. psql will now complain about unrecognized spellings (but still take them as true).
•
Use the pager for wide output (Bruce)
•
Require a space between a one-letter backslash command and its first argument (Bernd Helmle) This removes a historical source of ambiguity.
2353
Appendix E. Release Notes •
Improve tab completion support for schema-qualified and quoted identifiers (Greg Sabino Mullane)
•
Add optional on/off argument for \timing (David Fetter)
•
Display access control rights on multiple lines (Brendan Jurd, Andreas Scherbaum)
•
Make \l show database access privileges (Andrew Gilligan)
•
Make \l+ show database sizes, if permissions allow (Andrew Gilligan)
•
Add the \ef command to edit function definitions (Abhijit Menon-Sen)
E.116.3.8.2. psql \d* commands •
Make \d* commands that do not have a pattern argument show system objects only if the S modifier is specified (Greg Sabino Mullane, Bruce) The former behavior was inconsistent across different variants of \d, and in most cases it provided no easy way to see just user objects.
•
Improve \d* commands to work with older PostgreSQL server versions (back to 7.4), not only the current server version (Guillaume Lelarge)
•
Make \d show foreign-key constraints that reference the selected table (Kenneth D’Souza)
•
Make \d on a sequence show its column values (Euler Taveira de Oliveira)
•
Add column storage type and other relation options to the \d+ display (Gregory Stark, Euler Taveira de Oliveira)
•
Show relation size in \dt+ output (Dickson S. Guedes)
•
Show the possible values of enum types in \dT+ (David Fetter)
•
Allow \dC to accept a wildcard pattern, which matches either datatype involved in the cast (Tom)
•
Add a function type column to \df’s output, and add options to list only selected types of functions (David Fetter)
•
Make \df not hide functions that take or return type cstring (Tom) Previously, such functions were hidden because most of them are datatype I/O functions, which were deemed uninteresting. The new policy about hiding system functions by default makes this wart unnecessary.
E.116.3.8.3. pg_dump •
Add a --no-tablespaces option to pg_dump/pg_dumpall/pg_restore so that dumps can be restored to clusters that have non-matching tablespace layouts (Gavin Roy)
•
Remove -d and -D options from pg_dump and pg_dumpall (Tom) These options were too frequently confused with the option to select a database name in other PostgreSQL client applications. The functionality is still available, but you must now spell out the long option name --inserts or --column-inserts.
•
Remove -i/--ignore-version option from pg_dump and pg_dumpall (Tom) Use of this option does not throw an error, but it has no effect. This option was removed because the version checks are necessary for safety.
•
Disable statement_timeout during dump and restore (Joshua Drake)
2354
Appendix E. Release Notes •
Add pg_dump/pg_dumpall option --lock-wait-timeout (David Gould) This allows dumps to fail if unable to acquire a shared lock within the specified amount of time.
•
Reorder pg_dump --data-only output to dump tables referenced by foreign keys before the referencing tables (Tom) This allows data loads when foreign keys are already present. If circular references make a safe ordering impossible, a NOTICE is issued.
•
Allow pg_dump, pg_dumpall, and pg_restore to use a specified role (Benedek László)
•
Allow pg_restore to use multiple concurrent connections to do the restore (Andrew) The number of concurrent connections is controlled by the option --jobs. This is supported only for custom-format archives.
E.116.3.9. Programming Tools E.116.3.9.1. libpq •
Allow the OID to be specified when importing a large object, via new function lo_import_with_oid() (Tatsuo)
•
Add “events” support (Andrew Chernow, Merlin Moncure) This adds the ability to register callbacks to manage private data associated with PGconn and PGresult objects.
•
Improve error handling to allow the return of multiple error messages as multi-line error reports (Magnus)
•
Make PQexecParams() and related functions return PGRES_EMPTY_QUERY for an empty query (Tom) They previously returned PGRES_COMMAND_OK.
•
Document how to avoid the overhead of WSACleanup() on Windows (Andrew Chernow)
•
Do not rely on Kerberos tickets to determine the default database username (Magnus) Previously, a Kerberos-capable build of libpq would use the principal name from any available Kerberos ticket as default database username, even if the connection wasn’t using Kerberos authentication. This was deemed inconsistent and confusing. The default username is now determined the same way with or without Kerberos. Note however that the database username must still match the ticket when Kerberos authentication is used.
E.116.3.9.2. libpq SSL (Secure Sockets Layer) support •
Fix certificate validation for SSL connections (Magnus) libpq now supports verifying both the certificate and the name of the server when making SSL connections. If a root certificate is not available to use for verification, SSL connections will fail. The sslmode parameter is used to enable certificate verification and set the level of checking. The default is still not to do any verification, allowing connections to SSL-enabled servers without requiring a root certificate on the client.
•
Support wildcard server certificates (Magnus)
2355
Appendix E. Release Notes If a certificate CN starts with *, it will be treated as a wildcard when matching the hostname, allowing the use of the same certificate for multiple servers. •
Allow the file locations for client certificates to be specified (Mark Woodward, Alvaro, Magnus)
•
Add a PQinitOpenSSL function to allow greater control over OpenSSL/libcrypto initialization (Andrew Chernow)
•
Make libpq unregister its OpenSSL callbacks when no database connections remain open (Bruce, Magnus, Russell Smith) This is required for applications that unload the libpq library, otherwise invalid OpenSSL callbacks will remain.
E.116.3.9.3. ecpg •
Add localization support for messages (Euler Taveira de Oliveira)
•
ecpg parser is now automatically generated from the server parser (Michael) Previously the ecpg parser was hand-maintained.
E.116.3.9.4. Server Programming Interface (SPI) •
Add support for single-use plans with out-of-line parameters (Tom)
•
Add new SPI_OK_REWRITTEN return code for SPI_execute() (Heikki) This is used when a command is rewritten to another type of command.
•
Remove unnecessary inclusions from executor/spi.h (Tom) SPI-using modules might need to add some #include lines if they were depending on spi.h to include things for them.
E.116.3.10. Build Options •
Update build system to use Autoconf 2.61 (Peter)
•
Require GNU bison for source code builds (Peter) This has effectively been required for several years, but now there is no infrastructure claiming to support other parser tools.
•
Add pg_config --htmldir option (Peter)
•
Pass float4 by value inside the server (Zoltan Boszormenyi) Add configure option --disable-float4-byval to use the old behavior. External C functions that use old-style (version 0) call convention and pass or return float4 values will be broken by this change, so you may need the configure option if you have such functions and don’t want to update them.
•
Pass float8, int8, and related datatypes by value inside the server on 64-bit platforms (Zoltan Boszormenyi) Add configure option --disable-float8-byval to use the old behavior. As above, this change might break old-style external C functions.
2356
Appendix E. Release Notes •
Add configure options --with-segsize, --with-blocksize, --with-wal-blocksize, --with-wal-segsize (Zdenek Kotala, Tom) This simplifies build-time control over several constants that previously could only be changed by editing pg_config_manual.h.
•
Allow threaded builds on Solaris 2.5 (Bruce)
•
Use the system’s getopt_long() on Solaris (Zdenek Kotala, Tom) This makes option processing more consistent with what Solaris users expect.
•
Add support for the Sun Studio compiler on Linux (Julius Stroffek)
•
Append the major version number to the backend gettext domain, and the soname major version number to libraries’ gettext domain (Peter) This simplifies parallel installations of multiple versions.
•
Add support for code coverage testing with gcov (Michelle Caisse)
•
Allow out-of-tree builds on Mingw and Cygwin (Richard Evans)
•
Fix the use of Mingw as a cross-compiling source platform (Peter)
E.116.3.11. Source Code •
Support 64-bit time zone data files (Heikki) This adds support for daylight saving time (DST) calculations beyond the year 2038.
•
Deprecate use of platform’s time_t data type (Tom) Some platforms have migrated to 64-bit time_t, some have not, and Windows can’t make up its mind what it’s doing. Define pg_time_t to have the same meaning as time_t, but always be 64 bits (unless the platform has no 64-bit integer type), and use that type in all module APIs and on-disk data formats.
•
Fix bug in handling of the time zone database when cross-compiling (Richard Evans)
•
Link backend object files in one step, rather than in stages (Peter)
•
Improve gettext support to allow better translation of plurals (Peter)
•
Add message translation support to the PL languages (Alvaro, Peter)
•
Add more DTrace probes (Robert Lor)
•
Enable DTrace support on Mac OS X Leopard and other non-Solaris platforms (Robert Lor)
•
Simplify and standardize conversions between C strings and text datums, by providing common functions for the purpose (Brendan Jurd, Tom)
•
Clean up the include/catalog/ header files so that frontend programs can include them without including postgres.h (Zdenek Kotala)
•
Make name char-aligned, and suppress zero-padding of name entries in indexes (Tom)
•
Recover better if dynamically-loaded code executes exit() (Tom)
•
Add a hook to let plug-ins monitor the executor (Itagaki Takahiro)
•
Add a hook to allow the planner’s statistics lookup behavior to be overridden (Simon Riggs)
•
Add shmem_startup_hook() for custom shared memory requirements (Tom)
2357
Appendix E. Release Notes •
Replace the index access method amgetmulti entry point with amgetbitmap, and extend the API for amgettuple to support run-time determination of operator lossiness (Heikki, Tom, Teodor) The API for GIN and GiST opclass consistent functions has been extended as well.
•
Add support for partial-match searches in GIN indexes (Teodor Sigaev, Oleg Bartunov)
•
Replace pg_class column reltriggers with boolean relhastriggers (Simon) Also remove unused pg_class columns relukeys, relfkeys, and relrefs.
•
Add a relistemp column to pg_class to ease identification of temporary tables (Tom)
•
Move platform FAQs into the main documentation (Peter)
•
Prevent parser input files from being built with any conflicts (Peter)
•
Add support for the KOI8U (Ukrainian) encoding (Peter)
•
Add Japanese message translations (Japan PostgreSQL Users Group) This used to be maintained as a separate project.
•
Fix problem when setting LC_MESSAGES on MSVC-built systems (Hiroshi Inoue, Hiroshi Saito, Magnus)
E.116.3.12. Contrib •
Add contrib/auto_explain to automatically run EXPLAIN on queries exceeding a specified duration (Itagaki Takahiro, Tom)
•
Add contrib/btree_gin to allow GIN indexes to handle more datatypes (Oleg, Teodor)
•
Add contrib/citext to provide a case-insensitive, multibyte-aware text data type (David Wheeler)
•
Add contrib/pg_stat_statements for server-wide tracking of statement execution statistics (Itagaki Takahiro)
•
Add duration and query mode options to contrib/pgbench (Itagaki Takahiro)
•
Make contrib/pgbench use table names pgbench_accounts, pgbench_branches, pgbench_history, and pgbench_tellers, rather than just accounts, branches, history, and tellers (Tom) This is to reduce the risk of accidentally destroying real data by running pgbench.
•
Fix contrib/pgstattuple to handle tables and indexes with over 2 billion pages (Tatsuhito Kasahara)
•
In contrib/fuzzystrmatch, add a version of the Levenshtein string-distance function that allows the user to specify the costs of insertion, deletion, and substitution (Volkan Yazici)
•
Make contrib/ltree support multibyte encodings (laser)
•
Enable contrib/dblink to use connection information stored in the SQL/MED catalogs (Joe Conway)
•
Improve contrib/dblink’s reporting of errors from the remote server (Joe Conway)
•
Make contrib/dblink set client_encoding to match the local database’s encoding (Joe Conway) This prevents encoding problems when communicating with a remote database that uses a different encoding.
2358
Appendix E. Release Notes •
Make sure contrib/dblink uses a password supplied by the user, and not accidentally taken from the server’s .pgpass file (Joe Conway) This is a minor security enhancement.
•
Add fsm_page_contents() to contrib/pageinspect (Heikki)
•
Modify get_raw_page() to support free space map (*_fsm) files. Also update contrib/pg_freespacemap.
•
Add support for multibyte encodings to contrib/pg_trgm (Teodor)
•
Rewrite contrib/intagg to use new functions array_agg() and unnest() (Tom)
•
Make contrib/pg_standby recover all available WAL before failover (Fujii Masao, Simon, Heikki) To make this work safely, you now need to set the new recovery_end_command option in recovery.conf to clean up the trigger file after failover. pg_standby will no longer remove the trigger file itself.
• contrib/pg_standby’s -l
option is now a no-op, because it is unsafe to use a symlink (Simon)
E.117. Release 8.3.23 Release date: 2013-02-07 This release contains a variety of fixes from 8.3.22. For information about new features in the 8.3 major release, see Section E.140. This is expected to be the last PostgreSQL release in the 8.3.X series. Users are encouraged to update to a newer release branch soon.
E.117.1. Migration to Version 8.3.23 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.117.2. Changes •
Prevent execution of enum_recv from SQL (Tom Lane) The function was misdeclared, allowing a simple SQL command to crash the server. In principle an attacker might be able to use it to examine the contents of server memory. Our thanks to Sumit Soni (via Secunia SVCRP) for reporting this issue. (CVE-2013-0255)
•
Fix SQL grammar to allow subscripting or field selection from a sub-SELECT result (Tom Lane)
•
Protect against race conditions when scanning pg_tablespace (Stephen Frost, Tom Lane) CREATE DATABASE and DROP DATABASE could misbehave if there were concurrent updates of pg_tablespace entries.
•
Prevent DROP OWNED from trying to drop whole databases or tablespaces (Álvaro Herrera)
2359
Appendix E. Release Notes For safety, ownership of these objects must be reassigned, not dropped. •
Prevent misbehavior when a RowExpr or XmlExpr is parse-analyzed twice (Andres Freund, Tom Lane) This mistake could be user-visible in contexts such as CREATE TABLE LIKE INCLUDING INDEXES.
•
Improve defenses against integer overflow in hashtable sizing calculations (Jeff Davis)
•
Ensure that non-ASCII prompt strings are translated to the correct code page on Windows (Alexander Law, Noah Misch) This bug affected psql and some other client programs.
•
Fix possible crash in psql’s \? command when not connected to a database (Meng Qingzhong)
•
Fix one-byte buffer overrun in libpq’s PQprintTuples (Xi Wang) This ancient function is not used anywhere by PostgreSQL itself, but it might still be used by some client code.
•
Rearrange configure’s tests for supplied functions so it is not fooled by bogus exports from libedit/libreadline (Christoph Berg)
•
Ensure Windows build number increases over time (Magnus Hagander)
•
Make pgxs build executables with the right .exe suffix when cross-compiling for Windows (Zoltan Boszormenyi)
•
Add new timezone abbreviation FET (Tom Lane) This is now used in some eastern-European time zones.
E.118. Release 8.3.22 Release date: 2012-12-06 This release contains a variety of fixes from 8.3.21. For information about new features in the 8.3 major release, see Section E.140. The PostgreSQL community will stop releasing updates for the 8.3.X release series in February 2013. Users are encouraged to update to a newer release branch soon.
E.118.1. Migration to Version 8.3.22 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.118.2. Changes •
Fix multiple bugs associated with CREATE INDEX CONCURRENTLY (Andres Freund, Tom Lane) Fix CREATE INDEX CONCURRENTLY to use in-place updates when changing the state of an index’s pg_index row. This prevents race conditions that could cause concurrent sessions to miss updating the target index, thus resulting in corrupt concurrently-created indexes.
2360
Appendix E. Release Notes Also, fix various other operations to ensure that they ignore invalid indexes resulting from a failed CREATE INDEX CONCURRENTLY command. The most important of these is VACUUM, because an auto-vacuum could easily be launched on the table before corrective action can be taken to fix or remove the invalid index. •
Avoid corruption of internal hash tables when out of memory (Hitoshi Harada)
•
Fix planning of non-strict equivalence clauses above outer joins (Tom Lane) The planner could derive incorrect constraints from a clause equating a non-strict construct to something else, for example WHERE COALESCE(foo, 0) = 0 when foo is coming from the nullable side of an outer join.
•
Improve planner’s ability to prove exclusion constraints from equivalence classes (Tom Lane)
•
Fix partial-row matching in hashed subplans to handle cross-type cases correctly (Tom Lane) This affects multicolumn NOT IN subplans, such as WHERE (a, b) NOT IN (SELECT x, y FROM ...) when for instance b and y are int4 and int8 respectively. This mistake led to wrong answers or crashes depending on the specific datatypes involved.
•
Acquire buffer lock when re-fetching the old tuple for an AFTER ROW UPDATE/DELETE trigger (Andres Freund) In very unusual circumstances, this oversight could result in passing incorrect data to the precheck logic for a foreign-key enforcement trigger. That could result in a crash, or in an incorrect decision about whether to fire the trigger.
•
Fix REASSIGN OWNED to handle grants on tablespaces (Álvaro Herrera)
•
Ignore incorrect pg_attribute entries for system columns for views (Tom Lane) Views do not have any system columns. However, we forgot to remove such entries when converting a table to a view. That’s fixed properly for 9.3 and later, but in previous branches we need to defend against existing mis-converted views.
•
Fix rule printing to dump INSERT INTO table DEFAULT VALUES correctly (Tom Lane)
•
Guard against stack overflow when there are too many UNION/INTERSECT/EXCEPT clauses in a query (Tom Lane)
•
Prevent platform-dependent failures when dividing the minimum possible integer value by -1 (Xi Wang, Tom Lane)
•
Fix possible access past end of string in date parsing (Hitoshi Harada)
•
Produce an understandable error message if the length of the path name for a Unix-domain socket exceeds the platform-specific limit (Tom Lane, Andrew Dunstan) Formerly, this would result in something quite unhelpful, such as “Non-recoverable failure in name resolution”.
•
Fix memory leaks when sending composite column values to the client (Tom Lane)
•
Make pg_ctl more robust about reading the postmaster.pid file (Heikki Linnakangas) Fix race conditions and possible file descriptor leakage.
•
Fix possible crash in psql if incorrectly-encoded data is presented and the client_encoding setting is a client-only encoding, such as SJIS (Jiang Guiqing)
•
Fix bugs in the restore.sql script emitted by pg_dump in tar output format (Tom Lane) The script would fail outright on tables whose names include upper-case characters. Also, make the script capable of restoring data in --inserts mode as well as the regular COPY mode.
2361
Appendix E. Release Notes •
Fix pg_restore to accept POSIX-conformant tar files (Brian Weaver, Tom Lane) The original coding of pg_dump’s tar output mode produced files that are not fully conformant with the POSIX standard. This has been corrected for version 9.3. This patch updates previous branches so that they will accept both the incorrect and the corrected formats, in hopes of avoiding compatibility problems when 9.3 comes out.
•
Fix pg_resetxlog to locate postmaster.pid correctly when given a relative path to the data directory (Tom Lane) This mistake could lead to pg_resetxlog not noticing that there is an active postmaster using the data directory.
•
Fix libpq’s lo_import() and lo_export() functions to report file I/O errors properly (Tom Lane)
•
Fix ecpg’s processing of nested structure pointer variables (Muhammad Usama)
•
Make contrib/pageinspect’s btree page inspection functions take buffer locks while examining pages (Tom Lane)
•
Fix pgxs support for building loadable modules on AIX (Tom Lane) Building modules outside the original source tree didn’t work on AIX.
•
Update time zone data files to tzdata release 2012j for DST law changes in Cuba, Israel, Jordan, Libya, Palestine, Western Samoa, and portions of Brazil.
E.119. Release 8.3.21 Release date: 2012-09-24 This release contains a variety of fixes from 8.3.20. For information about new features in the 8.3 major release, see Section E.140. The PostgreSQL community will stop releasing updates for the 8.3.X release series in February 2013. Users are encouraged to update to a newer release branch soon.
E.119.1. Migration to Version 8.3.21 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.119.2. Changes •
Improve page-splitting decisions in GiST indexes (Alexander Korotkov, Robert Haas, Tom Lane) Multi-column GiST indexes might suffer unexpected bloat due to this error.
•
Fix cascading privilege revoke to stop if privileges are still held (Tom Lane) If we revoke a grant option from some role X , but X still holds that option via a grant from someone else, we should not recursively revoke the corresponding privilege from role(s) Y that X had granted it to.
2362
Appendix E. Release Notes •
Fix handling of SIGFPE when PL/Perl is in use (Andres Freund) Perl resets the process’s SIGFPE handler to SIG_IGN, which could result in crashes later on. Restore the normal Postgres signal handler after initializing PL/Perl.
•
Prevent PL/Perl from crashing if a recursive PL/Perl function is redefined while being executed (Tom Lane)
•
Work around possible misoptimization in PL/Perl (Tom Lane) Some Linux distributions contain an incorrect version of pthread.h that results in incorrect compiled code in PL/Perl, leading to crashes if a PL/Perl function calls another one that throws an error.
•
Update time zone data files to tzdata release 2012f for DST law changes in Fiji
E.120. Release 8.3.20 Release date: 2012-08-17 This release contains a variety of fixes from 8.3.19. For information about new features in the 8.3 major release, see Section E.140. The PostgreSQL community will stop releasing updates for the 8.3.X release series in February 2013. Users are encouraged to update to a newer release branch soon.
E.120.1. Migration to Version 8.3.20 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.120.2. Changes •
Prevent access to external files/URLs via XML entity references (Noah Misch, Tom Lane) xml_parse() would attempt to fetch external files or URLs as needed to resolve DTD and entity
references in an XML value, thus allowing unprivileged database users to attempt to fetch data with the privileges of the database server. While the external data wouldn’t get returned directly to the user, portions of it could be exposed in error messages if the data didn’t parse as valid XML; and in any case the mere ability to check existence of a file might be useful to an attacker. (CVE-20123489) •
Prevent access to external files/URLs via contrib/xml2’s xslt_process() (Peter Eisentraut) libxslt offers the ability to read and write both files and URLs through stylesheet commands, thus allowing unprivileged database users to both read and write data with the privileges of the database server. Disable that through proper use of libxslt’s security options. (CVE-2012-3488) Also, remove xslt_process()’s ability to fetch documents and stylesheets from external files/URLs. While this was a documented “feature”, it was long regarded as a bad idea. The fix for CVE-2012-3489 broke that capability, and rather than expend effort on trying to fix it, we’re just going to summarily remove it.
•
Prevent too-early recycling of btree index pages (Noah Misch)
2363
Appendix E. Release Notes When we allowed read-only transactions to skip assigning XIDs, we introduced the possibility that a deleted btree page could be recycled while a read-only transaction was still in flight to it. This would result in incorrect index search results. The probability of such an error occurring in the field seems very low because of the timing requirements, but nonetheless it should be fixed. •
Fix crash-safety bug with newly-created-or-reset sequences (Tom Lane) If ALTER SEQUENCE was executed on a freshly created or reset sequence, and then precisely one nextval() call was made on it, and then the server crashed, WAL replay would restore the sequence to a state in which it appeared that no nextval() had been done, thus allowing the first sequence value to be returned again by the next nextval() call. In particular this could manifest for serial columns, since creation of a serial column’s sequence includes an ALTER SEQUENCE OWNED BY step.
•
Ensure the backup_label file is fsync’d after pg_start_backup() (Dave Kerr)
•
Back-patch 9.1 improvement to compress the fsync request queue (Robert Haas) This improves performance during checkpoints. The 9.1 change has now seen enough field testing to seem safe to back-patch.
•
Only allow autovacuum to be auto-canceled by a directly blocked process (Tom Lane) The original coding could allow inconsistent behavior in some cases; in particular, an autovacuum could get canceled after less than deadlock_timeout grace period.
•
Improve logging of autovacuum cancels (Robert Haas)
•
Fix log collector so that log_truncate_on_rotation works during the very first log rotation after server start (Tom Lane)
•
Ensure that a whole-row reference to a subquery doesn’t include any extra GROUP BY or ORDER BY columns (Tom Lane)
•
Disallow copying whole-row references in CHECK constraints and index definitions during CREATE TABLE (Tom Lane) This situation can arise in CREATE TABLE with LIKE or INHERITS. The copied whole-row variable was incorrectly labeled with the row type of the original table not the new one. Rejecting the case seems reasonable for LIKE, since the row types might well diverge later. For INHERITS we should ideally allow it, with an implicit coercion to the parent table’s row type; but that will require more work than seems safe to back-patch.
•
Fix memory leak in ARRAY(SELECT ...) subqueries (Heikki Linnakangas, Tom Lane)
•
Fix extraction of common prefixes from regular expressions (Tom Lane) The code could get confused by quantified parenthesized subexpressions, such as ^(foo)?bar. This would lead to incorrect index optimization of searches for such patterns.
•
Report errors properly in contrib/xml2’s xslt_process() (Tom Lane)
•
Update time zone data files to tzdata release 2012e for DST law changes in Morocco and Tokelau
E.121. Release 8.3.19 Release date: 2012-06-04 This release contains a variety of fixes from 8.3.18. For information about new features in the 8.3 major release, see Section E.140.
2364
Appendix E. Release Notes
E.121.1. Migration to Version 8.3.19 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.121.2. Changes •
Fix incorrect password transformation in contrib/pgcrypto’s DES crypt() function (Solar Designer) If a password string contained the byte value 0x80, the remainder of the password was ignored, causing the password to be much weaker than it appeared. With this fix, the rest of the string is properly included in the DES hash. Any stored password values that are affected by this bug will thus no longer match, so the stored values may need to be updated. (CVE-2012-2143)
•
Ignore SECURITY DEFINER and SET attributes for a procedural language’s call handler (Tom Lane) Applying such attributes to a call handler could crash the server. (CVE-2012-2655)
•
Allow numeric timezone offsets in timestamp input to be up to 16 hours away from UTC (Tom Lane) Some historical time zones have offsets larger than 15 hours, the previous limit. This could result in dumped data values being rejected during reload.
•
Fix timestamp conversion to cope when the given time is exactly the last DST transition time for the current timezone (Tom Lane) This oversight has been there a long time, but was not noticed previously because most DST-using zones are presumed to have an indefinite sequence of future DST transitions.
•
Fix text to name and char to name casts to perform string truncation correctly in multibyte encodings (Karl Schnaitter)
•
Fix memory copying bug in to_tsquery() (Heikki Linnakangas)
•
Fix slow session startup when pg_attribute is very large (Tom Lane) If pg_attribute exceeds one-fourth of shared_buffers, cache rebuilding code that is sometimes needed during session start would trigger the synchronized-scan logic, causing it to take many times longer than normal. The problem was particularly acute if many new sessions were starting at once.
•
Ensure sequential scans check for query cancel reasonably often (Merlin Moncure) A scan encountering many consecutive pages that contain no live tuples would not respond to interrupts meanwhile.
•
Ensure the Windows implementation of PGSemaphoreLock() clears ImmediateInterruptOK before returning (Tom Lane) This oversight meant that a query-cancel interrupt received later in the same query could be accepted at an unsafe time, with unpredictable but not good consequences.
•
Show whole-row variables safely when printing views or rules (Abbas Butt, Tom Lane) Corner cases involving ambiguous names (that is, the name could be either a table or column name of the query) were printed in an ambiguous way, risking that the view or rule would be interpreted differently after dump and reload. Avoid the ambiguous case by attaching a no-op cast.
2365
Appendix E. Release Notes •
Ensure autovacuum worker processes perform stack depth checking properly (Heikki Linnakangas) Previously, infinite recursion in a function invoked by auto-ANALYZE could crash worker processes.
•
Fix logging collector to not lose log coherency under high load (Andrew Dunstan) The collector previously could fail to reassemble large messages if it got too busy.
•
Fix logging collector to ensure it will restart file rotation after receiving SIGHUP (Tom Lane)
•
Fix PL/pgSQL’s GET DIAGNOSTICS command when the target is the function’s first variable (Tom Lane)
•
Fix several performance problems in pg_dump when the database contains many objects (Jeff Janes, Tom Lane) pg_dump could get very slow if the database contained many schemas, or if many objects are in dependency loops, or if there are many owned sequences.
•
Fix contrib/dblink’s dblink_exec() to not leak temporary database connections upon error (Tom Lane)
•
Update time zone data files to tzdata release 2012c for DST law changes in Antarctica, Armenia, Chile, Cuba, Falkland Islands, Gaza, Haiti, Hebron, Morocco, Syria, and Tokelau Islands; also historical corrections for Canada.
E.122. Release 8.3.18 Release date: 2012-02-27 This release contains a variety of fixes from 8.3.17. For information about new features in the 8.3 major release, see Section E.140.
E.122.1. Migration to Version 8.3.18 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.17, see Section E.123.
E.122.2. Changes •
Require execute permission on the trigger function for CREATE TRIGGER (Robert Haas) This missing check could allow another user to execute a trigger function with forged input data, by installing it on a table he owns. This is only of significance for trigger functions marked SECURITY DEFINER, since otherwise trigger functions run as the table owner anyway. (CVE-2012-0866)
•
Convert newlines to spaces in names written in pg_dump comments (Robert Haas) pg_dump was incautious about sanitizing object names that are emitted within SQL comments in its output script. A name containing a newline would at least render the script syntactically incorrect. Maliciously crafted object names could present a SQL injection risk when the script is reloaded. (CVE-2012-0868)
•
Fix btree index corruption from insertions concurrent with vacuuming (Tom Lane)
2366
Appendix E. Release Notes An index page split caused by an insertion could sometimes cause a concurrently-running VACUUM to miss removing index entries that it should remove. After the corresponding table rows are removed, the dangling index entries would cause errors (such as “could not read block N in file ...”) or worse, silently wrong query results after unrelated rows are re-inserted at the now-free table locations. This bug has been present since release 8.2, but occurs so infrequently that it was not diagnosed until now. If you have reason to suspect that it has happened in your database, reindexing the affected index will fix things. •
Allow non-existent values for some settings in ALTER USER/DATABASE SET (Heikki Linnakangas) Allow default_text_search_config, default_tablespace, and temp_tablespaces to be set to names that are not known. This is because they might be known in another database where the setting is intended to be used, or for the tablespace cases because the tablespace might not be created yet. The same issue was previously recognized for search_path, and these settings now act like that one.
•
Track the OID counter correctly during WAL replay, even when it wraps around (Tom Lane) Previously the OID counter would remain stuck at a high value until the system exited replay mode. The practical consequences of that are usually nil, but there are scenarios wherein a standby server that’s been promoted to master might take a long time to advance the OID counter to a reasonable value once values are needed.
•
Fix regular expression back-references with * attached (Tom Lane) Rather than enforcing an exact string match, the code would effectively accept any string that satisfies the pattern sub-expression referenced by the back-reference symbol. A similar problem still afflicts back-references that are embedded in a larger quantified expression, rather than being the immediate subject of the quantifier. This will be addressed in a future PostgreSQL release.
•
Fix recently-introduced memory leak in processing of inet/cidr values (Heikki Linnakangas) A patch in the December 2011 releases of PostgreSQL caused memory leakage in these operations, which could be significant in scenarios such as building a btree index on such a column.
•
Avoid double close of file handle in syslogger on Windows (MauMau) Ordinarily this error was invisible, but it would cause an exception when running on a debug version of Windows.
•
Fix I/O-conversion-related memory leaks in plpgsql (Andres Freund, Jan Urbanski, Tom Lane) Certain operations would leak memory until the end of the current function.
•
Improve pg_dump’s handling of inherited table columns (Tom Lane) pg_dump mishandled situations where a child column has a different default expression than its parent column. If the default is textually identical to the parent’s default, but not actually the same (for instance, because of schema search path differences) it would not be recognized as different, so that after dump and restore the child would be allowed to inherit the parent’s default. Child columns that are NOT NULL where their parent is not could also be restored subtly incorrectly.
•
Fix pg_restore’s direct-to-database mode for INSERT-style table data (Tom Lane) Direct-to-database restores from archive files made with --inserts or --column-inserts options fail when using pg_restore from a release dated September or December 2011, as a result of an oversight in a fix for another problem. The archive file itself is not at fault, and text-mode output is okay.
•
Fix error in contrib/intarray’s int[] & int[] operator (Guillaume Lelarge)
2367
Appendix E. Release Notes If the smallest integer the two input arrays have in common is 1, and there are smaller values in either array, then 1 would be incorrectly omitted from the result. •
Fix error detection in contrib/pgcrypto’s encrypt_iv() and decrypt_iv() (Marko Kreen) These functions failed to report certain types of invalid-input errors, and would instead return random garbage values for incorrect input.
•
Fix one-byte buffer overrun in contrib/test_parser (Paul Guyot) The code would try to read one more byte than it should, which would crash in corner cases. Since contrib/test_parser is only example code, this is not a security issue in itself, but bad example code is still bad.
•
Use __sync_lock_test_and_set() for spinlocks on ARM, if available (Martin Pitt) This function replaces our previous use of the SWPB instruction, which is deprecated and not available on ARMv6 and later. Reports suggest that the old code doesn’t fail in an obvious way on recent ARM boards, but simply doesn’t interlock concurrent accesses, leading to bizarre failures in multiprocess operation.
•
Use -fexcess-precision=standard option when building with gcc versions that accept it (Andrew Dunstan) This prevents assorted scenarios wherein recent versions of gcc will produce creative results.
•
Allow use of threaded Python on FreeBSD (Chris Rees) Our configure script previously believed that this combination wouldn’t work; but FreeBSD fixed the problem, so remove that error check.
E.123. Release 8.3.17 Release date: 2011-12-05 This release contains a variety of fixes from 8.3.16. For information about new features in the 8.3 major release, see Section E.140.
E.123.1. Migration to Version 8.3.17 A dump/restore is not required for those running 8.3.X. However,
a
longstanding
error
was
discovered
in
the
definition
of
the
information_schema.referential_constraints view. If you rely on correct results from
that view, you should replace its definition as explained in the first changelog item below. Also, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.123.2. Changes •
Fix bugs in information_schema.referential_constraints view (Tom Lane) This view was being insufficiently careful about matching the foreign-key constraint to the depended-on primary or unique key constraint. That could result in failure to show a foreign key
2368
Appendix E. Release Notes constraint at all, or showing it multiple times, or claiming that it depends on a different constraint than the one it really does. Since the view definition is installed by initdb, merely upgrading will not fix the problem. If you need to fix this in an existing installation, you can (as a superuser) drop the information_schema schema then re-create it by sourcing SHAREDIR/information_schema.sql. (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) This must be repeated in each database to be fixed. •
Fix TOAST-related data corruption during CREATE TABLE dest AS SELECT * FROM src or INSERT INTO dest SELECT * FROM src (Tom Lane) If a table has been modified by ALTER TABLE ADD COLUMN, attempts to copy its data verbatim to another table could produce corrupt results in certain corner cases. The problem can only manifest in this precise form in 8.4 and later, but we patched earlier versions as well in case there are other code paths that could trigger the same bug.
•
Fix race condition during toast table access from stale syscache entries (Tom Lane) The typical symptom was transient errors like “missing chunk number 0 for toast value NNNNN in pg_toast_2619”, where the cited toast table would always belong to a system catalog.
•
Make DatumGetInetP() unpack inet datums that have a 1-byte header, and add a new macro, DatumGetInetPP(), that does not (Heikki Linnakangas) This change affects no core code, but might prevent crashes in add-on code that expects DatumGetInetP() to produce an unpacked datum as per usual convention.
•
Improve locale support in money type’s input and output (Tom Lane) Aside from not supporting all standard lc_monetary formatting options, the input and output functions were inconsistent, meaning there were locales in which dumped money values could not be re-read.
•
Don’t let transform_null_equals affect CASE foo WHEN NULL ... constructs (Heikki Linnakangas) transform_null_equals is only supposed to affect foo = NULL expressions written directly by the user, not equality checks generated internally by this form of CASE.
•
Change foreign-key trigger creation order to better support self-referential foreign keys (Tom Lane) For a cascading foreign key that references its own table, a row update will fire both the ON UPDATE trigger and the CHECK trigger as one event. The ON UPDATE trigger must execute first, else the CHECK will check a non-final state of the row and possibly throw an inappropriate error. However, the firing order of these triggers is determined by their names, which generally sort in creation order since the triggers have auto-generated names following the convention “RI_ConstraintTrigger_NNNN”. A proper fix would require modifying that convention, which we will do in 9.2, but it seems risky to change it in existing releases. So this patch just changes the creation order of the triggers. Users encountering this type of error should drop and re-create the foreign key constraint to get its triggers into the right order.
•
Avoid floating-point underflow while tracking buffer allocation rate (Greg Matthews) While harmless in itself, on certain platforms this would result in annoying kernel log messages.
•
Preserve blank lines within commands in psql’s command history (Robert Haas) The former behavior could cause problems if an empty line was removed from within a string literal, for example.
•
Fix pg_dump to dump user-defined casts between auto-generated types, such as table rowtypes (Tom Lane)
2369
Appendix E. Release Notes •
Use the preferred version of xsubpp to build PL/Perl, not necessarily the operating system’s main copy (David Wheeler and Alex Hunsaker)
•
Fix incorrect coding in contrib/dict_int and contrib/dict_xsyn (Tom Lane) Some functions incorrectly assumed that memory returned by palloc() is guaranteed zeroed.
•
Honor query cancel interrupts promptly in pgstatindex() (Robert Haas)
•
Ensure VPATH builds properly install all server header files (Peter Eisentraut)
•
Shorten file names reported in verbose error messages (Peter Eisentraut) Regular builds have always reported just the name of the C file containing the error message call, but VPATH builds formerly reported an absolute path name.
•
Fix interpretation of Windows timezone names for Central America (Tom Lane) Map “Central America Standard Time” to CST6, not CST6CDT, because DST is generally not observed anywhere in Central America.
•
Update time zone data files to tzdata release 2011n for DST law changes in Brazil, Cuba, Fiji, Palestine, Russia, and Samoa; also historical corrections for Alaska and British East Africa.
E.124. Release 8.3.16 Release date: 2011-09-26 This release contains a variety of fixes from 8.3.15. For information about new features in the 8.3 major release, see Section E.140.
E.124.1. Migration to Version 8.3.16 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.124.2. Changes •
Fix bugs in indexing of in-doubt HOT-updated tuples (Tom Lane) These bugs could result in index corruption after reindexing a system catalog. They are not believed to affect user indexes.
•
Fix multiple bugs in GiST index page split processing (Heikki Linnakangas) The probability of occurrence was low, but these could lead to index corruption.
•
Fix possible buffer overrun in tsvector_concat() (Tom Lane) The function could underestimate the amount of memory needed for its result, leading to server crashes.
•
Fix crash in xml_recv when processing a “standalone” parameter (Tom Lane)
•
Avoid possibly accessing off the end of memory in ANALYZE and in SJIS-2004 encoding conversion (Noah Misch) This fixes some very-low-probability server crash scenarios.
2370
Appendix E. Release Notes •
Fix race condition in relcache init file invalidation (Tom Lane) There was a window wherein a new backend process could read a stale init file but miss the inval messages that would tell it the data is stale. The result would be bizarre failures in catalog accesses, typically “could not read block 0 in file ...” later during startup.
•
Fix memory leak at end of a GiST index scan (Tom Lane) Commands that perform many separate GiST index scans, such as verification of a new GiSTbased exclusion constraint on a table already containing many rows, could transiently require large amounts of memory due to this leak.
•
Fix performance problem when constructing a large, lossy bitmap (Tom Lane)
•
Fix array- and path-creating functions to ensure padding bytes are zeroes (Tom Lane) This avoids some situations where the planner will think that semantically-equal constants are not equal, resulting in poor optimization.
•
Work around gcc 4.6.0 bug that breaks WAL replay (Tom Lane) This could lead to loss of committed transactions after a server crash.
•
Fix dump bug for VALUES in a view (Tom Lane)
•
Disallow SELECT FOR UPDATE/SHARE on sequences (Tom Lane) This operation doesn’t work as expected and can lead to failures.
•
Defend against integer overflow when computing size of a hash table (Tom Lane)
•
Fix cases where CLUSTER might attempt to access already-removed TOAST data (Tom Lane)
•
Fix portability bugs in use of credentials control messages for “peer” authentication (Tom Lane)
•
Fix SSPI login when multiple roundtrips are required (Ahmed Shinwari, Magnus Hagander) The typical symptom of this problem was “The function requested is not supported” errors during SSPI login.
•
Fix typo in pg_srand48 seed initialization (Andres Freund) This led to failure to use all bits of the provided seed. This function is not used on most platforms (only those without srandom), and the potential security exposure from a less-random-thanexpected seed seems minimal in any case.
•
Avoid integer overflow when the sum of LIMIT and OFFSET values exceeds 2^63 (Heikki Linnakangas)
•
Add overflow checks to int4 and int8 versions of generate_series() (Robert Haas)
•
Fix trailing-zero removal in to_char() (Marti Raudsepp) In a format with FM and no digit positions after the decimal point, zeroes to the left of the decimal point could be removed incorrectly.
•
Fix pg_size_pretty() to avoid overflow for inputs close to 2^63 (Tom Lane)
•
In pg_ctl, support silent mode for service registrations on Windows (MauMau)
•
Fix psql’s counting of script file line numbers during COPY from a different file (Tom Lane)
•
Fix pg_restore’s direct-to-database mode for standard_conforming_strings (Tom Lane) pg_restore could emit incorrect commands when restoring directly to a database server from an archive file that had been made with standard_conforming_strings set to on.
•
Fix write-past-buffer-end and memory leak in libpq’s LDAP service lookup code (Albe Laurenz)
2371
Appendix E. Release Notes •
In libpq, avoid failures when using nonblocking I/O and an SSL connection (Martin Pihlak, Tom Lane)
•
Improve libpq’s handling of failures during connection startup (Tom Lane) In particular, the response to a server report of fork() failure during SSL connection startup is now saner.
•
Improve libpq’s error reporting for SSL failures (Tom Lane)
•
Make ecpglib write double values with 15 digits precision (Akira Kurosawa)
•
In ecpglib, be sure LC_NUMERIC setting is restored after an error (Michael Meskes)
•
Apply upstream fix for blowfish signed-character bug (CVE-2011-2483) (Tom Lane) contrib/pg_crypto’s blowfish encryption code could give wrong results on platforms where
char is signed (which is most), leading to encrypted passwords being weaker than they should be. •
Fix memory leak in contrib/seg (Heikki Linnakangas)
•
Fix pgstatindex() to give consistent results for empty indexes (Tom Lane)
•
Allow building with perl 5.14 (Alex Hunsaker)
•
Update configure script’s method for probing existence of system functions (Tom Lane) The version of autoconf we used in 8.3 and 8.2 could be fooled by compilers that perform link-time optimization.
•
Fix assorted issues with build and install file paths containing spaces (Tom Lane)
•
Update time zone data files to tzdata release 2011i for DST law changes in Canada, Egypt, Russia, Samoa, and South Sudan.
E.125. Release 8.3.15 Release date: 2011-04-18 This release contains a variety of fixes from 8.3.14. For information about new features in the 8.3 major release, see Section E.140.
E.125.1. Migration to Version 8.3.15 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.125.2. Changes •
Disallow including a composite type in itself (Tom Lane) This prevents scenarios wherein the server could recurse infinitely while processing the composite type. While there are some possible uses for such a structure, they don’t seem compelling enough to justify the effort required to make sure it always works safely.
•
Avoid potential deadlock during catalog cache initialization (Nikhil Sontakke)
2372
Appendix E. Release Notes In some cases the cache loading code would acquire share lock on a system index before locking the index’s catalog. This could deadlock against processes trying to acquire exclusive locks in the other, more standard order. •
Fix dangling-pointer problem in BEFORE ROW UPDATE trigger handling when there was a concurrent update to the target tuple (Tom Lane) This bug has been observed to result in intermittent “cannot extract system attribute from virtual tuple” failures while trying to do UPDATE RETURNING ctid. There is a very small probability of more serious errors, such as generating incorrect index entries for the updated tuple.
•
Disallow DROP TABLE when there are pending deferred trigger events for the table (Tom Lane) Formerly the DROP would go through, leading to “could not open relation with OID nnn” errors when the triggers were eventually fired.
Fix pg_restore to cope with long lines (over 1KB) in TOC files (Tom Lane)
•
Put in more safeguards against crashing due to division-by-zero with overly enthusiastic compiler optimization (Aurelien Jarno)
•
Support use of dlopen() in FreeBSD and OpenBSD on MIPS (Tom Lane) There was a hard-wired assumption that this system function was not available on MIPS hardware on these systems. Use a compile-time test instead, since more recent versions have it.
•
Fix compilation failures on HP-UX (Heikki Linnakangas)
•
Fix version-incompatibility problem with libintl on Windows (Hiroshi Inoue)
•
Fix usage of xcopy in Windows build scripts to work correctly under Windows 7 (Andrew Dunstan) This affects the build scripts only, not installation or usage.
•
Fix path separator used by pg_regress on Cygwin (Andrew Dunstan)
•
Update time zone data files to tzdata release 2011f for DST law changes in Chile, Cuba, Falkland Islands, Morocco, Samoa, and Turkey; also historical corrections for South Australia, Alaska, and Hawaii.
E.126. Release 8.3.14 Release date: 2011-01-31 This release contains a variety of fixes from 8.3.13. For information about new features in the 8.3 major release, see Section E.140.
E.126.1. Migration to Version 8.3.14 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
2373
Appendix E. Release Notes
E.126.2. Changes •
Avoid failures when EXPLAIN tries to display a simple-form CASE expression (Tom Lane) If the CASE’s test expression was a constant, the planner could simplify the CASE into a form that confused the expression-display code, resulting in “unexpected CASE WHEN clause” errors.
•
Fix assignment to an array slice that is before the existing range of subscripts (Tom Lane) If there was a gap between the newly added subscripts and the first pre-existing subscript, the code miscalculated how many entries needed to be copied from the old array’s null bitmap, potentially leading to data corruption or crash.
•
Avoid unexpected conversion overflow in planner for very distant date values (Tom Lane) The date type supports a wider range of dates than can be represented by the timestamp types, but the planner assumed it could always convert a date to timestamp with impunity.
•
Fix pg_restore’s text output for large objects (BLOBs) when standard_conforming_strings is on (Tom Lane) Although restoring directly to a database worked correctly, string escaping was incorrect if pg_restore was asked for SQL text output and standard_conforming_strings had been enabled in the source database.
•
Fix erroneous parsing of tsquery values containing ... & !(subexpression) | ... (Tom Lane) Queries containing this combination of operators were not executed correctly. The same error existed in contrib/intarray’s query_int type and contrib/ltree’s ltxtquery type.
•
Fix buffer overrun in contrib/intarray’s input function for the query_int type (Apple) This bug is a security risk since the function’s return address could be overwritten. Thanks to Apple Inc’s security team for reporting this issue and supplying the fix. (CVE-2010-4015)
•
Fix bug in contrib/seg’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a seg column. If you have such an index, consider REINDEXing it after installing this update. (This is identical to the bug that was fixed in contrib/cube in the previous update.)
E.127. Release 8.3.13 Release date: 2010-12-16 This release contains a variety of fixes from 8.3.12. For information about new features in the 8.3 major release, see Section E.140.
E.127.1. Migration to Version 8.3.13 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
2374
Appendix E. Release Notes
E.127.2. Changes •
Force the default wal_sync_method to be fdatasync on Linux (Tom Lane, Marti Raudsepp) The default on Linux has actually been fdatasync for many years, but recent kernel changes caused PostgreSQL to choose open_datasync instead. This choice did not result in any performance improvement, and caused outright failures on certain filesystems, notably ext4 with the data=journal mount option.
•
Fix assorted bugs in WAL replay logic for GIN indexes (Tom Lane) This could result in “bad buffer id: 0” failures or corruption of index contents during replication.
•
Fix recovery from base backup when the starting checkpoint WAL record is not in the same WAL segment as its redo point (Jeff Davis)
•
Fix persistent slowdown of autovacuum workers when multiple workers remain active for a long time (Tom Lane) The effective vacuum_cost_limit for an autovacuum worker could drop to nearly zero if it processed enough tables, causing it to run extremely slowly.
•
Add support for detecting register-stack overrun on IA64 (Tom Lane) The IA64 architecture has two hardware stacks. Full prevention of stack-overrun failures requires checking both.
•
Add a check for stack overflow in copyObject() (Tom Lane) Certain code paths could crash due to stack overflow given a sufficiently complex query.
•
Fix detection of page splits in temporary GiST indexes (Heikki Linnakangas) It is possible to have a “concurrent” page split in a temporary index, if for example there is an open cursor scanning the index when an insertion is done. GiST failed to detect this case and hence could deliver wrong results when execution of the cursor continued.
•
Avoid memory leakage while ANALYZE’ing complex index expressions (Tom Lane)
•
Ensure an index that uses a whole-row Var still depends on its table (Tom Lane) An index declared like create index i on t (foo(t.*)) would not automatically get dropped when its table was dropped.
•
Do not “inline” a SQL function with multiple OUT parameters (Tom Lane) This avoids a possible crash due to loss of information about the expected result rowtype.
•
Behave correctly if ORDER BY, LIMIT, FOR UPDATE, or WITH is attached to the VALUES part of INSERT ... VALUES (Tom Lane)
•
Fix constant-folding of COALESCE() expressions (Tom Lane) The planner would sometimes attempt to evaluate sub-expressions that in fact could never be reached, possibly leading to unexpected errors.
•
Fix postmaster crash when connection acceptance (accept() or one of the calls made immediately after it) fails, and the postmaster was compiled with GSSAPI support (Alexander Chernikov)
•
Fix missed unlink of temporary files when log_temp_files is active (Tom Lane) If an error occurred while attempting to emit the log message, the unlink was not done, resulting in accumulation of temp files.
•
Add print functionality for InhRelation nodes (Tom Lane)
2375
Appendix E. Release Notes This avoids a failure when debug_print_parse is enabled and certain types of query are executed. •
Fix incorrect calculation of distance from a point to a horizontal line segment (Tom Lane) This bug affected several different geometric distance-measurement operators.
•
Fix PL/pgSQL’s handling of “simple” expressions to not fail in recursion or error-recovery cases (Tom Lane)
•
Fix PL/Python’s handling of set-returning functions (Jan Urbanski) Attempts to call SPI functions within the iterator generating a set result would fail.
•
Fix bug in contrib/cube’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a cube column. If you have such an index, consider REINDEXing it after installing this update.
•
Don’t emit “identifier will be truncated” notices in contrib/dblink except when creating new connections (Itagaki Takahiro)
•
Fix potential coredump on missing public key in contrib/pgcrypto (Marti Raudsepp)
•
Fix memory leak in contrib/xml2’s XPath query functions (Tom Lane)
•
Update time zone data files to tzdata release 2010o for DST law changes in Fiji and Samoa; also historical corrections for Hong Kong.
E.128. Release 8.3.12 Release date: 2010-10-04 This release contains a variety of fixes from 8.3.11. For information about new features in the 8.3 major release, see Section E.140.
E.128.1. Migration to Version 8.3.12 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.128.2. Changes •
Use a separate interpreter for each calling SQL userid in PL/Perl and PL/Tcl (Tom Lane) This change prevents security problems that can be caused by subverting Perl or Tcl code that will be executed later in the same session under another SQL user identity (for example, within a SECURITY DEFINER function). Most scripting languages offer numerous ways that that might be done, such as redefining standard functions or operators called by the target function. Without this change, any SQL user with Perl or Tcl language usage rights can do essentially anything with the SQL privileges of the target function’s owner. The cost of this change is that intentional communication among Perl and Tcl functions becomes more difficult. To provide an escape hatch, PL/PerlU and PL/TclU functions continue to use only
2376
Appendix E. Release Notes one interpreter per session. This is not considered a security issue since all such functions execute at the trust level of a database superuser already. It is likely that third-party procedural languages that claim to offer trusted execution have similar security issues. We advise contacting the authors of any PL you are depending on for securitycritical purposes. Our thanks to Tim Bunce for pointing out this issue (CVE-2010-3433). •
Prevent possible crashes in pg_get_expr() by disallowing it from being called with an argument that is not one of the system catalog columns it’s intended to be used with (Heikki Linnakangas, Tom Lane)
•
Treat exit code 128 (ERROR_WAIT_NO_CHILDREN) as non-fatal on Windows (Magnus Hagander) Under high load, Windows processes will sometimes fail at startup with this error code. Formerly the postmaster treated this as a panic condition and restarted the whole database, but that seems to be an overreaction.
•
Fix incorrect usage of non-strict OR joinclauses in Append indexscans (Tom Lane) This is a back-patch of an 8.4 fix that was missed in the 8.3 branch. This corrects an error introduced in 8.3.8 that could cause incorrect results for outer joins when the inner relation is an inheritance tree or UNION ALL subquery.
•
Fix possible duplicate scans of UNION ALL member relations (Tom Lane)
•
Fix “cannot handle unplanned sub-select” error (Tom Lane) This occurred when a sub-select contains a join alias reference that expands into an expression containing another sub-select.
•
Fix failure to mark cached plans as transient (Tom Lane) If a plan is prepared while CREATE INDEX CONCURRENTLY is in progress for one of the referenced tables, it is supposed to be re-planned once the index is ready for use. This was not happening reliably.
•
Reduce PANIC to ERROR in some occasionally-reported btree failure cases, and provide additional detail in the resulting error messages (Tom Lane) This should improve the system’s robustness with corrupted indexes.
•
Prevent show_session_authorization() from crashing within autovacuum processes (Tom Lane)
•
Defend against functions returning setof record where not all the returned rows are actually of the same rowtype (Tom Lane)
•
Fix possible failure when hashing a pass-by-reference function result (Tao Ma, Tom Lane)
•
Improve merge join’s handling of NULLs in the join columns (Tom Lane) A merge join can now stop entirely upon reaching the first NULL, if the sort order is such that NULLs sort high.
•
Take care to fsync the contents of lockfiles (both postmaster.pid and the socket lockfile) while writing them (Tom Lane) This omission could result in corrupted lockfile contents if the machine crashes shortly after postmaster start. That could in turn prevent subsequent attempts to start the postmaster from succeeding, until the lockfile is manually removed.
•
Avoid recursion while assigning XIDs to heavily-nested subtransactions (Andres Freund, Robert Haas) The original coding could result in a crash if there was limited stack space.
2377
Appendix E. Release Notes •
Avoid holding open old WAL segments in the walwriter process (Magnus Hagander, Heikki Linnakangas) The previous coding would prevent removal of no-longer-needed segments.
•
Fix log_line_prefix’s %i escape, which could produce junk early in backend startup (Tom Lane)
•
Fix possible data corruption in ALTER TABLE ... SET TABLESPACE when archiving is enabled (Jeff Davis)
•
Allow CREATE DATABASE and ALTER DATABASE ... SET TABLESPACE to be interrupted by query-cancel (Guillaume Lelarge)
•
Fix REASSIGN OWNED to handle operator classes and families (Asko Tiidumaa)
•
Fix possible core dump when comparing two empty tsquery values (Tom Lane)
•
Fix LIKE’s handling of patterns containing % followed by _ (Tom Lane) We’ve fixed this before, but there were still some incorrectly-handled cases.
•
In PL/Python, defend against null pointer results from PyCObject_AsVoidPtr and PyCObject_FromVoidPtr (Peter Eisentraut)
•
Make psql recognize DISCARD ALL as a command that should not be encased in a transaction block in autocommit-off mode (Itagaki Takahiro)
•
Fix ecpg to process data from RETURNING clauses correctly (Michael Meskes)
•
Improve contrib/dblink’s handling of tables containing dropped columns (Tom Lane)
•
Fix connection leak after “duplicate connection name” errors in contrib/dblink (Itagaki Takahiro)
•
Fix contrib/dblink to handle connection names longer than 62 bytes correctly (Itagaki Takahiro)
•
Add hstore(text, text) function to contrib/hstore (Robert Haas) This function is the recommended substitute for the now-deprecated => operator. It was backpatched so that future-proofed code can be used with older server versions. Note that the patch will be effective only after contrib/hstore is installed or reinstalled in a particular database. Users might prefer to execute the CREATE FUNCTION command by hand, instead.
•
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
•
Update time zone data files to tzdata release 2010l for DST law changes in Egypt and Palestine; also historical corrections for Finland. This change also adds new names for two Micronesian timezones: Pacific/Chuuk is now preferred over Pacific/Truk (and the preferred abbreviation is CHUT not TRUT) and Pacific/Pohnpei is preferred over Pacific/Ponape.
•
Make Windows’ “N. Central Asia Standard Time” timezone map to Asia/Novosibirsk, not Asia/Almaty (Magnus Hagander) Microsoft changed the DST behavior of this zone in the timezone update from KB976098. Asia/Novosibirsk is a better match to its new behavior.
2378
Appendix E. Release Notes
E.129. Release 8.3.11 Release date: 2010-05-17 This release contains a variety of fixes from 8.3.10. For information about new features in the 8.3 major release, see Section E.140.
E.129.1. Migration to Version 8.3.11 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.129.2. Changes •
Enforce restrictions in plperl using an opmask applied to the whole interpreter, instead of using Safe.pm (Tim Bunce, Andrew Dunstan) Recent developments have convinced us that Safe.pm is too insecure to rely on for making plperl trustable. This change removes use of Safe.pm altogether, in favor of using a separate interpreter with an opcode mask that is always applied. Pleasant side effects of the change include that it is now possible to use Perl’s strict pragma in a natural way in plperl, and that Perl’s $a and $b variables work as expected in sort routines, and that function compilation is significantly faster. (CVE-2010-1169)
•
Prevent PL/Tcl from executing untrustworthy code from pltcl_modules (Tom) PL/Tcl’s feature for autoloading Tcl code from a database table could be exploited for trojan-horse attacks, because there was no restriction on who could create or insert into that table. This change disables the feature unless pltcl_modules is owned by a superuser. (However, the permissions on the table are not checked, so installations that really need a less-than-secure modules table can still grant suitable privileges to trusted non-superusers.) Also, prevent loading code into the unrestricted “normal” Tcl interpreter unless we are really going to execute a pltclu function. (CVE-2010-1170)
•
Fix possible crash if a cache reset message is received during rebuild of a relcache entry (Heikki) This error was introduced in 8.3.10 while fixing a related failure.
•
Apply per-function GUC settings while running the language validator for the function (Itagaki Takahiro) This avoids failures if the function’s code is invalid without the setting; an example is that SQL functions may not parse if the search_path is not correct.
•
Do not allow an unprivileged user to reset superuser-only parameter settings (Alvaro) Previously, if an unprivileged user ran ALTER USER ... RESET ALL for himself, or ALTER DATABASE ... RESET ALL for a database he owns, this would remove all special parameter settings for the user or database, even ones that are only supposed to be changeable by a superuser. Now, the ALTER will only remove the parameters that the user has permission to change.
•
Avoid possible crash during backend shutdown if shutdown occurs when a CONTEXT addition would be made to log entries (Tom) In some cases the context-printing function would fail because the current transaction had already been rolled back when it came time to print a log message.
2379
Appendix E. Release Notes •
Ensure the archiver process responds to changes in archive_command as soon as possible (Tom)
•
Update pl/perl’s ppport.h for modern Perl versions (Andrew)
•
Fix assorted memory leaks in pl/python (Andreas Freund, Tom)
•
Prevent infinite recursion in psql when expanding a variable that refers to itself (Tom)
•
Fix psql’s \copy to not add spaces around a dot within \copy (select ...) (Tom) Addition of spaces around the decimal point in a numeric literal would result in a syntax error.
•
Fix unnecessary “GIN indexes do not support whole-index scans” errors for unsatisfiable queries using contrib/intarray operators (Tom)
•
Ensure that contrib/pgstattuple functions respond to cancel interrupts promptly (Tatsuhito Kasahara)
•
Make server startup deal properly with the case that shmget() returns EINVAL for an existing shared memory segment (Tom) This behavior has been observed on BSD-derived kernels including OS X. It resulted in an entirelymisleading startup failure complaining that the shared memory request size was too large.
•
Avoid possible crashes in syslogger process on Windows (Heikki)
•
Deal more robustly with incomplete time zone information in the Windows registry (Magnus)
•
Update the set of known Windows time zone names (Magnus)
•
Update time zone data files to tzdata release 2010j for DST law changes in Argentina, Australian Antarctic, Bangladesh, Mexico, Morocco, Pakistan, Palestine, Russia, Syria, Tunisia; also historical corrections for Taiwan. Also, add PKST (Pakistan Summer Time) to the default set of timezone abbreviations.
E.130. Release 8.3.10 Release date: 2010-03-15 This release contains a variety of fixes from 8.3.9. For information about new features in the 8.3 major release, see Section E.140.
E.130.1. Migration to Version 8.3.10 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.130.2. Changes •
Add new configuration parameter ssl_renegotiation_limit to control how often we do session key renegotiation for an SSL connection (Magnus) This can be set to zero to disable renegotiation completely, which may be required if a broken SSL library is used. In particular, some vendors are shipping stopgap patches for CVE-2009-3555 that cause renegotiation attempts to fail.
2380
Appendix E. Release Notes •
Fix possible deadlock during backend startup (Tom)
•
Fix possible crashes due to not handling errors during relcache reload cleanly (Tom)
•
Fix possible crash due to use of dangling pointer to a cached plan (Tatsuo)
•
Fix possible crashes when trying to recover from a failure in subtransaction start (Tom)
•
Fix server memory leak associated with use of savepoints and a client encoding different from server’s encoding (Tom)
•
Fix incorrect WAL data emitted during end-of-recovery cleanup of a GIST index page split (Yoichi Hirai) This would result in index corruption, or even more likely an error during WAL replay, if we were unlucky enough to crash during end-of-recovery cleanup after having completed an incomplete GIST insertion.
•
Make substring() for bit types treat any negative length as meaning “all the rest of the string” (Tom) The previous coding treated only -1 that way, and would produce an invalid result value for other negative values, possibly leading to a crash (CVE-2010-0442).
•
Fix integer-to-bit-string conversions to handle the first fractional byte correctly when the output bit width is wider than the given integer by something other than a multiple of 8 bits (Tom)
•
Fix some cases of pathologically slow regular expression matching (Tom)
•
Fix assorted crashes in xml processing caused by sloppy memory management (Tom) This is a back-patch of changes first applied in 8.4. The 8.3 code was known buggy, but the new code was sufficiently different to not want to back-patch it until it had gotten some field testing.
•
Fix bug with trying to update a field of an element of a composite-type array column (Tom)
•
Fix the STOP WAL LOCATION entry in backup history files to report the next WAL segment’s name when the end location is exactly at a segment boundary (Itagaki Takahiro)
•
Fix some more cases of temporary-file leakage (Heikki) This corrects a problem introduced in the previous minor release. One case that failed is when a plpgsql function returning set is called within another function’s exception handler.
•
Improve constraint exclusion processing of boolean-variable cases, in particular make it possible to exclude a partition that has a “bool_column = false” constraint (Tom)
•
When reading pg_hba.conf and related files, do not treat @something as a file inclusion request if the @ appears inside quote marks; also, never treat @ by itself as a file inclusion request (Tom) This prevents erratic behavior if a role or database name starts with @. If you need to include a file whose path name contains spaces, you can still do so, but you must write @"/path to/file" rather than putting the quotes around the whole construct.
•
Prevent infinite loop on some platforms if a directory is named as an inclusion target in pg_hba.conf and related files (Tom)
•
Fix possible infinite loop if SSL_read or SSL_write fails without setting errno (Tom) This is reportedly possible with some Windows versions of openssl.
•
Disallow GSSAPI authentication on local connections, since it requires a hostname to function correctly (Magnus)
•
Make ecpg report the proper SQLSTATE if the connection disappears (Michael)
2381
Appendix E. Release Notes •
Fix psql’s numericlocale option to not format strings it shouldn’t in latex and troff output formats (Heikki)
•
Make psql return the correct exit status (3) when ON_ERROR_STOP and --single-transaction are both specified and an error occurs during the implied COMMIT (Bruce)
•
Fix plpgsql failure in one case where a composite column is set to NULL (Tom)
•
Fix possible failure when calling PL/Perl functions from PL/PerlU or vice versa (Tim Bunce)
•
Add volatile markings in PL/Python to avoid possible compiler-specific misbehavior (Zdenek Kotala)
•
Ensure PL/Tcl initializes the Tcl interpreter fully (Tom) The only known symptom of this oversight is that the Tcl clock command misbehaves if using Tcl 8.5 or later.
•
Prevent crash in contrib/dblink when too many key columns are specified to a dblink_build_sql_* function (Rushabh Lathia, Joe Conway)
•
Allow zero-dimensional arrays in contrib/ltree operations (Tom) This case was formerly rejected as an error, but it’s more convenient to treat it the same as a zeroelement array. In particular this avoids unnecessary failures when an ltree operation is applied to the result of ARRAY(SELECT ...) and the sub-select returns no rows.
•
Fix assorted crashes in contrib/xml2 caused by sloppy memory management (Tom)
•
Make building of contrib/xml2 more robust on Windows (Andrew)
•
Fix race condition in Windows signal handling (Radu Ilie) One known symptom of this bug is that rows in pg_listener could be dropped under heavy load.
•
Update time zone data files to tzdata release 2010e for DST law changes in Bangladesh, Chile, Fiji, Mexico, Paraguay, Samoa.
E.131. Release 8.3.9 Release date: 2009-12-14 This release contains a variety of fixes from 8.3.8. For information about new features in the 8.3 major release, see Section E.140.
E.131.1. Migration to Version 8.3.9 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.8, see Section E.132.
E.131.2. Changes •
Protect against indirect security threats caused by index functions changing session-local state (Gurjeet Singh, Tom)
2382
Appendix E. Release Notes This change prevents allegedly-immutable index functions from possibly subverting a superuser’s session (CVE-2009-4136). •
Reject SSL certificates containing an embedded null byte in the common name (CN) field (Magnus) This prevents unintended matching of a certificate to a server or client name during SSL validation (CVE-2009-4034).
•
Fix possible crash during backend-startup-time cache initialization (Tom)
•
Avoid crash on empty thesaurus dictionary (Tom)
•
Prevent signals from interrupting VACUUM at unsafe times (Alvaro) This fix prevents a PANIC if a VACUUM FULL is canceled after it’s already committed its tuple movements, as well as transient errors if a plain VACUUM is interrupted after having truncated the table.
•
Fix possible crash due to integer overflow in hash table size calculation (Tom) This could occur with extremely large planner estimates for the size of a hashjoin’s result.
•
Fix very rare crash in inet/cidr comparisons (Chris Mikkelson)
•
Ensure that shared tuple-level locks held by prepared transactions are not ignored (Heikki)
•
Fix premature drop of temporary files used for a cursor that is accessed within a subtransaction (Heikki)
•
Fix memory leak in syslogger process when rotating to a new CSV logfile (Tom)
•
Fix Windows permission-downgrade logic (Jesse Morris) This fixes some cases where the database failed to start on Windows, often with misleading error messages such as “could not locate matching postgres executable”.
•
Fix incorrect logic for GiST index page splits, when the split depends on a non-first column of the index (Paul Ramsey)
•
Don’t error out if recycling or removing an old WAL file fails at the end of checkpoint (Heikki) It’s better to treat the problem as non-fatal and allow the checkpoint to complete. Future checkpoints will retry the removal. Such problems are not expected in normal operation, but have been seen to be caused by misdesigned Windows anti-virus and backup software.
•
Ensure WAL files aren’t repeatedly archived on Windows (Heikki) This is another symptom that could happen if some other process interfered with deletion of a no-longer-needed file.
•
Fix PAM password processing to be more robust (Tom) The previous code is known to fail with the combination of the Linux pam_krb5 PAM module with Microsoft Active Directory as the domain controller. It might have problems elsewhere too, since it was making unjustified assumptions about what arguments the PAM stack would pass to it.
•
Raise the maximum authentication token (Kerberos ticket) size in GSSAPI and SSPI authentication methods (Ian Turner) While the old 2000-byte limit was more than enough for Unix Kerberos implementations, tickets issued by Windows Domain Controllers can be much larger.
•
Re-enable collection of access statistics for sequences (Akira Kurosawa) This used to work but was broken in 8.3.
•
Fix processing of ownership dependencies during CREATE OR REPLACE FUNCTION (Tom)
2383
Appendix E. Release Notes •
Fix incorrect handling of WHERE x =x conditions (Tom) In some cases these could get ignored as redundant, but they aren’t — they’re equivalent to x IS NOT NULL.
•
Make text search parser accept underscores in XML attributes (Peter)
•
Fix encoding handling in xml binary input (Heikki) If the XML header doesn’t specify an encoding, we now assume UTF-8 by default; the previous handling was inconsistent.
•
Fix bug with calling plperl from plperlu or vice versa (Tom) An error exit from the inner function could result in crashes due to failure to re-select the correct Perl interpreter for the outer function.
•
Fix session-lifespan memory leak when a PL/Perl function is redefined (Tom)
•
Ensure that Perl arrays are properly converted to PostgreSQL arrays when returned by a setreturning PL/Perl function (Andrew Dunstan, Abhijit Menon-Sen) This worked correctly already for non-set-returning functions.
•
Fix rare crash in exception processing in PL/Python (Peter)
•
In contrib/pg_standby, disable triggering failover with a signal on Windows (Fujii Masao) This never did anything useful, because Windows doesn’t have Unix-style signals, but recent changes made it actually crash.
•
Ensure psql’s flex module is compiled with the correct system header definitions (Tom) This fixes build failures on platforms where --enable-largefile causes incompatible changes in the generated code.
•
Make the postmaster ignore any application_name parameter in connection request packets, to improve compatibility with future libpq versions (Tom)
•
Update the timezone abbreviation files to match current reality (Joachim Wieland) This includes adding IDT and SGT to the default timezone abbreviation set.
•
Update time zone data files to tzdata release 2009s for DST law changes in Antarctica, Argentina, Bangladesh, Fiji, Novokuznetsk, Pakistan, Palestine, Samoa, Syria; also historical corrections for Hong Kong.
E.132. Release 8.3.8 Release date: 2009-09-09 This release contains a variety of fixes from 8.3.7. For information about new features in the 8.3 major release, see Section E.140.
E.132.1. Migration to Version 8.3.8 A dump/restore is not required for those running 8.3.X. However, if you have any hash indexes on interval columns, you must REINDEX them after updating to 8.3.8. Also, if you are upgrading from a version earlier than 8.3.5, see Section E.135.
2384
Appendix E. Release Notes
E.132.2. Changes •
Fix Windows shared-memory allocation code (Tsutomu Yamada, Magnus) This bug led to the often-reported “could not reattach to shared memory” error message.
•
Force WAL segment switch during pg_start_backup() (Heikki) This avoids corner cases that could render a base backup unusable.
•
Disallow RESET ROLE and RESET SESSION AUTHORIZATION inside security-definer functions (Tom, Heikki) This covers a case that was missed in the previous patch that disallowed SET ROLE and SET SESSION AUTHORIZATION inside security-definer functions. (See CVE-2007-6600)
•
Make LOAD of an already-loaded loadable module into a no-op (Tom) Formerly, LOAD would attempt to unload and re-load the module, but this is unsafe and not all that useful.
•
Disallow empty passwords during LDAP authentication (Magnus)
•
Fix handling of sub-SELECTs appearing in the arguments of an outer-level aggregate function (Tom)
•
Fix bugs associated with fetching a whole-row value from the output of a Sort or Materialize plan node (Tom)
•
Prevent synchronize_seqscans from changing the results of scrollable and WITH HOLD cursors (Tom)
•
Revert planner change that disabled partial-index and constraint exclusion optimizations when there were more than 100 clauses in an AND or OR list (Tom)
•
Fix hash calculation for data type interval (Tom) This corrects wrong results for hash joins on interval values. It also changes the contents of hash indexes on interval columns. If you have any such indexes, you must REINDEX them after updating.
•
Treat to_char(..., ’TH’) as an uppercase ordinal suffix with ’HH’/’HH12’ (Heikki) It was previously handled as ’th’ (lowercase).
•
Fix overflow for INTERVAL ’x ms’ when x is more than 2 million and integer datetimes are in use (Alex Hunsaker)
•
Fix calculation of distance between a point and a line segment (Tom) This led to incorrect results from a number of geometric operators.
•
Fix money data type to work in locales where currency amounts have no fractional digits, e.g. Japan (Itagaki Takahiro)
•
Fix LIKE for case where pattern contains %_ (Tom)
•
Properly round datetime input like 00:12:57.9999999999999999999999999999 (Tom)
•
Fix memory leaks in XML operations (Tom)
•
Fix poor choice of page split point in GiST R-tree operator classes (Teodor)
•
Ensure that a “fast shutdown” request will forcibly terminate open sessions, even if a “smart shutdown” was already in progress (Fujii Masao)
•
Avoid performance degradation in bulk inserts into GIN indexes when the input values are (nearly) in sorted order (Tom)
2385
Appendix E. Release Notes •
Correctly enforce NOT NULL domain constraints in some contexts in PL/pgSQL (Tom)
•
Fix portability issues in plperl initialization (Andrew Dunstan)
•
Fix pg_ctl to not go into an infinite loop if postgresql.conf is empty (Jeff Davis)
•
Improve pg_dump’s efficiency when there are many large objects (Tamas Vincze)
•
Use SIGUSR1, not SIGQUIT, as the failover signal for pg_standby (Heikki)
•
Make pg_standby’s maxretries option behave as documented (Fujii Masao)
•
Make contrib/hstore throw an error when a key or value is too long to fit in its data structure, rather than silently truncating it (Andrew Gierth)
•
Fix contrib/xml2’s xslt_process() to properly handle the maximum number of parameters (twenty) (Tom)
•
Improve robustness of libpq’s code to recover from errors during COPY FROM STDIN (Tom)
•
Avoid including conflicting readline and editline header files when both libraries are installed (Zdenek Kotala)
•
Update time zone data files to tzdata release 2009l for DST law changes in Bangladesh, Egypt, Jordan, Pakistan, Argentina/San_Luis, Cuba, Jordan (historical correction only), Mauritius, Morocco, Palestine, Syria, Tunisia.
E.133. Release 8.3.7 Release date: 2009-03-16 This release contains a variety of fixes from 8.3.6. For information about new features in the 8.3 major release, see Section E.140.
E.133.1. Migration to Version 8.3.7 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.5, see Section E.135.
E.133.2. Changes •
Prevent error recursion crashes when encoding conversion fails (Tom) This change extends fixes made in the last two minor releases for related failure scenarios. The previous fixes were narrowly tailored for the original problem reports, but we have now recognized that any error thrown by an encoding conversion function could potentially lead to infinite recursion while trying to report the error. The solution therefore is to disable translation and encoding conversion and report the plain-ASCII form of any error message, if we find we have gotten into a recursive error reporting situation. (CVE-2009-0922)
•
Disallow CREATE CONVERSION with the wrong encodings for the specified conversion function (Heikki) This prevents one possible scenario for encoding conversion failure. The previous change is a backstop to guard against other kinds of failures in the same area.
2386
Appendix E. Release Notes •
Fix xpath() to not modify the path expression unless necessary, and to make a saner attempt at it when necessary (Andrew) The SQL standard suggests that xpath should work on data that is a document fragment, but libxml doesn’t support that, and indeed it’s not clear that this is sensible according to the XPath standard. xpath attempted to work around this mismatch by modifying both the data and the path expression, but the modification was buggy and could cause valid searches to fail. Now, xpath checks whether the data is in fact a well-formed document, and if so invokes libxml with no change to the data or path expression. Otherwise, a different modification method that is somewhat less likely to fail is used. Note: The new modification method is still not 100% satisfactory, and it seems likely that no real solution is possible. This patch should therefore be viewed as a band-aid to keep from breaking existing applications unnecessarily. It is likely that PostgreSQL 8.4 will simply reject use of xpath on data that is not a well-formed document.
•
Fix core dump when to_char() is given format codes that are inappropriate for the type of the data argument (Tom)
•
Fix possible failure in text search when C locale is used with a multi-byte encoding (Teodor) Crashes were possible on platforms where wchar_t is narrower than int; Windows in particular.
•
Fix extreme inefficiency in text search parser’s handling of an email-like string containing multiple @ characters (Heikki)
•
Fix planner problem with sub-SELECT in the output list of a larger subquery (Tom) The known symptom of this bug is a “failed to locate grouping columns” error that is dependent on the datatype involved; but there could be other issues as well.
•
Fix decompilation of CASE WHEN with an implicit coercion (Tom) This mistake could lead to Assert failures in an Assert-enabled build, or an “unexpected CASE WHEN clause” error message in other cases, when trying to examine or dump a view.
•
Fix possible misassignment of the owner of a TOAST table’s rowtype (Tom) If CLUSTER or a rewriting variant of ALTER TABLE were executed by someone other than the table owner, the pg_type entry for the table’s TOAST table would end up marked as owned by that someone. This caused no immediate problems, since the permissions on the TOAST rowtype aren’t examined by any ordinary database operation. However, it could lead to unexpected failures if one later tried to drop the role that issued the command (in 8.1 or 8.2), or “owner of data type appears to be invalid” warnings from pg_dump after having done so (in 8.3).
•
Change UNLISTEN to exit quickly if the current session has never executed any LISTEN command (Tom) Most of the time this is not a particularly useful optimization, but since DISCARD ALL invokes UNLISTEN, the previous coding caused a substantial performance problem for applications that made heavy use of DISCARD ALL.
•
Fix PL/pgSQL to not treat INTO after INSERT as an INTO-variables clause anywhere in the string, not only at the start; in particular, don’t fail for INSERT INTO within CREATE RULE (Tom)
•
Clean up PL/pgSQL error status variables fully at block exit (Ashesh Vashi and Dave Page) This is not a problem for PL/pgSQL itself, but the omission could cause the PL/pgSQL Debugger to crash while examining the state of a function.
2387
Appendix E. Release Notes •
Retry failed calls to CallNamedPipe() on Windows (Steve Marshall, Magnus) It appears that this function can sometimes fail transiently; we previously treated any failure as a hard error, which could confuse LISTEN/NOTIFY as well as other operations.
•
Add MUST (Mauritius Island Summer Time) to the default list of known timezone abbreviations (Xavier Bugaud)
E.134. Release 8.3.6 Release date: 2009-02-02 This release contains a variety of fixes from 8.3.5. For information about new features in the 8.3 major release, see Section E.140.
E.134.1. Migration to Version 8.3.6 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.5, see Section E.135.
E.134.2. Changes •
Make DISCARD ALL release advisory locks, in addition to everything it already did (Tom) This was decided to be the most appropriate behavior. This could affect existing applications, however.
•
Fix whole-index GiST scans to work correctly (Teodor) This error could cause rows to be lost if a table is clustered on a GiST index.
•
Fix crash of xmlconcat(NULL) (Peter)
•
Fix possible crash in ispell dictionary if high-bit-set characters are used as flags (Teodor) This is known to be done by one widely available Norwegian dictionary, and the same condition may exist in others.
•
Fix misordering of pg_dump output for composite types (Tom) The most likely problem was for user-defined operator classes to be dumped after indexes or views that needed them.
•
Improve handling of URLs in headline() function (Teodor)
•
Improve handling of overlength headlines in headline() function (Teodor)
•
Prevent possible Assert failure or misconversion if an encoding conversion is created with the wrong conversion function for the specified pair of encodings (Tom, Heikki)
•
Fix possible Assert failure if a statement executed in PL/pgSQL is rewritten into another kind of statement, for example if an INSERT is rewritten into an UPDATE (Heikki)
•
Ensure that a snapshot is available to datatype input functions (Tom) This primarily affects domains that are declared with CHECK constraints involving user-defined stable or immutable functions. Such functions typically fail if no snapshot has been set.
2388
Appendix E. Release Notes •
Make it safer for SPI-using functions to be used within datatype I/O; in particular, to be used in domain check constraints (Tom)
•
Avoid unnecessary locking of small tables in VACUUM (Heikki)
•
Fix a problem that sometimes kept ALTER TABLE ENABLE/DISABLE RULE from being recognized by active sessions (Tom)
•
Fix a problem that made UPDATE RETURNING tableoid return zero instead of the correct OID (Tom)
•
Allow functions declared as taking ANYARRAY to work on the pg_statistic columns of that type (Tom) This used to work, but was unintentionally broken in 8.3.
•
Fix planner misestimation of selectivity when transitive equality is applied to an outer-join clause (Tom) This could result in bad plans for queries like ... from a left join b on a.a1 = b.b1 where a.a1 = 42 ...
•
Improve optimizer’s handling of long IN lists (Tom) This change avoids wasting large amounts of time on such lists when constraint exclusion is enabled.
•
Prevent synchronous scan during GIN index build (Tom) Because GIN is optimized for inserting tuples in increasing TID order, choosing to use a synchronous scan could slow the build by a factor of three or more.
•
Ensure that the contents of a holdable cursor don’t depend on the contents of TOAST tables (Tom) Previously, large field values in a cursor result might be represented as TOAST pointers, which would fail if the referenced table got dropped before the cursor is read, or if the large value is deleted and then vacuumed away. This cannot happen with an ordinary cursor, but it could with a cursor that is held past its creating transaction.
•
Fix memory leak when a set-returning function is terminated without reading its whole result (Tom)
•
Fix encoding conversion problems in XML functions when the database encoding isn’t UTF-8 (Tom)
•
Fix contrib/dblink’s dblink_get_result(text,bool) function (Joe)
•
Fix possible garbage output from contrib/sslinfo functions (Tom)
•
Fix incorrect behavior of contrib/tsearch2 compatibility trigger when it’s fired more than once in a command (Teodor)
•
Fix possible mis-signaling in autovacuum (Heikki)
•
Support running as a service on Windows 7 beta (Dave and Magnus)
•
Fix ecpg’s handling of varchar structs (Michael)
•
Fix configure script to properly report failure when unable to obtain linkage information for PL/Perl (Andrew)
•
Make all documentation reference pgsql-bugs and/or pgsql-hackers as appropriate, instead of the now-decommissioned pgsql-ports and pgsql-patches mailing lists (Tom)
•
Update time zone data files to tzdata release 2009a (for Kathmandu and historical DST corrections in Switzerland, Cuba)
2389
Appendix E. Release Notes
E.135. Release 8.3.5 Release date: 2008-11-03 This release contains a variety of fixes from 8.3.4. For information about new features in the 8.3 major release, see Section E.140.
E.135.1. Migration to Version 8.3.5 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see Section E.139. Also, if you were running a previous 8.3.X release, it is recommended to REINDEX all GiST indexes after the upgrade.
E.135.2. Changes •
Fix GiST index corruption due to marking the wrong index entry “dead” after a deletion (Teodor) This would result in index searches failing to find rows they should have found. Corrupted indexes can be fixed with REINDEX.
•
Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the “character has no equivalent” message itself couldn’t be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation.
•
Fix possible crash in bytea-to-XML mapping (Michael McMaster)
•
Fix possible crash when deeply nested functions are invoked from a trigger (Tom)
•
Improve optimization of expression IN (expression-list) queries (Tom, per an idea from Robert Haas) Cases in which there are query variables on the right-hand side had been handled less efficiently in 8.2.x and 8.3.x than in prior versions. The fix restores 8.1 behavior for such cases.
•
Fix mis-expansion of rule queries when a sub-SELECT appears in a function call in FROM, a multirow VALUES list, or a RETURNING list (Tom) The usual symptom of this problem is an “unrecognized node type” error.
•
Fix Assert failure during rescan of an IS NULL search of a GiST index (Teodor)
•
Fix memory leak during rescan of a hashed aggregation plan (Neil)
•
Ensure an error is reported when a newly-defined PL/pgSQL trigger function is invoked as a normal function (Tom)
•
Force a checkpoint before CREATE DATABASE starts to copy files (Heikki) This prevents a possible failure if files had recently been deleted in the source database.
•
Prevent possible collision of relfilenode numbers when moving a table to another tablespace with ALTER SET TABLESPACE (Heikki) The command tried to re-use the existing filename, instead of picking one that is known unused in the destination directory.
•
Fix incorrect text search headline generation when single query item matches first word of text (Sushant Sinha)
2390
Appendix E. Release Notes •
Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an --enable-integer-datetimes build (Ron Mayer)
•
Make ILIKE compare characters case-insensitively even when they’re escaped (Andrew)
•
Ensure DISCARD is handled properly by statement logging (Tom)
•
Fix incorrect logging of last-completed-transaction time during PITR recovery (Tom)
•
Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn’t handle it properly. The only likely consequence is an incorrect error indication.
•
Mark SessionReplicationRole as PGDLLIMPORT so it can be used by Slony on Windows (Magnus)
•
Fix small memory leak when using libpq’s gsslib parameter (Magnus) The space used by the parameter string was not freed at connection close.
•
Ensure libgssapi is linked into libpq if needed (Markus Schaaf)
•
Fix ecpg’s parsing of CREATE ROLE (Michael)
•
Fix recent breakage of pg_ctl restart (Tom)
•
Ensure pg_control is opened in binary mode (Itagaki Takahiro) pg_controldata and pg_resetxlog did this incorrectly, and so could fail on Windows.
•
Update time zone data files to tzdata release 2008i (for DST law changes in Argentina, Brazil, Mauritius, Syria)
E.136. Release 8.3.4 Release date: 2008-09-22 This release contains a variety of fixes from 8.3.3. For information about new features in the 8.3 major release, see Section E.140.
E.136.1. Migration to Version 8.3.4 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see Section E.139.
E.136.2. Changes •
Fix bug in btree WAL recovery code (Heikki) Recovery failed if the WAL ended partway through a page split operation.
•
Fix potential use of wrong cutoff XID for HOT page pruning (Alvaro) This error created a risk of corruption in system catalogs that are consulted by VACUUM: dead tuple versions might be removed too soon. The impact of this on actual database operations would be
2391
Appendix E. Release Notes minimal, since the system doesn’t follow MVCC rules while examining catalogs, but it might result in transiently wrong output from pg_dump or other client programs. •
Fix potential miscalculation of datfrozenxid (Alvaro) This error may explain some recent reports of failure to remove old pg_clog data.
•
Fix incorrect HOT updates after pg_class is reindexed (Tom) Corruption of pg_class could occur if REINDEX TABLE pg_class was followed in the same session by an ALTER TABLE RENAME or ALTER TABLE SET SCHEMA command.
•
Fix missed “combo cid” case (Karl Schnaitter) This error made rows incorrectly invisible to a transaction in which they had been deleted by multiple subtransactions that all aborted.
•
Prevent autovacuum from crashing if the table it’s currently checking is deleted at just the wrong time (Alvaro)
•
Widen local lock counters from 32 to 64 bits (Tom) This responds to reports that the counters could overflow in sufficiently long transactions, leading to unexpected “lock is already held” errors.
•
Fix possible duplicate output of tuples during a GiST index scan (Teodor)
•
Regenerate foreign key checking queries from scratch when either table is modified (Tom) Previously, 8.3 would attempt to replan the query, but would work from previously generated query text. This led to failures if a table or column was renamed.
•
Fix missed permissions checks when a view contains a simple UNION ALL construct (Heikki) Permissions for the referenced tables were checked properly, but not permissions for the view itself.
•
Add checks in executor startup to ensure that the tuples produced by an INSERT or UPDATE will match the target table’s current rowtype (Tom) This situation is believed to be impossible in 8.3, but it can happen in prior releases, so a check seems prudent.
•
Fix possible repeated drops during DROP OWNED (Tom) This would typically result in strange errors such as “cache lookup failed for relation NNN”.
•
Fix several memory leaks in XML operations (Kris Jurka, Tom)
•
Fix xmlserialize() to raise error properly for unacceptable target data type (Tom)
•
Fix a couple of places that mis-handled multibyte characters in text search configuration file parsing (Tom) Certain characters occurring in configuration files would always cause “invalid byte sequence for encoding” failures.
•
Provide file name and line number location for all errors reported in text search configuration files (Tom)
•
Fix AT TIME ZONE to first try to interpret its timezone argument as a timezone abbreviation, and only try it as a full timezone name if that fails, rather than the other way around as formerly (Tom) The timestamp input functions have always resolved ambiguous zone names in this order. Making AT TIME ZONE do so as well improves consistency, and fixes a compatibility bug introduced in 8.1: in ambiguous cases we now behave the same as 8.0 and before did, since in the older versions AT TIME ZONE accepted only abbreviations.
2392
Appendix E. Release Notes •
Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom)
•
Prevent integer overflows during units conversion when displaying a configuration parameter that has units (Tom)
•
Improve performance of writing very long log messages to syslog (Tom)
•
Allow spaces in the suffix part of an LDAP URL in pg_hba.conf (Tom)
•
Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom)
•
Fix planner bug that could improperly push down IS NULL tests below an outer join (Tom) This was triggered by occurrence of IS NULL tests for the same relation in all arms of an upper OR clause.
•
Fix planner bug with nested sub-select expressions (Tom) If the outer sub-select has no direct dependency on the parent query, but the inner one does, the outer value might not get recalculated for new parent query rows.
•
Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions’ contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like col IS NULL.
•
Fix PL/pgSQL to not fail when a FOR loop’s target variable is a record containing composite-type fields (Tom)
•
Fix PL/Tcl to behave correctly with Tcl 8.5, and to be more careful about the encoding of data sent to or from Tcl (Tom)
•
Improve performance of PQescapeBytea() (Rudolf Leitgeb)
•
On Windows, work around a Microsoft bug by preventing libpq from trying to send more than 64kB per system call (Magnus)
•
Fix ecpg to handle variables properly in SET commands (Michael)
•
Improve pg_dump and pg_restore’s error reporting after failure to send a SQL command (Tom)
•
Fix pg_ctl to properly preserve postmaster command-line arguments across a restart (Bruce)
•
Fix erroneous WAL file cutoff point calculation in pg_standby (Simon)
•
Update time zone data files to tzdata release 2008f (for DST law changes in Argentina, Bahamas, Brazil, Mauritius, Morocco, Pakistan, Palestine, and Paraguay)
E.137. Release 8.3.3 Release date: 2008-06-12 This release contains one serious and one minor bug fix over 8.3.2. For information about new features in the 8.3 major release, see Section E.140.
E.137.1. Migration to Version 8.3.3 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see Section E.139.
2393
Appendix E. Release Notes
E.137.2. Changes •
Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid SELECT DISTINCT view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7.
•
Make ALTER AGGREGATE ... OWNER TO update pg_shdepend (Tom) This oversight could lead to problems if the aggregate was later involved in a DROP OWNED or REASSIGN OWNED operation.
E.138. Release 8.3.2 Release date: never released This release contains a variety of fixes from 8.3.1. For information about new features in the 8.3 major release, see Section E.140.
E.138.1. Migration to Version 8.3.2 A dump/restore is not required for those running 8.3.X. However, if you are upgrading from a version earlier than 8.3.1, see Section E.139.
E.138.2. Changes •
Fix ERRORDATA_STACK_SIZE exceeded crash that occurred on Windows when using UTF-8 database encoding and a different client encoding (Tom)
•
Fix incorrect archive truncation point calculation for the %r macro in recovery_command parameters (Simon) This could lead to data loss if a warm-standby script relied on %r to decide when to throw away WAL segment files.
•
Fix ALTER TABLE ADD COLUMN ... PRIMARY KEY so that the new column is correctly checked to see if it’s been initialized to all non-nulls (Brendan Jurd) Previous versions neglected to check this requirement at all.
•
Fix REASSIGN OWNED so that it works on procedural languages too (Alvaro)
•
Fix problems with SELECT FOR UPDATE/SHARE occurring as a subquery in a query with a nonSELECT top-level operation (Tom)
•
Fix possible CREATE TABLE failure when inheriting the “same” constraint from multiple parent relations that inherited that constraint from a common ancestor (Tom)
2394
Appendix E. Release Notes •
Fix pg_get_ruledef() to show the alias, if any, attached to the target table of an UPDATE or DELETE (Tom)
•
Restore the pre-8.3 behavior that an out-of-range block number in a TID being used in a TidScan plan results in silently not matching any rows (Tom) 8.3.0 and 8.3.1 threw an error instead.
•
Fix GIN bug that could result in a too many LWLocks taken failure (Teodor)
•
Fix broken GiST comparison function for tsquery (Teodor)
•
Fix tsvector_update_trigger() and ts_stat() to accept domains over the types they expect to work with (Tom)
•
Fix failure to support enum data types as foreign keys (Tom)
•
Avoid possible crash when decompressing corrupted data (Zdenek Kotala)
•
Fix race conditions between delayed unlinks and DROP DATABASE (Heikki) In the worst case this could result in deleting a newly created table in a new database that happened to get the same OID as the recently-dropped one; but of course that is an extremely low-probability scenario.
•
Repair two places where SIGTERM exit of a backend could leave corrupted state in shared memory (Tom) Neither case is very important if SIGTERM is used to shut down the whole database cluster together, but there was a problem if someone tried to SIGTERM individual backends.
•
Fix possible crash due to incorrect plan generated for an x IN (SELECT y FROM ...) clause when x and y have different data types; and make sure the behavior is semantically correct when the conversion from y ’s type to x ’s type is lossy (Tom)
•
Fix oversight that prevented the planner from substituting known Param values as if they were constants (Tom) This mistake partially disabled optimization of unnamed extended-Query statements in 8.3.0 and 8.3.1: in particular the LIKE-to-indexscan optimization would never be applied if the LIKE pattern was passed as a parameter, and constraint exclusion depending on a parameter value didn’t work either.
•
Fix planner failure when an indexable MIN or MAX aggregate is used with DISTINCT or ORDER BY (Tom)
•
Fix planner to ensure it never uses a “physical tlist” for a plan node that is feeding a Sort node (Tom) This led to the sort having to push around more data than it really needed to, since unused column values were included in the sorted data.
•
Avoid unnecessary copying of query strings (Tom) This fixes a performance problem introduced in 8.3.0 when a very large number of commands are submitted as a single query string.
•
Make TransactionIdIsCurrentTransactionId() use binary search instead of linear search when checking child-transaction XIDs (Heikki) This fixes some cases in which 8.3.0 was significantly slower than earlier releases.
•
Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic “Yo” characters (e and E with two dots) (Sergey Burladyan)
2395
Appendix E. Release Notes •
Fix several datatype input functions, notably array_in(), that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions.
•
Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn’t got a match. An example is substring(’foo’ from ’foo(bar)?’). This should return NULL, since (bar) isn’t matched, but it was mistakenly returning the whole-pattern match instead (ie, foo).
•
Prevent cancellation of an auto-vacuum that was launched to prevent XID wraparound (Alvaro)
•
Improve ANALYZE’s handling of in-doubt tuples (those inserted or deleted by a not-yet-committed transaction) so that the counts it reports to the stats collector are more likely to be correct (Pavan Deolasee)
•
Fix initdb to reject a relative path for its --xlogdir (-X) option (Tom)
•
Make psql print tab characters as an appropriate number of spaces, rather than \x09 as was done in 8.3.0 and 8.3.1 (Bruce)
•
Update time zone data files to tzdata release 2008c (for DST law changes in Morocco, Iraq, Choibalsan, Pakistan, Syria, Cuba, and Argentina/San_Luis)
•
Add ECPGget_PGconn() function to ecpglib (Michael)
•
Fix incorrect result from ecpg’s PGTYPEStimestamp_sub() function (Michael)
•
Fix handling of continuation line markers in ecpg (Michael)
•
Fix possible crashes in contrib/cube functions (Tom)
•
Fix core dump in contrib/xml2’s xpath_table() function when the input query returns a NULL value (Tom)
•
Fix contrib/xml2’s makefile to not override CFLAGS, and make it auto-configure properly for libxslt present or not (Tom)
E.139. Release 8.3.1 Release date: 2008-03-17 This release contains a variety of fixes from 8.3.0. For information about new features in the 8.3 major release, see Section E.140.
E.139.1. Migration to Version 8.3.1 A dump/restore is not required for those running 8.3.X. However, you might need to REINDEX indexes on textual columns after updating, if you are affected by the Windows locale issue described below.
2396
Appendix E. Release Notes
E.139.2. Changes •
Fix character string comparison for Windows locales that consider different character combinations as equal (Tom) This fix applies only on Windows and only when using UTF-8 database encoding. The same fix was made for all other cases over two years ago, but Windows with UTF-8 uses a separate code path that was not updated. If you are using a locale that considers some non-identical strings as equal, you may need to REINDEX to fix existing indexes on textual columns.
•
Repair corner-case bugs in VACUUM FULL (Tom) A potential deadlock between concurrent VACUUM FULL operations on different system catalogs was introduced in 8.2. This has now been corrected. 8.3 made this worse because the deadlock could occur within a critical code section, making it a PANIC rather than just ERROR condition. Also, a VACUUM FULL that failed partway through vacuuming a system catalog could result in cache corruption in concurrent database sessions. Another VACUUM FULL bug introduced in 8.3 could result in a crash or out-of-memory report when dealing with pages containing no live tuples.
•
Fix misbehavior of foreign key checks involving character or bit columns (Tom) If the referencing column were of a different but compatible type (for instance varchar), the constraint was enforced incorrectly.
Fix possible core dump when re-planning a prepared query (Tom) This bug affected only protocol-level prepare operations, not SQL PREPARE, and so tended to be seen only with JDBC, DBI, and other client-side drivers that use prepared statements heavily.
•
Fix possible failure when re-planning a query that calls an SPI-using function (Tom)
•
Fix failure in row-wise comparisons involving columns of different datatypes (Tom)
•
Fix longstanding LISTEN/NOTIFY race condition (Tom) In rare cases a session that had just executed a LISTEN might not get a notification, even though one would be expected because the concurrent transaction executing NOTIFY was observed to commit later. A side effect of the fix is that a transaction that has executed a not-yet-committed LISTEN command will not see any row in pg_listener for the LISTEN, should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior.
•
Disallow LISTEN and UNLISTEN within a prepared transaction (Tom) This was formerly allowed but trying to do it had various unpleasant consequences, notably that the originating backend could not exit as long as an UNLISTEN remained uncommitted.
•
Disallow dropping a temporary table within a prepared transaction (Heikki) This was correctly disallowed by 8.1, but the check was inadvertently broken in 8.2 and 8.3.
•
Fix rare crash when an error occurs during a query using a hash index (Heikki)
•
Fix incorrect comparison of tsquery values (Teodor)
•
Fix incorrect behavior of LIKE with non-ASCII characters in single-byte encodings (Rolf Jentsch)
•
Disable xmlvalidate (Tom)
2397
Appendix E. Release Notes This function should have been removed before 8.3 release, but was inadvertently left in the source code. It poses a small security risk since unprivileged users could use it to read the first few characters of any file accessible to the server. •
Fix memory leaks in certain usages of set-returning functions (Neil)
•
Make encode(bytea, ’escape’) convert all high-bit-set byte values into \nnn octal escape sequences (Tom) This is necessary to avoid encoding problems when the database encoding is multi-byte. This change could pose compatibility issues for applications that are expecting specific results from encode.
•
Fix input of datetime values for February 29 in years BC (Tom) The former coding was mistaken about which years were leap years.
•
Fix “unrecognized node type” error in some variants of ALTER OWNER (Tom)
•
Avoid tablespace permissions errors in CREATE TABLE LIKE INCLUDING INDEXES (Tom)
•
Ensure pg_stat_activity.waiting flag is cleared when a lock wait is aborted (Tom)
•
Fix handling of process permissions on Windows Vista (Dave, Magnus) In particular, this fix allows starting the server as the Administrator user.
•
Update time zone data files to tzdata release 2008a (in particular, recent Chile changes); adjust timezone abbreviation VET (Venezuela) to mean UTC-4:30, not UTC-4:00 (Tom)
•
Fix ecpg problems with arrays (Michael)
•
Fix pg_ctl to correctly extract the postmaster’s port number from command-line options (Itagaki Takahiro, Tom) Previously, pg_ctl start -w could try to contact the postmaster on the wrong port, leading to bogus reports of startup failure.
•
Use -fwrapv to defend against possible misoptimization in recent gcc versions (Tom) This is known to be necessary when building PostgreSQL with gcc 4.3 or later.
•
Enable building contrib/uuid-ossp with MSVC (Hiroshi Saito)
E.140. Release 8.3 Release date: 2008-02-04
E.140.1. Overview With significant new functionality and performance enhancements, this release represents a major leap forward for PostgreSQL. This was made possible by a growing community that has dramatically accelerated the pace of development. This release adds the following major features: •
Full text search is integrated into the core database system
•
Support for the SQL/XML standard, including new operators and an XML data type
•
Enumerated data types (ENUM)
•
Arrays of composite types
2398
Appendix E. Release Notes •
Universally Unique Identifier (UUID) data type
•
Add control over whether NULLs sort first or last
•
Updatable cursors
•
Server configuration parameters can now be set on a per-function basis
•
User-defined types can now have type modifiers
•
Automatically re-plan cached queries when table definitions change or statistics are updated
•
Numerous improvements in logging and statistics collection
•
Support Security Service Provider Interface (SSPI) for authentication on Windows
•
Support multiple concurrent autovacuum processes, and other autovacuum improvements
•
Allow the whole PostgreSQL distribution to be compiled with Microsoft Visual C++
Major performance improvements are listed below. Most of these enhancements are automatic and do not require user changes or tuning: •
Asynchronous commit delays writes to WAL during transaction commit
•
Checkpoint writes can be spread over a longer time period to smooth the I/O spike during each checkpoint
•
Heap-Only Tuples (HOT) accelerate space reuse for most UPDATEs and DELETEs
•
Just-in-time background writer strategy improves disk write efficiency
•
Using non-persistent transaction IDs for read-only transactions reduces overhead and VACUUM requirements
•
Per-field and per-row storage overhead has been reduced
•
Large sequential scans no longer force out frequently used cached pages
•
Concurrent large sequential scans can now share disk reads
• ORDER BY ... LIMIT
can be done without sorting
The above items are explained in more detail in the sections below.
E.140.2. Migration to Version 8.3 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities:
E.140.2.1. General •
Non-character data types are no longer automatically cast to TEXT (Peter, Tom) Previously, if a non-character value was supplied to an operator or function that requires text input, it was automatically cast to text, for most (though not all) built-in data types. This no longer happens: an explicit cast to text is now required for all non-character-string types. For example, these expressions formerly worked: substr(current_date, 1, 4) 23 LIKE ’2%’
2399
Appendix E. Release Notes but will now draw “function does not exist” and “operator does not exist” errors respectively. Use an explicit cast instead: substr(current_date::text, 1, 4) 23::text LIKE ’2%’
(Of course, you can use the more verbose CAST() syntax too.) The reason for the change is that these automatic casts too often caused surprising behavior. An example is that in previous releases, this expression was accepted but did not do what was expected: current_date < 2017-11-17
This is actually comparing a date to an integer, which should be (and now is) rejected — but in the presence of automatic casts both sides were cast to text and a textual comparison was done, because the text < text operator was able to match the expression when no other < operator could. Types char(n) and varchar(n) still cast to text automatically. Also, automatic casting to text still works for inputs to the concatenation (||) operator, so long as least one input is a characterstring type. •
Full text search features from contrib/tsearch2 have been moved into the core server, with some minor syntax changes contrib/tsearch2 now contains a compatibility interface.
• ARRAY(SELECT ...), where the SELECT returns no rows, now returns an empty array, rather than
NULL (Tom) •
The array type name for a base data type is no longer always the base type’s name with an underscore prefix The old naming convention is still honored when possible, but application code should no longer depend on it. Instead use the new pg_type.typarray column to identify the array data type associated with a given type.
• ORDER BY ... USING operator
must now use a less-than or greater-than operator that is
defined in a btree operator class This restriction was added to prevent inconsistent results. • SET LOCAL
changes now persist until the end of the outermost transaction, unless rolled back
(Tom) Previously SET LOCAL’s effects were lost after subtransaction commit (RELEASE SAVEPOINT or exit from a PL/pgSQL exception block). •
Commands rejected in transaction blocks are now also rejected in multiple-statement query strings (Tom) For example, "BEGIN; DROP DATABASE; COMMIT" will now be rejected even if submitted as a single query message.
• ROLLBACK •
outside a transaction block now issues NOTICE instead of WARNING (Bruce)
Prevent NOTIFY/LISTEN/UNLISTEN from accepting schema-qualified names (Bruce) Formerly, these commands accepted schema.relation but ignored the schema part, which was confusing.
• ALTER SEQUENCE •
no longer affects the sequence’s currval() state (Tom)
Foreign keys now must match indexable conditions for cross-data-type references (Tom) This improves semantic consistency and helps avoid performance problems.
2400
Appendix E. Release Notes •
Restrict object size functions to users who have reasonable permissions to view such information (Tom) For example, pg_database_size() now requires CONNECT permission, which is granted to everyone by default. pg_tablespace_size() requires CREATE permission in the tablespace, or is allowed if the tablespace is the default tablespace for the database.
•
Remove the undocumented !!= (not in) operator (Tom) NOT IN (SELECT ...) is the proper way to perform this operation.
•
Internal hashing functions are now more uniformly-distributed (Tom) If application code was computing and storing hash values using internal PostgreSQL hashing functions, the hash values must be regenerated.
•
C-code conventions for handling variable-length data values have changed (Greg Stark, Tom) The new SET_VARSIZE() macro must be used to set the length of generated varlena values. Also, it might be necessary to expand (“de-TOAST”) input values in more cases.
•
Continuous archiving no longer reports each successful archive operation to the server logs unless DEBUG level is used (Simon)
E.140.2.2. Configuration Parameters •
Numerous changes in administrative server parameters bgwriter_lru_percent, bgwriter_all_percent, bgwriter_all_maxpages, stats_start_collector, and stats_reset_on_server_start are removed. redirect_stderr is renamed to logging_collector. stats_command_string is renamed to track_activities. stats_block_level and stats_row_level are merged into track_counts. A new boolean configuration parameter, archive_mode, controls archiving.
Autovacuum’s default settings have changed. •
Remove stats_start_collector parameter (Tom) We now always start the collector process, unless UDP socket creation fails.
•
Remove stats_reset_on_server_start parameter (Tom) This was removed because pg_stat_reset() can be used for this purpose.
•
Commenting out a parameter in postgresql.conf now causes it to revert to its default value (Joachim Wieland) Previously, commenting out an entry left the parameter’s value unchanged until the next server restart.
E.140.2.3. Character Encodings •
Add more checks for invalidly-encoded data (Andrew) This change plugs some holes that existed in literal backslash escape string processing and COPY escape processing. Now the de-escaped string is rechecked to see if the result created an invalid multi-byte character.
•
Disallow database encodings that are inconsistent with the server’s locale setting (Tom)
2401
Appendix E. Release Notes On most platforms, C locale is the only locale that will work with any database encoding. Other locale settings imply a specific encoding and will misbehave if the database encoding is something different. (Typical symptoms include bogus textual sort order and wrong results from upper() or lower().) The server now rejects attempts to create databases that have an incompatible encoding. •
Ensure that chr() cannot create invalidly-encoded values (Andrew) In UTF8-encoded databases the argument of chr() is now treated as a Unicode code point. In other multi-byte encodings chr()’s argument must designate a 7-bit ASCII character. Zero is no longer accepted. ascii() has been adjusted to match.
•
Adjust convert() behavior to ensure encoding validity (Andrew) The two argument form of convert() has been removed. The three argument form now takes a bytea first argument and returns a bytea. To cover the loss of functionality, three new functions have been added: •
convert_from(bytea, name) returns text — converts the first argument from the named
encoding to the database encoding •
convert_to(text, name) returns bytea — converts the first argument from the database
encoding to the named encoding •
length(bytea, name) returns integer — gives the length of the first argument in characters
in the named encoding
•
Remove convert(argument USING conversion_name) (Andrew) Its behavior did not match the SQL standard.
•
Make JOHAB encoding client-only (Tatsuo) JOHAB is not safe as a server-side encoding.
E.140.3. Changes Below you will find a detailed account of the changes between PostgreSQL 8.3 and the previous major release.
E.140.3.1. Performance •
Asynchronous commit delays writes to WAL during transaction commit (Simon) This feature dramatically increases performance for short data-modifying transactions. The disadvantage is that because disk writes are delayed, if the database or operating system crashes before data is written to the disk, committed data will be lost. This feature is useful for applications that can accept some data loss. Unlike turning off fsync, using asynchronous commit does not put database consistency at risk; the worst case is that after a crash the last few reportedly-committed transactions might not be committed after all. This feature is enabled by turning off synchronous_commit (which can be done per-session or per-transaction, if some transactions are critical and others are not). wal_writer_delay can be adjusted to control the maximum delay before transactions actually reach disk.
•
Checkpoint writes can be spread over a longer time period to smooth the I/O spike during each checkpoint (Itagaki Takahiro and Heikki Linnakangas)
2402
Appendix E. Release Notes Previously all modified buffers were forced to disk as quickly as possible during a checkpoint, causing an I/O spike that decreased server performance. This new approach spreads out disk writes during checkpoints, reducing peak I/O usage. (User-requested and shutdown checkpoints are still written as quickly as possible.) •
Heap-Only Tuples (HOT) accelerate space reuse for most UPDATEs and DELETEs (Pavan Deolasee, with ideas from many others) UPDATEs and DELETEs leave dead tuples behind, as do failed INSERTs. Previously only VACUUM
could reclaim space taken by dead tuples. With HOT dead tuple space can be automatically reclaimed at the time of INSERT or UPDATE if no changes are made to indexed columns. This allows for more consistent performance. Also, HOT avoids adding duplicate index entries. •
Just-in-time background writer strategy improves disk write efficiency (Greg Smith, Itagaki Takahiro) This greatly reduces the need for manual tuning of the background writer.
•
Per-field and per-row storage overhead have been reduced (Greg Stark, Heikki Linnakangas) Variable-length data types with data values less than 128 bytes long will see a storage decrease of 3 to 6 bytes. For example, two adjacent char(1) fields now use 4 bytes instead of 16. Row headers are also 4 bytes shorter than before.
•
Using non-persistent transaction IDs for read-only transactions reduces overhead and VACUUM requirements (Florian Pflug) Non-persistent transaction IDs do not increment the global transaction counter. Therefore, they reduce the load on pg_clog and increase the time between forced vacuums to prevent transaction ID wraparound. Other performance improvements were also made that should improve concurrency.
•
Avoid incrementing the command counter after a read-only command (Tom) There was formerly a hard limit of 232 (4 billion) commands per transaction. Now only commands that actually changed the database count, so while this limit still exists, it should be significantly less annoying.
•
Create a dedicated WAL writer process to off-load work from backends (Simon)
•
Skip unnecessary WAL writes for CLUSTER and COPY (Simon) Unless WAL archiving is enabled, the system now avoids WAL writes for CLUSTER and just fsync()s the table at the end of the command. It also does the same for COPY if the table was created in the same transaction.
•
Large sequential scans no longer force out frequently used cached pages (Simon, Heikki, Tom)
•
Concurrent large sequential scans can now share disk reads (Jeff Davis) This is accomplished by starting the new sequential scan in the middle of the table (where another sequential scan is already in-progress) and wrapping around to the beginning to finish. This can affect the order of returned rows in a query that does not specify ORDER BY. The synchronize_seqscans configuration parameter can be used to disable this if necessary.
• ORDER BY ... LIMIT
can be done without sorting (Greg Stark)
This is done by sequentially scanning the table and tracking just the “top N” candidate rows, rather than performing a full sort of the entire table. This is useful when there is no matching index and the LIMIT is not large. •
Put a rate limit on messages sent to the statistics collector by backends (Tom) This reduces overhead for short transactions, but might sometimes increase the delay before statistics are tallied.
2403
Appendix E. Release Notes •
Improve hash join performance for cases with many NULLs (Tom)
•
Speed up operator lookup for cases with non-exact datatype matches (Tom)
E.140.3.2. Server •
Autovacuum is now enabled by default (Alvaro) Several changes were made to eliminate disadvantages of having autovacuum enabled, thereby justifying the change in default. Several other autovacuum parameter defaults were also modified.
•
Support multiple concurrent autovacuum processes (Alvaro, Itagaki Takahiro) This allows multiple vacuums to run concurrently. This prevents vacuuming of a large table from delaying vacuuming of smaller tables.
•
Automatically re-plan cached queries when table definitions change or statistics are updated (Tom) Previously PL/pgSQL functions that referenced temporary tables would fail if the temporary table was dropped and recreated between function invocations, unless EXECUTE was used. This improvement fixes that problem and many related issues.
•
Add a temp_tablespaces parameter to control the tablespaces for temporary tables and files (Jaime Casanova, Albert Cervera, Bernd Helmle) This parameter defines a list of tablespaces to be used. This enables spreading the I/O load across multiple tablespaces. A random tablespace is chosen each time a temporary object is created. Temporary files are no longer stored in per-database pgsql_tmp/ directories but in per-tablespace directories.
•
Place temporary tables’ TOAST tables in special schemas named pg_toast_temp_nnn (Tom) This allows low-level code to recognize these tables as temporary, which enables various optimizations such as not WAL-logging changes and using local rather than shared buffers for access. This also fixes a bug wherein backends unexpectedly held open file references to temporary TOAST tables.
•
Fix problem that a constant flow of new connection requests could indefinitely delay the postmaster from completing a shutdown or a crash restart (Tom)
•
Guard against a very-low-probability data loss scenario by preventing re-use of a deleted table’s relfilenode until after the next checkpoint (Heikki)
•
Fix CREATE CONSTRAINT TRIGGER to convert old-style foreign key trigger definitions into regular foreign key constraints (Tom) This will ease porting of foreign key constraints carried forward from pre-7.3 databases, if they were never converted using contrib/adddepend.
•
Fix DEFAULT NULL to override inherited defaults (Tom) DEFAULT NULL was formerly considered a noise phrase, but it should (and now does) override
non-null defaults that would otherwise be inherited from a parent table or domain. •
Add new encodings EUC_JIS_2004 and SHIFT_JIS_2004 (Tatsuo) These new encodings can be converted to and from UTF-8.
•
Change server startup log message from “database system is ready” to “database system is ready to accept connections”, and adjust its timing The message now appears only when the postmaster is really ready to accept connections.
2404
Appendix E. Release Notes
E.140.3.3. Monitoring •
Add log_autovacuum_min_duration parameter to support configurable logging of autovacuum activity (Simon, Alvaro)
•
Add log_lock_waits parameter to log lock waiting (Simon)
•
Add log_temp_files parameter to log temporary file usage (Bill Moran)
•
Add log_checkpoints parameter to improve logging of checkpoints (Greg Smith, Heikki)
• log_line_prefix
now supports %s and %c escapes in all processes (Andrew)
Previously these escapes worked only for user sessions, not for background database processes. •
Add log_restartpoints to control logging of point-in-time recovery restart points (Simon)
•
Last transaction end time is now logged at end of recovery and at each logged restart point (Simon)
•
Autovacuum now reports its activity start time in pg_stat_activity (Tom)
•
Allow server log output in comma-separated value (CSV) format (Arul Shaji, Greg Smith, Andrew Dunstan) CSV-format log files can easily be loaded into a database table for subsequent analysis.
•
Use PostgreSQL-supplied timezone support for formatting timestamps displayed in the server log (Tom) This avoids Windows-specific problems with localized time zone names that are in the wrong encoding. There is a new log_timezone parameter that controls the timezone used in log messages, independently of the client-visible timezone parameter.
•
New system view pg_stat_bgwriter displays statistics about background writer activity (Magnus)
•
Add new columns for database-wide tuple statistics to pg_stat_database (Magnus)
•
Add an xact_start (transaction start time) column to pg_stat_activity (Neil) This makes it easier to identify long-running transactions.
•
Add n_live_tuples and n_dead_tuples columns to pg_stat_all_tables and related views (Glen Parker)
•
Merge stats_block_level and stats_row_level parameters into a single parameter track_counts, which controls all messages sent to the statistics collector process (Tom)
•
Rename stats_command_string parameter to track_activities (Tom)
•
Fix statistical counting of live and dead tuples to recognize that committed and aborted transactions have different effects (Tom)
E.140.3.4. Authentication •
Support Security Service Provider Interface (SSPI) for authentication on Windows (Magnus)
•
Support GSSAPI authentication (Henry Hotz, Magnus) This should be preferred to native Kerberos authentication because GSSAPI is an industry standard.
•
Support a global SSL configuration file (Victor Wagner)
•
Add ssl_ciphers parameter to control accepted SSL ciphers (Victor Wagner)
2405
Appendix E. Release Notes •
Add a Kerberos realm parameter, krb_realm (Magnus)
E.140.3.5. Write-Ahead Log (WAL) and Continuous Archiving •
Change the timestamps recorded in transaction WAL records from time_t to TimestampTz representation (Tom) This provides sub-second resolution in WAL, which can be useful for point-in-time recovery.
•
Reduce WAL disk space needed by warm standby servers (Simon) This change allows a warm standby server to pass the name of the earliest still-needed WAL file to the recovery script, allowing automatic removal of no-longer-needed WAL files. This is done using %r in the restore_command parameter of recovery.conf.
•
New boolean configuration parameter, archive_mode, controls archiving (Simon) Previously setting archive_command to an empty string turned off archiving. Now archive_mode turns archiving on and off, independently of archive_command. This is useful for stopping archiving temporarily.
E.140.3.6. Queries •
Full text search is integrated into the core database system (Teodor, Oleg) Text search has been improved, moved into the core code, and is now installed by default. contrib/tsearch2 now contains a compatibility interface.
•
Add control over whether NULLs sort first or last (Teodor, Tom) The syntax is ORDER BY ... NULLS FIRST/LAST.
•
Allow per-column ascending/descending (ASC/DESC) ordering options for indexes (Teodor, Tom) Previously a query using ORDER BY with mixed ASC/DESC specifiers could not fully use an index. Now an index can be fully used in such cases if the index was created with matching ASC/DESC specifications. NULL sort order within an index can be controlled, too.
•
Allow col IS NULL to use an index (Teodor)
•
Updatable cursors (Arul Shaji, Tom) This eliminates the need to reference a primary key to UPDATE or DELETE rows returned by a cursor. The syntax is UPDATE/DELETE WHERE CURRENT OF.
• •
Allow FOR UPDATE in cursors (Arul Shaji, Tom) Create a general mechanism that supports casts to and from the standard string types (TEXT, VARCHAR, CHAR) for every datatype, by invoking the datatype’s I/O functions (Tom)
Previously, such casts were available only for types that had specialized function(s) for the purpose. These new casts are assignment-only in the to-string direction, explicit-only in the other direction, and therefore should create no surprising behavior. •
Allow UNION and related constructs to return a domain type, when all inputs are of that domain type (Tom) Formerly, the output would be considered to be of the domain’s base type.
•
Allow limited hashing when using two different data types (Tom)
2406
Appendix E. Release Notes This allows hash joins, hash indexes, hashed subplans, and hash aggregation to be used in situations involving cross-data-type comparisons, if the data types have compatible hash functions. Currently, cross-data-type hashing support exists for smallint/integer/bigint, and for float4/float8. •
Improve optimizer logic for detecting when variables are equal in a WHERE clause (Tom) This allows mergejoins to work with descending sort orders, and improves recognition of redundant sort columns.
•
Improve performance when planning large inheritance trees in cases where most tables are excluded by constraints (Tom)
E.140.3.7. Object Manipulation •
Arrays of composite types (David Fetter, Andrew, Tom) In addition to arrays of explicitly-declared composite types, arrays of the rowtypes of regular tables and views are now supported, except for rowtypes of system catalogs, sequences, and TOAST tables.
•
Server configuration parameters can now be set on a per-function basis (Tom) For example, functions can now set their own search_path to prevent unexpected behavior if a different search_path exists at run-time. Security definer functions should set search_path to avoid security loopholes.
• CREATE/ALTER FUNCTION
now supports COST and ROWS options (Tom)
COST allows specification of the cost of a function call. ROWS allows specification of the average
number or rows returned by a set-returning function. These values are used by the optimizer in choosing the best plan. •
Implement CREATE TABLE LIKE ... INCLUDING INDEXES (Trevor Hardcastle, Nikhil Sontakke, Neil)
•
Allow CREATE INDEX CONCURRENTLY to ignore transactions in other databases (Simon)
•
Add ALTER VIEW ... RENAME TO and ALTER SEQUENCE ... RENAME TO (David Fetter, Neil) Previously this could only be done via ALTER TABLE ... RENAME TO.
•
Make CREATE/DROP/RENAME DATABASE wait briefly for conflicting backends to exit before failing (Tom) This increases the likelihood that these commands will succeed.
•
Allow triggers and rules to be deactivated in groups using a configuration parameter, for replication purposes (Jan) This allows replication systems to disable triggers and rewrite rules as a group without modifying the system catalogs directly. The behavior is controlled by ALTER TABLE and a new parameter session_replication_role.
•
User-defined types can now have type modifiers (Teodor, Tom) This allows a user-defined type to take a modifier, like ssnum(7). Previously only built-in data types could have modifiers.
2407
Appendix E. Release Notes
E.140.3.8. Utility Commands •
Non-superuser database owners now are able to add trusted procedural languages to their databases by default (Jeremy Drake) While this is reasonably safe, some administrators might wish to revoke the privilege. It is controlled by pg_pltemplate.tmpldbacreate.
•
Allow a session’s current parameter setting to be used as the default for future sessions (Tom) This is done with SET ... FROM CURRENT in CREATE/ALTER FUNCTION, ALTER DATABASE, or ALTER ROLE.
•
Implement new commands DISCARD ALL, DISCARD PLANS, DISCARD TEMPORARY, CLOSE ALL, and DEALLOCATE ALL (Marko Kreen, Neil) These commands simplify resetting a database session to its initial state, and are particularly useful for connection-pooling software.
•
Make CLUSTER MVCC-safe (Heikki Linnakangas) Formerly, CLUSTER would discard all tuples that were committed dead, even if there were still transactions that should be able to see them under MVCC visibility rules.
•
Add new CLUSTER syntax: CLUSTER table USING index (Holger Schurig) The old CLUSTER syntax is still supported, but the new form is considered more logical.
•
Fix EXPLAIN so it can show complex plans more accurately (Tom) References to subplan outputs are now always shown correctly, instead of using ?columnN ? for complicated cases.
•
Limit the amount of information reported when a user is dropped (Alvaro) Previously, dropping (or attempting to drop) a user who owned many objects could result in large NOTICE or ERROR messages listing all these objects; this caused problems for some client applications. The length of the message is now limited, although a full list is still sent to the server log.
E.140.3.9. Data Types •
Support for the SQL/XML standard, including new operators and an XML data type (Nikolay Samokhvalov, Pavel Stehule, Peter)
•
Enumerated data types (ENUM) (Tom Dunstan) This feature provides convenient support for fields that have a small, fixed set of allowed values. An example of creating an ENUM type is CREATE TYPE mood AS ENUM (’sad’, ’ok’, ’happy’).
•
Universally Unique Identifier (UUID) data type (Gevik Babakhani, Neil) This closely matches RFC 4122.
•
Widen the MONEY data type to 64 bits (D’Arcy Cain) This greatly increases the range of supported MONEY values.
•
Fix float4/float8 to handle Infinity and NAN (Not A Number) consistently (Bruce) The code formerly was not consistent about distinguishing Infinity from overflow conditions.
2408
Appendix E. Release Notes •
Allow leading and trailing whitespace during input of boolean values (Neil)
•
Prevent COPY from using digits and lowercase letters as delimiters (Tom)
E.140.3.10. Functions •
Add new regular expression functions regexp_matches(), regexp_split_to_array(), and regexp_split_to_table() (Jeremy Drake, Neil) These functions provide extraction of regular expression subexpressions and allow splitting a string using a POSIX regular expression.
•
Add lo_truncate() for large object truncation (Kris Jurka)
•
Implement width_bucket() for the float8 data type (Neil)
•
Add pg_stat_clear_snapshot() to discard statistics snapshots collected during the current transaction (Tom) The first request for statistics in a transaction takes a statistics snapshot that does not change during the transaction. This function allows the snapshot to be discarded and a new snapshot loaded during the next statistics query. This is particularly useful for PL/pgSQL functions, which are confined to a single transaction.
•
Add isodow option to EXTRACT() and date_part() (Bruce) This returns the day of the week, with Sunday as seven. (dow returns Sunday as zero.)
•
Add ID (ISO day of week) and IDDD (ISO day of year) format codes for to_char(), to_date(), and to_timestamp() (Brendan Jurd)
•
Make to_timestamp() and to_date() assume TM (trim) option for potentially variable-width fields (Bruce) This matches Oracle’s behavior.
•
Fix off-by-one conversion error in to_date()/to_timestamp() D (non-ISO day of week) fields (Bruce)
•
Make setseed() return void, rather than a useless integer value (Neil)
•
Add a hash function for NUMERIC (Neil) This allows hash indexes and hash-based plans to be used with NUMERIC columns.
•
Improve efficiency of LIKE/ILIKE, especially for multi-byte character sets like UTF-8 (Andrew, Itagaki Takahiro)
•
Make currtid() functions require SELECT privileges on the target table (Tom)
•
Add several txid_*() functions to query active transaction IDs (Jan) This is useful for various replication solutions.
E.140.3.11. PL/pgSQL Server-Side Language •
Add scrollable cursor support, including directional control in FETCH (Pavel Stehule)
•
Allow IN as an alternative to FROM in PL/pgSQL’s FETCH statement, for consistency with the backend’s FETCH command (Pavel Stehule)
2409
Appendix E. Release Notes •
Add MOVE to PL/pgSQL (Magnus, Pavel Stehule, Neil)
•
Implement RETURN QUERY (Pavel Stehule, Neil) This adds convenient syntax for PL/pgSQL set-returning functions that want to return the result of a query. RETURN QUERY is easier and more efficient than a loop around RETURN NEXT.
•
Allow function parameter names to be qualified with the function’s name (Tom) For example, myfunc.myvar. This is particularly useful for specifying variables in a query where the variable name might match a column name.
•
Make qualification of variables with block labels work properly (Tom) Formerly, outer-level block labels could unexpectedly interfere with recognition of inner-level record or row references.
•
Tighten requirements for FOR loop STEP values (Tom) Prevent non-positive STEP values, and handle loop overflows.
•
Improve accuracy when reporting syntax error locations (Tom)
E.140.3.12. Other Server-Side Languages •
Allow type-name arguments to PL/Perl spi_prepare() to be data type aliases in addition to names found in pg_type (Andrew)
•
Allow type-name arguments to PL/Python plpy.prepare() to be data type aliases in addition to names found in pg_type (Andrew)
•
Allow type-name arguments to PL/Tcl spi_prepare to be data type aliases in addition to names found in pg_type (Andrew)
•
Enable PL/PythonU to compile on Python 2.5 (Marko Kreen)
•
Support a true PL/Python boolean type in compatible Python versions (Python 2.3 and later) (Marko Kreen)
•
Fix PL/Tcl problems with thread-enabled libtcl spawning multiple threads within the backend (Steve Marshall, Paul Bayer, Doug Knight) This caused all sorts of unpleasantness.
E.140.3.13. psql •
List disabled triggers separately in \d output (Brendan Jurd)
•
In \d patterns, always match $ literally (Tom)
•
Show aggregate return types in \da output (Greg Sabino Mullane)
•
Add the function’s volatility status to the output of \df+ (Neil)
•
Add \prompt capability (Chad Wagner)
•
Allow \pset, \t, and \x to specify on or off, rather than just toggling (Chad Wagner)
•
Add \sleep capability (Jan)
•
Enable \timing output for \copy (Andrew)
•
Improve \timing resolution on Windows (Itagaki Takahiro)
2410
Appendix E. Release Notes •
Flush \o output after each backslash command (Tom)
•
Correctly detect and report errors while reading a -f input file (Peter)
•
Remove -u option (this option has long been deprecated) (Tom)
E.140.3.14. pg_dump •
Add --tablespaces-only and --roles-only options to pg_dumpall (Dave Page)
•
Add an output file option to pg_dumpall (Dave Page) This is primarily useful on Windows, where output redirection of child pg_dump processes does not work.
•
Allow pg_dumpall to accept an initial-connection database name rather than the default template1 (Dave Page)
•
In -n and -t switches, always match $ literally (Tom)
•
Improve performance when a database has thousands of objects (Tom)
•
Remove -u option (this option has long been deprecated) (Tom)
E.140.3.15. Other Client Applications •
In initdb, allow the location of the pg_xlog directory to be specified (Euler Taveira de Oliveira)
•
Enable server core dump generation in pg_regress on supported operating systems (Andrew)
•
Add a -t (timeout) parameter to pg_ctl (Bruce) This controls how long pg_ctl will wait when waiting for server startup or shutdown. Formerly the timeout was hard-wired as 60 seconds.
•
Add a pg_ctl option to control generation of server core dumps (Andrew)
•
Allow Control-C to cancel clusterdb, reindexdb, and vacuumdb (Itagaki Takahiro, Magnus)
•
Suppress command tag output for createdb, createuser, dropdb, and dropuser (Peter) The --quiet option is ignored and will be removed in 8.4. Progress messages when acting on all databases now go to stdout instead of stderr because they are not actually errors.
E.140.3.16. libpq •
Interpret the dbName parameter of PQsetdbLogin() as a conninfo string if it contains an equals sign (Andrew) This allows use of conninfo strings in client programs that still use PQsetdbLogin().
•
Support a global SSL configuration file (Victor Wagner)
•
Add environment variable PGSSLKEY to control SSL hardware keys (Victor Wagner)
•
Add lo_truncate() for large object truncation (Kris Jurka)
•
Add PQconnectionNeedsPassword() that returns true if the server required a password but none was supplied (Joe Conway, Tom)
2411
Appendix E. Release Notes If this returns true after a failed connection attempt, a client application should prompt the user for a password. In the past applications have had to check for a specific error message string to decide whether a password is needed; that approach is now deprecated. •
Add PQconnectionUsedPassword() that returns true if the supplied password was actually used (Joe Conway, Tom) This is useful in some security contexts where it is important to know whether a user-supplied password is actually valid.
E.140.3.17. ecpg •
Use V3 frontend/backend protocol (Michael) This adds support for server-side prepared statements.
•
Use native threads, instead of pthreads, on Windows (Magnus)
•
Improve thread-safety of ecpglib (Itagaki Takahiro)
•
Make the ecpg libraries export only necessary API symbols (Michael)
E.140.3.18. Windows Port •
Allow the whole PostgreSQL distribution to be compiled with Microsoft Visual C++ (Magnus and others) This allows Windows-based developers to use familiar development and debugging tools. Windows executables made with Visual C++ might also have better stability and performance than those made with other tool sets. The client-only Visual C++ build scripts have been removed.
•
Drastically reduce postmaster’s memory usage when it has many child processes (Magnus)
•
Allow regression tests to be started by an administrative user (Magnus)
•
Add native shared memory implementation (Magnus)
E.140.3.19. Server Programming Interface (SPI) •
Add cursor-related functionality in SPI (Pavel Stehule) Allow access to the cursor-related planning options, and add FETCH/MOVE routines.
•
Allow execution of cursor commands through SPI_execute (Tom) The macro SPI_ERROR_CURSOR still exists but will never be returned.
•
SPI plan pointers are now declared as SPIPlanPtr instead of void * (Tom) This does not break application code, but switching is recommended to help catch simple programming mistakes.
2412
Appendix E. Release Notes
E.140.3.20. Build Options •
Add configure option --enable-profiling to enable code profiling (works only with gcc) (Korry Douglas and Nikhil Sontakke)
•
Add configure option --with-system-tzdata to use the operating system’s time zone database (Peter)
•
Fix PGXS so extensions can be built against PostgreSQL installations whose pg_config program does not appear first in the PATH (Tom)
•
Support gmake draft when building the SGML documentation (Bruce) Unless draft is used, the documentation build will now be repeated if necessary to ensure the index is up-to-date.
E.140.3.21. Source Code •
Rename macro DLLIMPORT to PGDLLIMPORT to avoid conflicting with third party includes (like Tcl) that define DLLIMPORT (Magnus)
•
Create “operator families” to improve planning of queries involving cross-data-type comparisons (Tom)
•
Update GIN extractQuery() API to allow signalling that nothing can satisfy the query (Teodor)
•
Move NAMEDATALEN definition from postgres_ext.h to pg_config_manual.h (Peter)
•
Provide strlcpy() and strlcat() on all platforms, and replace error-prone uses of strncpy(), strncat(), etc (Peter)
•
Create hooks to let an external plugin monitor (or even replace) the planner and create plans for hypothetical situations (Gurjeet Singh, Tom)
•
Create a function variable join_search_hook to let plugins override the join search order portion of the planner (Julius Stroffek)
•
Add tas() support for Renesas’ M32R processor (Kazuhiro Inaoka)
• quote_identifier()
and pg_dump no longer quote keywords that are unreserved according to
the grammar (Tom) •
Change the on-disk representation of the NUMERIC data type so that the sign_dscale word comes before the weight (Tom)
•
Use SYSV semaphores rather than POSIX on Darwin >= 6.0, i.e., OS X 10.2 and up (Chris Marcellino)
•
Add acronym and NFS documentation sections (Bruce)
•
"Postgres" is now documented as an accepted alias for "PostgreSQL" (Peter)
•
Add documentation about preventing database server spoofing when the server is down (Bruce)
E.140.3.22. Contrib •
Move contrib README content into the main PostgreSQL documentation (Albert Cervera i Areny)
•
Add contrib/pageinspect module for low-level page inspection (Simon, Heikki)
2413
Appendix E. Release Notes •
Add contrib/pg_standby module for controlling warm standby operation (Simon)
•
Add contrib/uuid-ossp module for generating UUID values using the OSSP UUID library (Peter) Use configure --with-ossp-uuid to activate. This takes advantage of the new UUID builtin type.
•
Add contrib/dict_int, contrib/dict_xsyn, and contrib/test_parser modules to provide sample add-on text search dictionary templates and parsers (Sergey Karpov)
•
Allow contrib/pgbench to set the fillfactor (Pavan Deolasee)
•
Add timestamps to contrib/pgbench -l (Greg Smith)
•
Add usage count statistics to contrib/pgbuffercache (Greg Smith)
•
Add GIN support for contrib/hstore (Teodor)
•
Add GIN support for contrib/pg_trgm (Guillaume Smet, Teodor)
•
Update OS/X startup scripts in contrib/start-scripts (Mark Cotner, David Fetter)
•
Restrict pgrowlocks() and dblink_get_pkey() to users who have SELECT privilege on the target table (Tom)
•
Restrict contrib/pgstattuple functions to superusers (Tom)
• contrib/xml2
is deprecated and planned for removal in 8.4 (Peter)
The new XML support in core PostgreSQL supersedes this module.
E.141. Release 8.2.23 Release date: 2011-12-05 This release contains a variety of fixes from 8.2.22. For information about new features in the 8.2 major release, see Section E.164. This is expected to be the last PostgreSQL release in the 8.2.X series. Users are encouraged to update to a newer release branch soon.
E.141.1. Migration to Version 8.2.23 A dump/restore is not required for those running 8.2.X. However,
a
longstanding
error
was
discovered in the definition of the information_schema.referential_constraints view. If you rely on correct results from that view, you should replace its definition as explained in the first changelog item below. Also, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.141.2. Changes •
Fix bugs in information_schema.referential_constraints view (Tom Lane) This view was being insufficiently careful about matching the foreign-key constraint to the depended-on primary or unique key constraint. That could result in failure to show a foreign key
2414
Appendix E. Release Notes constraint at all, or showing it multiple times, or claiming that it depends on a different constraint than the one it really does. Since the view definition is installed by initdb, merely upgrading will not fix the problem. If you need to fix this in an existing installation, you can (as a superuser) drop the information_schema schema then re-create it by sourcing SHAREDIR/information_schema.sql. (Run pg_config --sharedir if you’re uncertain where SHAREDIR is.) This must be repeated in each database to be fixed. •
Fix TOAST-related data corruption during CREATE TABLE dest AS SELECT * FROM src or INSERT INTO dest SELECT * FROM src (Tom Lane) If a table has been modified by ALTER TABLE ADD COLUMN, attempts to copy its data verbatim to another table could produce corrupt results in certain corner cases. The problem can only manifest in this precise form in 8.4 and later, but we patched earlier versions as well in case there are other code paths that could trigger the same bug.
•
Fix race condition during toast table access from stale syscache entries (Tom Lane) The typical symptom was transient errors like “missing chunk number 0 for toast value NNNNN in pg_toast_2619”, where the cited toast table would always belong to a system catalog.
•
Improve locale support in money type’s input and output (Tom Lane) Aside from not supporting all standard lc_monetary formatting options, the input and output functions were inconsistent, meaning there were locales in which dumped money values could not be re-read.
•
Don’t let transform_null_equals affect CASE foo WHEN NULL ... constructs (Heikki Linnakangas) transform_null_equals is only supposed to affect foo = NULL expressions written directly by the user, not equality checks generated internally by this form of CASE.
•
Change foreign-key trigger creation order to better support self-referential foreign keys (Tom Lane) For a cascading foreign key that references its own table, a row update will fire both the ON UPDATE trigger and the CHECK trigger as one event. The ON UPDATE trigger must execute first, else the CHECK will check a non-final state of the row and possibly throw an inappropriate error. However, the firing order of these triggers is determined by their names, which generally sort in creation order since the triggers have auto-generated names following the convention “RI_ConstraintTrigger_NNNN”. A proper fix would require modifying that convention, which we will do in 9.2, but it seems risky to change it in existing releases. So this patch just changes the creation order of the triggers. Users encountering this type of error should drop and re-create the foreign key constraint to get its triggers into the right order.
•
Preserve blank lines within commands in psql’s command history (Robert Haas) The former behavior could cause problems if an empty line was removed from within a string literal, for example.
•
Use the preferred version of xsubpp to build PL/Perl, not necessarily the operating system’s main copy (David Wheeler and Alex Hunsaker)
•
Honor query cancel interrupts promptly in pgstatindex() (Robert Haas)
•
Ensure VPATH builds properly install all server header files (Peter Eisentraut)
•
Shorten file names reported in verbose error messages (Peter Eisentraut) Regular builds have always reported just the name of the C file containing the error message call, but VPATH builds formerly reported an absolute path name.
2415
Appendix E. Release Notes •
Fix interpretation of Windows timezone names for Central America (Tom Lane) Map “Central America Standard Time” to CST6, not CST6CDT, because DST is generally not observed anywhere in Central America.
•
Update time zone data files to tzdata release 2011n for DST law changes in Brazil, Cuba, Fiji, Palestine, Russia, and Samoa; also historical corrections for Alaska and British East Africa.
E.142. Release 8.2.22 Release date: 2011-09-26 This release contains a variety of fixes from 8.2.21. For information about new features in the 8.2 major release, see Section E.164. The PostgreSQL community will stop releasing updates for the 8.2.X release series in December 2011. Users are encouraged to update to a newer release branch soon.
E.142.1. Migration to Version 8.2.22 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.142.2. Changes •
Fix multiple bugs in GiST index page split processing (Heikki Linnakangas) The probability of occurrence was low, but these could lead to index corruption.
•
Avoid possibly accessing off the end of memory in ANALYZE (Noah Misch) This fixes a very-low-probability server crash scenario.
•
Fix race condition in relcache init file invalidation (Tom Lane) There was a window wherein a new backend process could read a stale init file but miss the inval messages that would tell it the data is stale. The result would be bizarre failures in catalog accesses, typically “could not read block 0 in file ...” later during startup.
•
Fix memory leak at end of a GiST index scan (Tom Lane) Commands that perform many separate GiST index scans, such as verification of a new GiSTbased exclusion constraint on a table already containing many rows, could transiently require large amounts of memory due to this leak.
•
Fix performance problem when constructing a large, lossy bitmap (Tom Lane)
•
Fix array- and path-creating functions to ensure padding bytes are zeroes (Tom Lane) This avoids some situations where the planner will think that semantically-equal constants are not equal, resulting in poor optimization.
•
Work around gcc 4.6.0 bug that breaks WAL replay (Tom Lane) This could lead to loss of committed transactions after a server crash.
•
Fix dump bug for VALUES in a view (Tom Lane)
2416
Appendix E. Release Notes •
Disallow SELECT FOR UPDATE/SHARE on sequences (Tom Lane) This operation doesn’t work as expected and can lead to failures.
•
Defend against integer overflow when computing size of a hash table (Tom Lane)
•
Fix portability bugs in use of credentials control messages for “peer” authentication (Tom Lane)
•
Fix typo in pg_srand48 seed initialization (Andres Freund) This led to failure to use all bits of the provided seed. This function is not used on most platforms (only those without srandom), and the potential security exposure from a less-random-thanexpected seed seems minimal in any case.
•
Avoid integer overflow when the sum of LIMIT and OFFSET values exceeds 2^63 (Heikki Linnakangas)
•
Add overflow checks to int4 and int8 versions of generate_series() (Robert Haas)
•
Fix trailing-zero removal in to_char() (Marti Raudsepp) In a format with FM and no digit positions after the decimal point, zeroes to the left of the decimal point could be removed incorrectly.
•
Fix pg_size_pretty() to avoid overflow for inputs close to 2^63 (Tom Lane)
•
Fix psql’s counting of script file line numbers during COPY from a different file (Tom Lane)
•
Fix pg_restore’s direct-to-database mode for standard_conforming_strings (Tom Lane) pg_restore could emit incorrect commands when restoring directly to a database server from an archive file that had been made with standard_conforming_strings set to on.
•
Fix write-past-buffer-end and memory leak in libpq’s LDAP service lookup code (Albe Laurenz)
•
In libpq, avoid failures when using nonblocking I/O and an SSL connection (Martin Pihlak, Tom Lane)
•
Improve libpq’s handling of failures during connection startup (Tom Lane) In particular, the response to a server report of fork() failure during SSL connection startup is now saner.
•
Make ecpglib write double values with 15 digits precision (Akira Kurosawa)
•
Apply upstream fix for blowfish signed-character bug (CVE-2011-2483) (Tom Lane) contrib/pg_crypto’s blowfish encryption code could give wrong results on platforms where
char is signed (which is most), leading to encrypted passwords being weaker than they should be. •
Fix memory leak in contrib/seg (Heikki Linnakangas)
•
Fix pgstatindex() to give consistent results for empty indexes (Tom Lane)
•
Allow building with perl 5.14 (Alex Hunsaker)
•
Update configure script’s method for probing existence of system functions (Tom Lane) The version of autoconf we used in 8.3 and 8.2 could be fooled by compilers that perform link-time optimization.
•
Fix assorted issues with build and install file paths containing spaces (Tom Lane)
•
Update time zone data files to tzdata release 2011i for DST law changes in Canada, Egypt, Russia, Samoa, and South Sudan.
2417
Appendix E. Release Notes
E.143. Release 8.2.21 Release date: 2011-04-18 This release contains a variety of fixes from 8.2.20. For information about new features in the 8.2 major release, see Section E.164.
E.143.1. Migration to Version 8.2.21 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.143.2. Changes •
Avoid potential deadlock during catalog cache initialization (Nikhil Sontakke) In some cases the cache loading code would acquire share lock on a system index before locking the index’s catalog. This could deadlock against processes trying to acquire exclusive locks in the other, more standard order.
•
Fix dangling-pointer problem in BEFORE ROW UPDATE trigger handling when there was a concurrent update to the target tuple (Tom Lane) This bug has been observed to result in intermittent “cannot extract system attribute from virtual tuple” failures while trying to do UPDATE RETURNING ctid. There is a very small probability of more serious errors, such as generating incorrect index entries for the updated tuple.
•
Disallow DROP TABLE when there are pending deferred trigger events for the table (Tom Lane) Formerly the DROP would go through, leading to “could not open relation with OID nnn” errors when the triggers were eventually fired.
Fix pg_restore to cope with long lines (over 1KB) in TOC files (Tom Lane)
•
Put in more safeguards against crashing due to division-by-zero with overly enthusiastic compiler optimization (Aurelien Jarno)
•
Support use of dlopen() in FreeBSD and OpenBSD on MIPS (Tom Lane) There was a hard-wired assumption that this system function was not available on MIPS hardware on these systems. Use a compile-time test instead, since more recent versions have it.
•
Fix compilation failures on HP-UX (Heikki Linnakangas)
•
Fix path separator used by pg_regress on Cygwin (Andrew Dunstan)
•
Update time zone data files to tzdata release 2011f for DST law changes in Chile, Cuba, Falkland Islands, Morocco, Samoa, and Turkey; also historical corrections for South Australia, Alaska, and Hawaii.
E.144. Release 8.2.20 Release date: 2011-01-31
2418
Appendix E. Release Notes This release contains a variety of fixes from 8.2.19. For information about new features in the 8.2 major release, see Section E.164.
E.144.1. Migration to Version 8.2.20 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.144.2. Changes •
Avoid failures when EXPLAIN tries to display a simple-form CASE expression (Tom Lane) If the CASE’s test expression was a constant, the planner could simplify the CASE into a form that confused the expression-display code, resulting in “unexpected CASE WHEN clause” errors.
•
Fix assignment to an array slice that is before the existing range of subscripts (Tom Lane) If there was a gap between the newly added subscripts and the first pre-existing subscript, the code miscalculated how many entries needed to be copied from the old array’s null bitmap, potentially leading to data corruption or crash.
•
Avoid unexpected conversion overflow in planner for very distant date values (Tom Lane) The date type supports a wider range of dates than can be represented by the timestamp types, but the planner assumed it could always convert a date to timestamp with impunity.
•
Fix pg_restore’s text output for large objects (BLOBs) when standard_conforming_strings is on (Tom Lane) Although restoring directly to a database worked correctly, string escaping was incorrect if pg_restore was asked for SQL text output and standard_conforming_strings had been enabled in the source database.
•
Fix erroneous parsing of tsquery values containing ... & !(subexpression) | ... (Tom Lane) Queries containing this combination of operators were not executed correctly. The same error existed in contrib/intarray’s query_int type and contrib/ltree’s ltxtquery type.
•
Fix buffer overrun in contrib/intarray’s input function for the query_int type (Apple) This bug is a security risk since the function’s return address could be overwritten. Thanks to Apple Inc’s security team for reporting this issue and supplying the fix. (CVE-2010-4015)
•
Fix bug in contrib/seg’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a seg column. If you have such an index, consider REINDEXing it after installing this update. (This is identical to the bug that was fixed in contrib/cube in the previous update.)
E.145. Release 8.2.19 Release date: 2010-12-16
2419
Appendix E. Release Notes This release contains a variety of fixes from 8.2.18. For information about new features in the 8.2 major release, see Section E.164.
E.145.1. Migration to Version 8.2.19 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.145.2. Changes •
Force the default wal_sync_method to be fdatasync on Linux (Tom Lane, Marti Raudsepp) The default on Linux has actually been fdatasync for many years, but recent kernel changes caused PostgreSQL to choose open_datasync instead. This choice did not result in any performance improvement, and caused outright failures on certain filesystems, notably ext4 with the data=journal mount option.
•
Fix assorted bugs in WAL replay logic for GIN indexes (Tom Lane) This could result in “bad buffer id: 0” failures or corruption of index contents during replication.
•
Fix recovery from base backup when the starting checkpoint WAL record is not in the same WAL segment as its redo point (Jeff Davis)
•
Add support for detecting register-stack overrun on IA64 (Tom Lane) The IA64 architecture has two hardware stacks. Full prevention of stack-overrun failures requires checking both.
•
Add a check for stack overflow in copyObject() (Tom Lane) Certain code paths could crash due to stack overflow given a sufficiently complex query.
•
Fix detection of page splits in temporary GiST indexes (Heikki Linnakangas) It is possible to have a “concurrent” page split in a temporary index, if for example there is an open cursor scanning the index when an insertion is done. GiST failed to detect this case and hence could deliver wrong results when execution of the cursor continued.
•
Avoid memory leakage while ANALYZE’ing complex index expressions (Tom Lane)
•
Ensure an index that uses a whole-row Var still depends on its table (Tom Lane) An index declared like create index i on t (foo(t.*)) would not automatically get dropped when its table was dropped.
•
Do not “inline” a SQL function with multiple OUT parameters (Tom Lane) This avoids a possible crash due to loss of information about the expected result rowtype.
•
Behave correctly if ORDER BY, LIMIT, FOR UPDATE, or WITH is attached to the VALUES part of INSERT ... VALUES (Tom Lane)
•
Fix constant-folding of COALESCE() expressions (Tom Lane) The planner would sometimes attempt to evaluate sub-expressions that in fact could never be reached, possibly leading to unexpected errors.
•
Add print functionality for InhRelation nodes (Tom Lane)
2420
Appendix E. Release Notes This avoids a failure when debug_print_parse is enabled and certain types of query are executed. •
Fix incorrect calculation of distance from a point to a horizontal line segment (Tom Lane) This bug affected several different geometric distance-measurement operators.
•
Fix PL/pgSQL’s handling of “simple” expressions to not fail in recursion or error-recovery cases (Tom Lane)
•
Fix PL/Python’s handling of set-returning functions (Jan Urbanski) Attempts to call SPI functions within the iterator generating a set result would fail.
•
Fix bug in contrib/cube’s GiST picksplit algorithm (Alexander Korotkov) This could result in considerable inefficiency, though not actually incorrect answers, in a GiST index on a cube column. If you have such an index, consider REINDEXing it after installing this update.
•
Don’t emit “identifier will be truncated” notices in contrib/dblink except when creating new connections (Itagaki Takahiro)
•
Fix potential coredump on missing public key in contrib/pgcrypto (Marti Raudsepp)
•
Fix memory leak in contrib/xml2’s XPath query functions (Tom Lane)
•
Update time zone data files to tzdata release 2010o for DST law changes in Fiji and Samoa; also historical corrections for Hong Kong.
E.146. Release 8.2.18 Release date: 2010-10-04 This release contains a variety of fixes from 8.2.17. For information about new features in the 8.2 major release, see Section E.164.
E.146.1. Migration to Version 8.2.18 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.146.2. Changes •
Use a separate interpreter for each calling SQL userid in PL/Perl and PL/Tcl (Tom Lane) This change prevents security problems that can be caused by subverting Perl or Tcl code that will be executed later in the same session under another SQL user identity (for example, within a SECURITY DEFINER function). Most scripting languages offer numerous ways that that might be done, such as redefining standard functions or operators called by the target function. Without this change, any SQL user with Perl or Tcl language usage rights can do essentially anything with the SQL privileges of the target function’s owner. The cost of this change is that intentional communication among Perl and Tcl functions becomes more difficult. To provide an escape hatch, PL/PerlU and PL/TclU functions continue to use only
2421
Appendix E. Release Notes one interpreter per session. This is not considered a security issue since all such functions execute at the trust level of a database superuser already. It is likely that third-party procedural languages that claim to offer trusted execution have similar security issues. We advise contacting the authors of any PL you are depending on for securitycritical purposes. Our thanks to Tim Bunce for pointing out this issue (CVE-2010-3433). •
Prevent possible crashes in pg_get_expr() by disallowing it from being called with an argument that is not one of the system catalog columns it’s intended to be used with (Heikki Linnakangas, Tom Lane)
•
Fix Windows shared-memory allocation code (Tsutomu Yamada, Magnus Hagander) This bug led to the often-reported “could not reattach to shared memory” error message. This is a back-patch of a fix that was applied to newer branches some time ago.
•
Treat exit code 128 (ERROR_WAIT_NO_CHILDREN) as non-fatal on Windows (Magnus Hagander) Under high load, Windows processes will sometimes fail at startup with this error code. Formerly the postmaster treated this as a panic condition and restarted the whole database, but that seems to be an overreaction.
•
Fix possible duplicate scans of UNION ALL member relations (Tom Lane)
•
Fix “cannot handle unplanned sub-select” error (Tom Lane) This occurred when a sub-select contains a join alias reference that expands into an expression containing another sub-select.
•
Reduce PANIC to ERROR in some occasionally-reported btree failure cases, and provide additional detail in the resulting error messages (Tom Lane) This should improve the system’s robustness with corrupted indexes.
•
Prevent show_session_authorization() from crashing within autovacuum processes (Tom Lane)
•
Defend against functions returning setof record where not all the returned rows are actually of the same rowtype (Tom Lane)
•
Fix possible failure when hashing a pass-by-reference function result (Tao Ma, Tom Lane)
•
Take care to fsync the contents of lockfiles (both postmaster.pid and the socket lockfile) while writing them (Tom Lane) This omission could result in corrupted lockfile contents if the machine crashes shortly after postmaster start. That could in turn prevent subsequent attempts to start the postmaster from succeeding, until the lockfile is manually removed.
•
Avoid recursion while assigning XIDs to heavily-nested subtransactions (Andres Freund, Robert Haas) The original coding could result in a crash if there was limited stack space.
•
Fix log_line_prefix’s %i escape, which could produce junk early in backend startup (Tom Lane)
•
Fix possible data corruption in ALTER TABLE ... SET TABLESPACE when archiving is enabled (Jeff Davis)
•
Allow CREATE DATABASE and ALTER DATABASE ... SET TABLESPACE to be interrupted by query-cancel (Guillaume Lelarge)
•
In PL/Python, defend against null pointer results from PyCObject_AsVoidPtr and PyCObject_FromVoidPtr (Peter Eisentraut)
2422
Appendix E. Release Notes •
Improve contrib/dblink’s handling of tables containing dropped columns (Tom Lane)
•
Fix connection leak after “duplicate connection name” errors in contrib/dblink (Itagaki Takahiro)
•
Fix contrib/dblink to handle connection names longer than 62 bytes correctly (Itagaki Takahiro)
•
Add hstore(text, text) function to contrib/hstore (Robert Haas) This function is the recommended substitute for the now-deprecated => operator. It was backpatched so that future-proofed code can be used with older server versions. Note that the patch will be effective only after contrib/hstore is installed or reinstalled in a particular database. Users might prefer to execute the CREATE FUNCTION command by hand, instead.
•
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
•
Update time zone data files to tzdata release 2010l for DST law changes in Egypt and Palestine; also historical corrections for Finland. This change also adds new names for two Micronesian timezones: Pacific/Chuuk is now preferred over Pacific/Truk (and the preferred abbreviation is CHUT not TRUT) and Pacific/Pohnpei is preferred over Pacific/Ponape.
•
Make Windows’ “N. Central Asia Standard Time” timezone map to Asia/Novosibirsk, not Asia/Almaty (Magnus Hagander) Microsoft changed the DST behavior of this zone in the timezone update from KB976098. Asia/Novosibirsk is a better match to its new behavior.
E.147. Release 8.2.17 Release date: 2010-05-17 This release contains a variety of fixes from 8.2.16. For information about new features in the 8.2 major release, see Section E.164.
E.147.1. Migration to Version 8.2.17 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.147.2. Changes •
Enforce restrictions in plperl using an opmask applied to the whole interpreter, instead of using Safe.pm (Tim Bunce, Andrew Dunstan) Recent developments have convinced us that Safe.pm is too insecure to rely on for making plperl trustable. This change removes use of Safe.pm altogether, in favor of using a separate interpreter with an opcode mask that is always applied. Pleasant side effects of the change include that it is now possible to use Perl’s strict pragma in a natural way in plperl, and that Perl’s $a and $b
2423
Appendix E. Release Notes variables work as expected in sort routines, and that function compilation is significantly faster. (CVE-2010-1169) •
Prevent PL/Tcl from executing untrustworthy code from pltcl_modules (Tom) PL/Tcl’s feature for autoloading Tcl code from a database table could be exploited for trojan-horse attacks, because there was no restriction on who could create or insert into that table. This change disables the feature unless pltcl_modules is owned by a superuser. (However, the permissions on the table are not checked, so installations that really need a less-than-secure modules table can still grant suitable privileges to trusted non-superusers.) Also, prevent loading code into the unrestricted “normal” Tcl interpreter unless we are really going to execute a pltclu function. (CVE-2010-1170)
•
Fix possible crash if a cache reset message is received during rebuild of a relcache entry (Heikki) This error was introduced in 8.2.16 while fixing a related failure.
•
Do not allow an unprivileged user to reset superuser-only parameter settings (Alvaro) Previously, if an unprivileged user ran ALTER USER ... RESET ALL for himself, or ALTER DATABASE ... RESET ALL for a database he owns, this would remove all special parameter settings for the user or database, even ones that are only supposed to be changeable by a superuser. Now, the ALTER will only remove the parameters that the user has permission to change.
•
Avoid possible crash during backend shutdown if shutdown occurs when a CONTEXT addition would be made to log entries (Tom) In some cases the context-printing function would fail because the current transaction had already been rolled back when it came time to print a log message.
•
Update pl/perl’s ppport.h for modern Perl versions (Andrew)
•
Fix assorted memory leaks in pl/python (Andreas Freund, Tom)
•
Prevent infinite recursion in psql when expanding a variable that refers to itself (Tom)
•
Fix psql’s \copy to not add spaces around a dot within \copy (select ...) (Tom) Addition of spaces around the decimal point in a numeric literal would result in a syntax error.
•
Ensure that contrib/pgstattuple functions respond to cancel interrupts promptly (Tatsuhito Kasahara)
•
Make server startup deal properly with the case that shmget() returns EINVAL for an existing shared memory segment (Tom) This behavior has been observed on BSD-derived kernels including OS X. It resulted in an entirelymisleading startup failure complaining that the shared memory request size was too large.
•
Avoid possible crashes in syslogger process on Windows (Heikki)
•
Deal more robustly with incomplete time zone information in the Windows registry (Magnus)
•
Update the set of known Windows time zone names (Magnus)
•
Update time zone data files to tzdata release 2010j for DST law changes in Argentina, Australian Antarctic, Bangladesh, Mexico, Morocco, Pakistan, Palestine, Russia, Syria, Tunisia; also historical corrections for Taiwan. Also, add PKST (Pakistan Summer Time) to the default set of timezone abbreviations.
2424
Appendix E. Release Notes
E.148. Release 8.2.16 Release date: 2010-03-15 This release contains a variety of fixes from 8.2.15. For information about new features in the 8.2 major release, see Section E.164.
E.148.1. Migration to Version 8.2.16 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
E.148.2. Changes •
Add new configuration parameter ssl_renegotiation_limit to control how often we do session key renegotiation for an SSL connection (Magnus) This can be set to zero to disable renegotiation completely, which may be required if a broken SSL library is used. In particular, some vendors are shipping stopgap patches for CVE-2009-3555 that cause renegotiation attempts to fail.
•
Fix possible deadlock during backend startup (Tom)
•
Fix possible crashes due to not handling errors during relcache reload cleanly (Tom)
•
Fix possible crashes when trying to recover from a failure in subtransaction start (Tom)
•
Fix server memory leak associated with use of savepoints and a client encoding different from server’s encoding (Tom)
•
Fix incorrect WAL data emitted during end-of-recovery cleanup of a GIST index page split (Yoichi Hirai) This would result in index corruption, or even more likely an error during WAL replay, if we were unlucky enough to crash during end-of-recovery cleanup after having completed an incomplete GIST insertion.
•
Make substring() for bit types treat any negative length as meaning “all the rest of the string” (Tom) The previous coding treated only -1 that way, and would produce an invalid result value for other negative values, possibly leading to a crash (CVE-2010-0442).
•
Fix integer-to-bit-string conversions to handle the first fractional byte correctly when the output bit width is wider than the given integer by something other than a multiple of 8 bits (Tom)
•
Fix some cases of pathologically slow regular expression matching (Tom)
•
Fix the STOP WAL LOCATION entry in backup history files to report the next WAL segment’s name when the end location is exactly at a segment boundary (Itagaki Takahiro)
•
Fix some more cases of temporary-file leakage (Heikki) This corrects a problem introduced in the previous minor release. One case that failed is when a plpgsql function returning set is called within another function’s exception handler.
•
Improve constraint exclusion processing of boolean-variable cases, in particular make it possible to exclude a partition that has a “bool_column = false” constraint (Tom)
2425
Appendix E. Release Notes •
When reading pg_hba.conf and related files, do not treat @something as a file inclusion request if the @ appears inside quote marks; also, never treat @ by itself as a file inclusion request (Tom) This prevents erratic behavior if a role or database name starts with @. If you need to include a file whose path name contains spaces, you can still do so, but you must write @"/path to/file" rather than putting the quotes around the whole construct.
•
Prevent infinite loop on some platforms if a directory is named as an inclusion target in pg_hba.conf and related files (Tom)
•
Fix possible infinite loop if SSL_read or SSL_write fails without setting errno (Tom) This is reportedly possible with some Windows versions of openssl.
•
Fix psql’s numericlocale option to not format strings it shouldn’t in latex and troff output formats (Heikki)
•
Make psql return the correct exit status (3) when ON_ERROR_STOP and --single-transaction are both specified and an error occurs during the implied COMMIT (Bruce)
•
Fix plpgsql failure in one case where a composite column is set to NULL (Tom)
•
Fix possible failure when calling PL/Perl functions from PL/PerlU or vice versa (Tim Bunce)
•
Add volatile markings in PL/Python to avoid possible compiler-specific misbehavior (Zdenek Kotala)
•
Ensure PL/Tcl initializes the Tcl interpreter fully (Tom) The only known symptom of this oversight is that the Tcl clock command misbehaves if using Tcl 8.5 or later.
•
Prevent crash in contrib/dblink when too many key columns are specified to a dblink_build_sql_* function (Rushabh Lathia, Joe Conway)
•
Fix assorted crashes in contrib/xml2 caused by sloppy memory management (Tom)
•
Make building of contrib/xml2 more robust on Windows (Andrew)
•
Fix race condition in Windows signal handling (Radu Ilie) One known symptom of this bug is that rows in pg_listener could be dropped under heavy load.
•
Update time zone data files to tzdata release 2010e for DST law changes in Bangladesh, Chile, Fiji, Mexico, Paraguay, Samoa.
E.149. Release 8.2.15 Release date: 2009-12-14 This release contains a variety of fixes from 8.2.14. For information about new features in the 8.2 major release, see Section E.164.
E.149.1. Migration to Version 8.2.15 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.14, see Section E.150.
2426
Appendix E. Release Notes
E.149.2. Changes •
Protect against indirect security threats caused by index functions changing session-local state (Gurjeet Singh, Tom) This change prevents allegedly-immutable index functions from possibly subverting a superuser’s session (CVE-2009-4136).
•
Reject SSL certificates containing an embedded null byte in the common name (CN) field (Magnus) This prevents unintended matching of a certificate to a server or client name during SSL validation (CVE-2009-4034).
•
Fix possible crash during backend-startup-time cache initialization (Tom)
•
Prevent signals from interrupting VACUUM at unsafe times (Alvaro) This fix prevents a PANIC if a VACUUM FULL is canceled after it’s already committed its tuple movements, as well as transient errors if a plain VACUUM is interrupted after having truncated the table.
•
Fix possible crash due to integer overflow in hash table size calculation (Tom) This could occur with extremely large planner estimates for the size of a hashjoin’s result.
•
Fix very rare crash in inet/cidr comparisons (Chris Mikkelson)
•
Ensure that shared tuple-level locks held by prepared transactions are not ignored (Heikki)
•
Fix premature drop of temporary files used for a cursor that is accessed within a subtransaction (Heikki)
•
Fix incorrect logic for GiST index page splits, when the split depends on a non-first column of the index (Paul Ramsey)
•
Don’t error out if recycling or removing an old WAL file fails at the end of checkpoint (Heikki) It’s better to treat the problem as non-fatal and allow the checkpoint to complete. Future checkpoints will retry the removal. Such problems are not expected in normal operation, but have been seen to be caused by misdesigned Windows anti-virus and backup software.
•
Ensure WAL files aren’t repeatedly archived on Windows (Heikki) This is another symptom that could happen if some other process interfered with deletion of a no-longer-needed file.
•
Fix PAM password processing to be more robust (Tom) The previous code is known to fail with the combination of the Linux pam_krb5 PAM module with Microsoft Active Directory as the domain controller. It might have problems elsewhere too, since it was making unjustified assumptions about what arguments the PAM stack would pass to it.
•
Fix processing of ownership dependencies during CREATE OR REPLACE FUNCTION (Tom)
•
Fix bug with calling plperl from plperlu or vice versa (Tom) An error exit from the inner function could result in crashes due to failure to re-select the correct Perl interpreter for the outer function.
•
Fix session-lifespan memory leak when a PL/Perl function is redefined (Tom)
•
Ensure that Perl arrays are properly converted to PostgreSQL arrays when returned by a setreturning PL/Perl function (Andrew Dunstan, Abhijit Menon-Sen) This worked correctly already for non-set-returning functions.
2427
Appendix E. Release Notes •
Fix rare crash in exception processing in PL/Python (Peter)
•
Ensure psql’s flex module is compiled with the correct system header definitions (Tom) This fixes build failures on platforms where --enable-largefile causes incompatible changes in the generated code.
•
Make the postmaster ignore any application_name parameter in connection request packets, to improve compatibility with future libpq versions (Tom)
•
Update the timezone abbreviation files to match current reality (Joachim Wieland) This includes adding IDT and SGT to the default timezone abbreviation set.
•
Update time zone data files to tzdata release 2009s for DST law changes in Antarctica, Argentina, Bangladesh, Fiji, Novokuznetsk, Pakistan, Palestine, Samoa, Syria; also historical corrections for Hong Kong.
E.150. Release 8.2.14 Release date: 2009-09-09 This release contains a variety of fixes from 8.2.13. For information about new features in the 8.2 major release, see Section E.164.
E.150.1. Migration to Version 8.2.14 A dump/restore is not required for those running 8.2.X. However, if you have any hash indexes on interval columns, you must REINDEX them after updating to 8.2.14. Also, if you are upgrading from a version earlier than 8.2.11, see Section E.153.
E.150.2. Changes •
Force WAL segment switch during pg_start_backup() (Heikki) This avoids corner cases that could render a base backup unusable.
•
Disallow RESET ROLE and RESET SESSION AUTHORIZATION inside security-definer functions (Tom, Heikki) This covers a case that was missed in the previous patch that disallowed SET ROLE and SET SESSION AUTHORIZATION inside security-definer functions. (See CVE-2007-6600)
•
Make LOAD of an already-loaded loadable module into a no-op (Tom) Formerly, LOAD would attempt to unload and re-load the module, but this is unsafe and not all that useful.
•
Disallow empty passwords during LDAP authentication (Magnus)
•
Fix handling of sub-SELECTs appearing in the arguments of an outer-level aggregate function (Tom)
•
Fix bugs associated with fetching a whole-row value from the output of a Sort or Materialize plan node (Tom)
2428
Appendix E. Release Notes •
Revert planner change that disabled partial-index and constraint exclusion optimizations when there were more than 100 clauses in an AND or OR list (Tom)
•
Fix hash calculation for data type interval (Tom) This corrects wrong results for hash joins on interval values. It also changes the contents of hash indexes on interval columns. If you have any such indexes, you must REINDEX them after updating.
•
Treat to_char(..., ’TH’) as an uppercase ordinal suffix with ’HH’/’HH12’ (Heikki) It was previously handled as ’th’ (lowercase).
•
Fix overflow for INTERVAL ’x ms’ when x is more than 2 million and integer datetimes are in use (Alex Hunsaker)
•
Fix calculation of distance between a point and a line segment (Tom) This led to incorrect results from a number of geometric operators.
•
Fix money data type to work in locales where currency amounts have no fractional digits, e.g. Japan (Itagaki Takahiro)
•
Properly round datetime input like 00:12:57.9999999999999999999999999999 (Tom)
•
Fix poor choice of page split point in GiST R-tree operator classes (Teodor)
•
Avoid performance degradation in bulk inserts into GIN indexes when the input values are (nearly) in sorted order (Tom)
•
Correctly enforce NOT NULL domain constraints in some contexts in PL/pgSQL (Tom)
•
Fix portability issues in plperl initialization (Andrew Dunstan)
•
Fix pg_ctl to not go into an infinite loop if postgresql.conf is empty (Jeff Davis)
•
Make contrib/hstore throw an error when a key or value is too long to fit in its data structure, rather than silently truncating it (Andrew Gierth)
•
Fix contrib/xml2’s xslt_process() to properly handle the maximum number of parameters (twenty) (Tom)
•
Improve robustness of libpq’s code to recover from errors during COPY FROM STDIN (Tom)
•
Avoid including conflicting readline and editline header files when both libraries are installed (Zdenek Kotala)
•
Update time zone data files to tzdata release 2009l for DST law changes in Bangladesh, Egypt, Jordan, Pakistan, Argentina/San_Luis, Cuba, Jordan (historical correction only), Mauritius, Morocco, Palestine, Syria, Tunisia.
E.151. Release 8.2.13 Release date: 2009-03-16 This release contains a variety of fixes from 8.2.12. For information about new features in the 8.2 major release, see Section E.164.
E.151.1. Migration to Version 8.2.13 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.11, see Section E.153.
2429
Appendix E. Release Notes
E.151.2. Changes •
Prevent error recursion crashes when encoding conversion fails (Tom) This change extends fixes made in the last two minor releases for related failure scenarios. The previous fixes were narrowly tailored for the original problem reports, but we have now recognized that any error thrown by an encoding conversion function could potentially lead to infinite recursion while trying to report the error. The solution therefore is to disable translation and encoding conversion and report the plain-ASCII form of any error message, if we find we have gotten into a recursive error reporting situation. (CVE-2009-0922)
•
Disallow CREATE CONVERSION with the wrong encodings for the specified conversion function (Heikki) This prevents one possible scenario for encoding conversion failure. The previous change is a backstop to guard against other kinds of failures in the same area.
•
Fix core dump when to_char() is given format codes that are inappropriate for the type of the data argument (Tom)
•
Fix possible failure in contrib/tsearch2 when C locale is used with a multi-byte encoding (Teodor) Crashes were possible on platforms where wchar_t is narrower than int; Windows in particular.
•
Fix extreme inefficiency in contrib/tsearch2 parser’s handling of an email-like string containing multiple @ characters (Heikki)
•
Fix decompilation of CASE WHEN with an implicit coercion (Tom) This mistake could lead to Assert failures in an Assert-enabled build, or an “unexpected CASE WHEN clause” error message in other cases, when trying to examine or dump a view.
•
Fix possible misassignment of the owner of a TOAST table’s rowtype (Tom) If CLUSTER or a rewriting variant of ALTER TABLE were executed by someone other than the table owner, the pg_type entry for the table’s TOAST table would end up marked as owned by that someone. This caused no immediate problems, since the permissions on the TOAST rowtype aren’t examined by any ordinary database operation. However, it could lead to unexpected failures if one later tried to drop the role that issued the command (in 8.1 or 8.2), or “owner of data type appears to be invalid” warnings from pg_dump after having done so (in 8.3).
•
Fix PL/pgSQL to not treat INTO after INSERT as an INTO-variables clause anywhere in the string, not only at the start; in particular, don’t fail for INSERT INTO within CREATE RULE (Tom)
•
Clean up PL/pgSQL error status variables fully at block exit (Ashesh Vashi and Dave Page) This is not a problem for PL/pgSQL itself, but the omission could cause the PL/pgSQL Debugger to crash while examining the state of a function.
•
Retry failed calls to CallNamedPipe() on Windows (Steve Marshall, Magnus) It appears that this function can sometimes fail transiently; we previously treated any failure as a hard error, which could confuse LISTEN/NOTIFY as well as other operations.
•
Add MUST (Mauritius Island Summer Time) to the default list of known timezone abbreviations (Xavier Bugaud)
2430
Appendix E. Release Notes
E.152. Release 8.2.12 Release date: 2009-02-02 This release contains a variety of fixes from 8.2.11. For information about new features in the 8.2 major release, see Section E.164.
E.152.1. Migration to Version 8.2.12 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.11, see Section E.153.
E.152.2. Changes •
Improve handling of URLs in headline() function (Teodor)
•
Improve handling of overlength headlines in headline() function (Teodor)
•
Prevent possible Assert failure or misconversion if an encoding conversion is created with the wrong conversion function for the specified pair of encodings (Tom, Heikki)
•
Fix possible Assert failure if a statement executed in PL/pgSQL is rewritten into another kind of statement, for example if an INSERT is rewritten into an UPDATE (Heikki)
•
Ensure that a snapshot is available to datatype input functions (Tom) This primarily affects domains that are declared with CHECK constraints involving user-defined stable or immutable functions. Such functions typically fail if no snapshot has been set.
•
Make it safer for SPI-using functions to be used within datatype I/O; in particular, to be used in domain check constraints (Tom)
•
Avoid unnecessary locking of small tables in VACUUM (Heikki)
•
Fix a problem that made UPDATE RETURNING tableoid return zero instead of the correct OID (Tom)
•
Fix planner misestimation of selectivity when transitive equality is applied to an outer-join clause (Tom) This could result in bad plans for queries like ... from a left join b on a.a1 = b.b1 where a.a1 = 42 ...
•
Improve optimizer’s handling of long IN lists (Tom) This change avoids wasting large amounts of time on such lists when constraint exclusion is enabled.
•
Ensure that the contents of a holdable cursor don’t depend on the contents of TOAST tables (Tom) Previously, large field values in a cursor result might be represented as TOAST pointers, which would fail if the referenced table got dropped before the cursor is read, or if the large value is deleted and then vacuumed away. This cannot happen with an ordinary cursor, but it could with a cursor that is held past its creating transaction.
•
Fix memory leak when a set-returning function is terminated without reading its whole result (Tom)
•
Fix contrib/dblink’s dblink_get_result(text,bool) function (Joe)
•
Fix possible garbage output from contrib/sslinfo functions (Tom)
2431
Appendix E. Release Notes •
Fix configure script to properly report failure when unable to obtain linkage information for PL/Perl (Andrew)
•
Make all documentation reference pgsql-bugs and/or pgsql-hackers as appropriate, instead of the now-decommissioned pgsql-ports and pgsql-patches mailing lists (Tom)
•
Update time zone data files to tzdata release 2009a (for Kathmandu and historical DST corrections in Switzerland, Cuba)
E.153. Release 8.2.11 Release date: 2008-11-03 This release contains a variety of fixes from 8.2.10. For information about new features in the 8.2 major release, see Section E.164.
E.153.1. Migration to Version 8.2.11 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see Section E.157. Also, if you were running a previous 8.2.X release, it is recommended to REINDEX all GiST indexes after the upgrade.
E.153.2. Changes •
Fix GiST index corruption due to marking the wrong index entry “dead” after a deletion (Teodor) This would result in index searches failing to find rows they should have found. Corrupted indexes can be fixed with REINDEX.
•
Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the “character has no equivalent” message itself couldn’t be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation.
•
Fix possible crash when deeply nested functions are invoked from a trigger (Tom)
•
Improve optimization of expression IN (expression-list) queries (Tom, per an idea from Robert Haas) Cases in which there are query variables on the right-hand side had been handled less efficiently in 8.2.x and 8.3.x than in prior versions. The fix restores 8.1 behavior for such cases.
•
Fix mis-expansion of rule queries when a sub-SELECT appears in a function call in FROM, a multirow VALUES list, or a RETURNING list (Tom) The usual symptom of this problem is an “unrecognized node type” error.
•
Fix memory leak during rescan of a hashed aggregation plan (Neil)
•
Ensure an error is reported when a newly-defined PL/pgSQL trigger function is invoked as a normal function (Tom)
•
Prevent possible collision of relfilenode numbers when moving a table to another tablespace with ALTER SET TABLESPACE (Heikki)
2432
Appendix E. Release Notes The command tried to re-use the existing filename, instead of picking one that is known unused in the destination directory. •
Fix incorrect tsearch2 headline generation when single query item matches first word of text (Sushant Sinha)
•
Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an --enable-integer-datetimes build (Ron Mayer)
•
Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn’t handle it properly. The only likely consequence is an incorrect error indication.
•
Fix ecpg’s parsing of CREATE ROLE (Michael)
•
Fix recent breakage of pg_ctl restart (Tom)
•
Ensure pg_control is opened in binary mode (Itagaki Takahiro) pg_controldata and pg_resetxlog did this incorrectly, and so could fail on Windows.
•
Update time zone data files to tzdata release 2008i (for DST law changes in Argentina, Brazil, Mauritius, Syria)
E.154. Release 8.2.10 Release date: 2008-09-22 This release contains a variety of fixes from 8.2.9. For information about new features in the 8.2 major release, see Section E.164.
E.154.1. Migration to Version 8.2.10 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see Section E.157.
E.154.2. Changes •
Fix bug in btree WAL recovery code (Heikki) Recovery failed if the WAL ended partway through a page split operation.
•
Fix potential miscalculation of datfrozenxid (Alvaro) This error may explain some recent reports of failure to remove old pg_clog data.
•
Widen local lock counters from 32 to 64 bits (Tom) This responds to reports that the counters could overflow in sufficiently long transactions, leading to unexpected “lock is already held” errors.
•
Fix possible duplicate output of tuples during a GiST index scan (Teodor)
•
Fix missed permissions checks when a view contains a simple UNION ALL construct (Heikki) Permissions for the referenced tables were checked properly, but not permissions for the view itself.
2433
Appendix E. Release Notes •
Add checks in executor startup to ensure that the tuples produced by an INSERT or UPDATE will match the target table’s current rowtype (Tom) ALTER COLUMN TYPE, followed by re-use of a previously cached plan, could produce this type of
situation. The check protects against data corruption and/or crashes that could ensue. •
Fix possible repeated drops during DROP OWNED (Tom) This would typically result in strange errors such as “cache lookup failed for relation NNN”.
•
Fix AT TIME ZONE to first try to interpret its timezone argument as a timezone abbreviation, and only try it as a full timezone name if that fails, rather than the other way around as formerly (Tom) The timestamp input functions have always resolved ambiguous zone names in this order. Making AT TIME ZONE do so as well improves consistency, and fixes a compatibility bug introduced in 8.1: in ambiguous cases we now behave the same as 8.0 and before did, since in the older versions AT TIME ZONE accepted only abbreviations.
•
Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom)
•
Prevent integer overflows during units conversion when displaying a configuration parameter that has units (Tom)
•
Improve performance of writing very long log messages to syslog (Tom)
•
Allow spaces in the suffix part of an LDAP URL in pg_hba.conf (Tom)
•
Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom)
•
Fix planner bug with nested sub-select expressions (Tom) If the outer sub-select has no direct dependency on the parent query, but the inner one does, the outer value might not get recalculated for new parent query rows.
•
Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions’ contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like col IS NULL.
•
Fix PL/pgSQL to not fail when a FOR loop’s target variable is a record containing composite-type fields (Tom)
•
Fix PL/Tcl to behave correctly with Tcl 8.5, and to be more careful about the encoding of data sent to or from Tcl (Tom)
•
On Windows, work around a Microsoft bug by preventing libpq from trying to send more than 64kB per system call (Magnus)
•
Improve pg_dump and pg_restore’s error reporting after failure to send a SQL command (Tom)
•
Fix pg_ctl to properly preserve postmaster command-line arguments across a restart (Bruce)
•
Update time zone data files to tzdata release 2008f (for DST law changes in Argentina, Bahamas, Brazil, Mauritius, Morocco, Pakistan, Palestine, and Paraguay)
E.155. Release 8.2.9 Release date: 2008-06-12
2434
Appendix E. Release Notes This release contains one serious and one minor bug fix over 8.2.8. For information about new features in the 8.2 major release, see Section E.164.
E.155.1. Migration to Version 8.2.9 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see Section E.157.
E.155.2. Changes •
Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid SELECT DISTINCT view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7.
•
Make ALTER AGGREGATE ... OWNER TO update pg_shdepend (Tom) This oversight could lead to problems if the aggregate was later involved in a DROP OWNED or REASSIGN OWNED operation.
E.156. Release 8.2.8 Release date: never released This release contains a variety of fixes from 8.2.7. For information about new features in the 8.2 major release, see Section E.164.
E.156.1. Migration to Version 8.2.8 A dump/restore is not required for those running 8.2.X. However, if you are upgrading from a version earlier than 8.2.7, see Section E.157.
E.156.2. Changes •
Fix ERRORDATA_STACK_SIZE exceeded crash that occurred on Windows when using UTF-8 database encoding and a different client encoding (Tom)
•
Fix ALTER TABLE ADD COLUMN ... PRIMARY KEY so that the new column is correctly checked to see if it’s been initialized to all non-nulls (Brendan Jurd) Previous versions neglected to check this requirement at all.
•
Fix possible CREATE TABLE failure when inheriting the “same” constraint from multiple parent relations that inherited that constraint from a common ancestor (Tom)
2435
Appendix E. Release Notes •
Fix pg_get_ruledef() to show the alias, if any, attached to the target table of an UPDATE or DELETE (Tom)
•
Fix GIN bug that could result in a too many LWLocks taken failure (Teodor)
•
Avoid possible crash when decompressing corrupted data (Zdenek Kotala)
•
Repair two places where SIGTERM exit of a backend could leave corrupted state in shared memory (Tom) Neither case is very important if SIGTERM is used to shut down the whole database cluster together, but there was a problem if someone tried to SIGTERM individual backends.
•
Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic “Yo” characters (e and E with two dots) (Sergey Burladyan)
•
Fix several datatype input functions, notably array_in(), that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions.
•
Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn’t got a match. An example is substring(’foo’ from ’foo(bar)?’). This should return NULL, since (bar) isn’t matched, but it was mistakenly returning the whole-pattern match instead (ie, foo).
•
Update time zone data files to tzdata release 2008c (for DST law changes in Morocco, Iraq, Choibalsan, Pakistan, Syria, Cuba, and Argentina/San_Luis)
•
Fix incorrect result from ecpg’s PGTYPEStimestamp_sub() function (Michael)
•
Fix broken GiST comparison function for contrib/tsearch2’s tsquery type (Teodor)
•
Fix possible crashes in contrib/cube functions (Tom)
•
Fix core dump in contrib/xml2’s xpath_table() function when the input query returns a NULL value (Tom)
•
Fix contrib/xml2’s makefile to not override CFLAGS (Tom)
•
Fix DatumGetBool macro to not fail with gcc 4.3 (Tom) This problem affects “old style” (V0) C functions that return boolean. The fix is already in 8.3, but the need to back-patch it was not realized at the time.
E.157. Release 8.2.7 Release date: 2008-03-17 This release contains a variety of fixes from 8.2.6. For information about new features in the 8.2 major release, see Section E.164.
E.157.1. Migration to Version 8.2.7 A dump/restore is not required for those running 8.2.X. However, you might need to REINDEX indexes on textual columns after updating, if you are affected by the Windows locale issue described below.
2436
Appendix E. Release Notes
E.157.2. Changes •
Fix character string comparison for Windows locales that consider different character combinations as equal (Tom) This fix applies only on Windows and only when using UTF-8 database encoding. The same fix was made for all other cases over two years ago, but Windows with UTF-8 uses a separate code path that was not updated. If you are using a locale that considers some non-identical strings as equal, you may need to REINDEX to fix existing indexes on textual columns.
•
Repair potential deadlock between concurrent VACUUM FULL operations on different system catalogs (Tom)
•
Fix longstanding LISTEN/NOTIFY race condition (Tom) In rare cases a session that had just executed a LISTEN might not get a notification, even though one would be expected because the concurrent transaction executing NOTIFY was observed to commit later. A side effect of the fix is that a transaction that has executed a not-yet-committed LISTEN command will not see any row in pg_listener for the LISTEN, should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior.
•
Disallow LISTEN and UNLISTEN within a prepared transaction (Tom) This was formerly allowed but trying to do it had various unpleasant consequences, notably that the originating backend could not exit as long as an UNLISTEN remained uncommitted.
•
Disallow dropping a temporary table within a prepared transaction (Heikki) This was correctly disallowed by 8.1, but the check was inadvertently broken in 8.2.
•
Fix rare crash when an error occurs during a query using a hash index (Heikki)
•
Fix memory leaks in certain usages of set-returning functions (Neil)
•
Fix input of datetime values for February 29 in years BC (Tom) The former coding was mistaken about which years were leap years.
•
Fix “unrecognized node type” error in some variants of ALTER OWNER (Tom)
•
Ensure pg_stat_activity.waiting flag is cleared when a lock wait is aborted (Tom)
•
Fix handling of process permissions on Windows Vista (Dave, Magnus) In particular, this fix allows starting the server as the Administrator user.
•
Update time zone data files to tzdata release 2008a (in particular, recent Chile changes); adjust timezone abbreviation VET (Venezuela) to mean UTC-4:30, not UTC-4:00 (Tom)
•
Fix pg_ctl to correctly extract the postmaster’s port number from command-line options (Itagaki Takahiro, Tom) Previously, pg_ctl start -w could try to contact the postmaster on the wrong port, leading to bogus reports of startup failure.
•
Use -fwrapv to defend against possible misoptimization in recent gcc versions (Tom) This is known to be necessary when building PostgreSQL with gcc 4.3 or later.
•
Correctly enforce statement_timeout values longer than INT_MAX microseconds (about 35 minutes) (Tom) This bug affects only builds with --enable-integer-datetimes.
2437
Appendix E. Release Notes •
Fix “unexpected PARAM_SUBLINK ID” planner error when constant-folding simplifies a subselect (Tom)
•
Fix logical errors in constraint-exclusion handling of IS NULL and NOT expressions (Tom) The planner would sometimes exclude partitions that should not have been excluded because of the possibility of NULL results.
•
Fix another cause of “failed to build any N-way joins” planner errors (Tom) This could happen in cases where a clauseless join needed to be forced before a join clause could be exploited.
•
Fix incorrect constant propagation in outer-join planning (Tom) The planner could sometimes incorrectly conclude that a variable could be constrained to be equal to a constant, leading to wrong query results.
•
Fix display of constant expressions in ORDER BY and GROUP BY (Tom) An explicitly casted constant would be shown incorrectly. This could for example lead to corruption of a view definition during dump and reload.
•
Fix libpq to handle NOTICE messages correctly during COPY OUT (Tom) This failure has only been observed to occur when a user-defined datatype’s output routine issues a NOTICE, but there is no guarantee it couldn’t happen due to other causes.
E.158. Release 8.2.6 Release date: 2008-01-07 This release contains a variety of fixes from 8.2.5, including fixes for significant security issues. For information about new features in the 8.2 major release, see Section E.164.
E.158.1. Migration to Version 8.2.6 A dump/restore is not required for those running 8.2.X.
E.158.2. Changes •
Prevent functions in indexes from executing with the privileges of the user running VACUUM, ANALYZE, etc (Tom)
Functions used in index expressions and partial-index predicates are evaluated whenever a new table entry is made. It has long been understood that this poses a risk of trojan-horse code execution if one modifies a table owned by an untrustworthy user. (Note that triggers, defaults, check constraints, etc. pose the same type of risk.) But functions in indexes pose extra danger because they will be executed by routine maintenance operations such as VACUUM FULL, which are commonly performed automatically under a superuser account. For example, a nefarious user can execute code with superuser privileges by setting up a trojan-horse index definition and waiting for the next routine vacuum. The fix arranges for standard maintenance operations (including VACUUM, ANALYZE, REINDEX, and CLUSTER) to execute as the table owner rather than the calling user, using the same privilege-switching mechanism already used for SECURITY DEFINER functions. To
2438
Appendix E. Release Notes prevent bypassing this security measure, execution of SET SESSION AUTHORIZATION and SET ROLE is now forbidden within a SECURITY DEFINER context. (CVE-2007-6600) •
Repair assorted bugs in the regular-expression package (Tom, Will Drewry) Suitably crafted regular-expression patterns could cause crashes, infinite or near-infinite looping, and/or massive memory consumption, all of which pose denial-of-service hazards for applications that accept regex search patterns from untrustworthy sources. (CVE-2007-4769, CVE-2007-4772, CVE-2007-6067)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe) The fix that appeared for this in 8.2.5 was incomplete, as it plugged the hole for only some dblink functions. (CVE-2007-6601, CVE-2007-3278)
•
Fix bugs in WAL replay for GIN indexes (Teodor)
•
Fix GIN index build to work properly when maintenance_work_mem is 4GB or more (Tom)
•
Update time zone data files to tzdata release 2007k (in particular, recent Argentina changes) (Tom)
•
Improve planner’s handling of LIKE/regex estimation in non-C locales (Tom)
•
Fix planning-speed problem for deep outer-join nests, as well as possible poor choice of join order (Tom)
•
Fix planner failure in some cases of WHERE false AND var IN (SELECT ...) (Tom)
•
Make CREATE TABLE ... SERIAL and ALTER SEQUENCE ... OWNED BY not change the currval() state of the sequence (Tom)
•
Preserve the tablespace and storage parameters of indexes that are rebuilt by ALTER TABLE ... ALTER COLUMN TYPE (Tom)
•
Make archive recovery always start a new WAL timeline, rather than only when a recovery stop time was used (Simon) This avoids a corner-case risk of trying to overwrite an existing archived copy of the last WAL segment, and seems simpler and cleaner than the original definition.
•
Make VACUUM not use all of maintenance_work_mem when the table is too small for it to be useful (Alvaro)
•
Fix potential crash in translate() when using a multibyte database encoding (Tom)
•
Make corr() return the correct result for negative correlation values (Neil)
•
Fix overflow in extract(epoch from interval) for intervals exceeding 68 years (Tom)
•
Fix PL/Perl to not fail when a UTF-8 regular expression is used in a trusted function (Andrew)
•
Fix PL/Perl to cope when platform’s Perl defines type bool as int rather than char (Tom) While this could theoretically happen anywhere, no standard build of Perl did things this way ... until Mac OS X 10.5.
•
Fix PL/Python to work correctly with Python 2.5 on 64-bit machines (Marko Kreen)
•
Fix PL/Python to not crash on long exception messages (Alvaro)
•
Fix pg_dump to correctly handle inheritance child tables that have default expressions different from their parent’s (Tom)
•
Fix libpq crash when PGPASSFILE refers to a file that is not a plain file (Martin Pitt)
•
ecpg parser fixes (Michael)
2439
Appendix E. Release Notes •
Make contrib/pgcrypto defend against OpenSSL libraries that fail on keys longer than 128 bits; which is the case at least on some Solaris versions (Marko Kreen)
•
Make contrib/tablefunc’s crosstab() handle NULL rowid as a category in its own right, rather than crashing (Joe)
•
Fix tsvector and tsquery output routines to escape backslashes correctly (Teodor, Bruce)
•
Fix crash of to_tsvector() on huge input strings (Teodor)
•
Require a specific version of Autoconf to be used when re-generating the configure script (Peter) This affects developers and packagers only. The change was made to prevent accidental use of untested combinations of Autoconf and PostgreSQL versions. You can remove the version check if you really want to use a different Autoconf version, but it’s your responsibility whether the result works or not.
•
Update gettimeofday configuration check so that PostgreSQL can be built on newer versions of MinGW (Magnus)
E.159. Release 8.2.5 Release date: 2007-09-17 This release contains a variety of fixes from 8.2.4. For information about new features in the 8.2 major release, see Section E.164.
E.159.1. Migration to Version 8.2.5 A dump/restore is not required for those running 8.2.X.
E.159.2. Changes •
Prevent index corruption when a transaction inserts rows and then aborts close to the end of a concurrent VACUUM on the same table (Tom)
•
Fix ALTER DOMAIN ADD CONSTRAINT for cases involving domains over domains (Tom)
•
Make CREATE DOMAIN ... DEFAULT NULL work properly (Tom)
•
Fix some planner problems with outer joins, notably poor size estimation for t1 LEFT JOIN t2 WHERE t2.col IS NULL (Tom)
•
Allow the interval data type to accept input consisting only of milliseconds or microseconds (Neil)
•
Allow timezone name to appear before the year in timestamp input (Tom)
•
Fixes for GIN indexes used by /contrib/tsearch2 (Teodor)
•
Speed up rtree index insertion (Teodor)
•
Fix excessive logging of SSL error messages (Tom)
•
Fix logging so that log messages are never interleaved when using the syslogger process (Andrew)
•
Fix crash when log_min_error_statement logging runs out of memory (Tom)
2440
Appendix E. Release Notes •
Fix incorrect handling of some foreign-key corner cases (Tom)
•
Fix stddev_pop(numeric) and var_pop(numeric) (Tom)
•
Prevent REINDEX and CLUSTER from failing due to attempting to process temporary tables of other sessions (Alvaro)
•
Update the time zone database rules, particularly New Zealand’s upcoming changes (Tom)
•
Windows socket and semaphore improvements (Magnus)
•
Make pg_ctl -w work properly in Windows service mode (Dave Page)
•
Fix memory allocation bug when using MIT Kerberos on Windows (Magnus)
•
Suppress timezone name (%Z) in log timestamps on Windows because of possible encoding mismatches (Tom)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe)
•
Restrict /contrib/pgstattuple functions to superusers, for security reasons (Tom)
•
Do not let /contrib/intarray try to make its GIN opclass the default (this caused problems at dump/restore) (Tom)
E.160. Release 8.2.4 Release date: 2007-04-23 This release contains a variety of fixes from 8.2.3, including a security fix. For information about new features in the 8.2 major release, see Section E.164.
E.160.1. Migration to Version 8.2.4 A dump/restore is not required for those running 8.2.X.
E.160.2. Changes •
Support explicit placement of the temporary-table schema within search_path, and disable searching it for functions and operators (Tom) This is needed to allow a security-definer function to set a truly secure value of search_path. Without it, an unprivileged SQL user can use temporary objects to execute code with the privileges of the security-definer function (CVE-2007-2138). See CREATE FUNCTION for more information.
•
Fix shared_preload_libraries for Windows by forcing reload in each backend (Korry Douglas)
•
Fix to_char() so it properly upper/lower cases localized day or month names (Pavel Stehule)
• /contrib/tsearch2
crash fixes (Teodor)
•
Require COMMIT PREPARED to be executed in the same database as the transaction was prepared in (Heikki)
•
Allow pg_dump to do binary backups larger than two gigabytes on Windows (Magnus)
2441
Appendix E. Release Notes •
New traditional (Taiwan) Chinese FAQ (Zhou Daojing)
•
Prevent the statistics collector from writing to disk too frequently (Tom)
•
Fix potential-data-corruption bug in how VACUUM FULL handles UPDATE chains (Tom, Pavan Deolasee)
•
Fix bug in domains that use array types (Tom)
•
Fix pg_dump so it can dump a serial column’s sequence using -t when not also dumping the owning table (Tom)
•
Planner fixes, including improving outer join and bitmap scan selection logic (Tom)
•
Fix possible wrong answers or crash when a PL/pgSQL function tries to RETURN from within an EXCEPTION block (Tom)
•
Fix PANIC during enlargement of a hash index (Tom)
•
Fix POSIX-style timezone specs to follow new USA DST rules (Tom)
E.161. Release 8.2.3 Release date: 2007-02-07 This release contains two fixes from 8.2.2. For information about new features in the 8.2 major release, see Section E.164.
E.161.1. Migration to Version 8.2.3 A dump/restore is not required for those running 8.2.X.
E.161.2. Changes •
Remove overly-restrictive check for type length in constraints and functional indexes(Tom)
•
Fix optimization so MIN/MAX in subqueries can again use indexes (Tom)
E.162. Release 8.2.2 Release date: 2007-02-05 This release contains a variety of fixes from 8.2.1, including a security fix. For information about new features in the 8.2 major release, see Section E.164.
E.162.1. Migration to Version 8.2.2 A dump/restore is not required for those running 8.2.X.
2442
Appendix E. Release Notes
E.162.2. Changes •
Remove security vulnerabilities that allowed connected users to read backend memory (Tom) The vulnerabilities involve suppressing the normal check that a SQL function returns the data type it’s declared to, and changing the data type of a table column (CVE-2007-0555, CVE-2007-0556). These errors can easily be exploited to cause a backend crash, and in principle might be used to read database content that the user should not be able to access.
•
Fix not-so-rare-anymore bug wherein btree index page splits could fail due to choosing an infeasible split point (Heikki Linnakangas)
•
Fix Borland C compile scripts (L Bayuk)
•
Properly handle to_char(’CC’) for years ending in 00 (Tom) Year 2000 is in the twentieth century, not the twenty-first.
• /contrib/tsearch2 •
localization improvements (Tatsuo, Teodor)
Fix incorrect permission check in information_schema.key_column_usage view (Tom) The symptom is “relation with OID nnnnn does not exist” errors. To get this fix without using initdb, use CREATE OR REPLACE VIEW to install the corrected definition found in share/information_schema.sql. Note you will need to do this in each database.
•
Improve VACUUM performance for databases with many tables (Tom)
•
Fix for rare Assert() crash triggered by UNION (Tom)
•
Fix potentially incorrect results from index searches using ROW inequality conditions (Tom)
•
Tighten security of multi-byte character processing for UTF8 sequences over three bytes long (Tom)
•
Fix bogus “permission denied” failures occurring on Windows due to attempts to fsync alreadydeleted files (Magnus, Tom)
•
Fix bug that could cause the statistics collector to hang on Windows (Magnus) This would in turn lead to autovacuum not working.
•
Fix possible crashes when an already-in-use PL/pgSQL function is updated (Tom)
•
Improve PL/pgSQL handling of domain types (Sergiy Vyshnevetskiy, Tom)
•
Fix possible errors in processing PL/pgSQL exception blocks (Tom)
E.163. Release 8.2.1 Release date: 2007-01-08 This release contains a variety of fixes from 8.2. For information about new features in the 8.2 major release, see Section E.164.
E.163.1. Migration to Version 8.2.1 A dump/restore is not required for those running 8.2.
2443
Appendix E. Release Notes
E.163.2. Changes •
Fix crash with SELECT ... LIMIT ALL (also LIMIT NULL) (Tom)
• Several /contrib/tsearch2 •
fixes (Teodor)
On Windows, make log messages coming from the operating system use ASCII encoding (Hiroshi Saito) This fixes a conversion problem when there is a mismatch between the encoding of the operating system and database server.
•
Fix Windows linking of pg_dump using win32.mak (Hiroshi Saito)
•
Fix planner mistakes for outer join queries (Tom)
•
Fix several problems in queries involving sub-SELECTs (Tom)
•
Fix potential crash in SPI during subtransaction abort (Tom) This affects all PL functions since they all use SPI.
•
Improve build speed of PDF documentation (Peter)
•
Re-add JST (Japan) timezone abbreviation (Tom)
•
Improve optimization decisions related to index scans (Tom)
•
Have psql print multi-byte combining characters as before, rather than output as \u (Tom)
•
Improve index usage of regular expressions that use parentheses (Tom) This improves psql \d performance also.
•
Make pg_dumpall assume that databases have public CONNECT privilege, when dumping from a pre-8.2 server (Tom) This preserves the previous behavior that anyone can connect to a database if allowed by pg_hba.conf.
E.164. Release 8.2 Release date: 2006-12-05
E.164.1. Overview This release adds many functionality and performance improvements that were requested by users, including:
•
Query language enhancements including INSERT/UPDATE/DELETE RETURNING, multirow VALUES lists, and optional target-table alias in UPDATE/DELETE
•
Index creation without blocking concurrent INSERT/UPDATE/DELETE operations
•
Many query optimization improvements, including support for reordering outer joins
•
Improved sorting performance with lower memory usage
•
More efficient locking with better concurrency
2444
Appendix E. Release Notes •
More efficient vacuuming
•
Easier administration of warm standby servers
•
New FILLFACTOR support for tables and indexes
•
Monitoring, logging, and performance tuning additions
•
More control over creating and dropping objects
•
Table inheritance relationships can be defined for and removed from pre-existing tables
• COPY TO
can copy the output of an arbitrary SELECT statement
•
Array improvements, including nulls in arrays
•
Aggregate-function improvements, including multiple-input aggregates and SQL:2003 statistical functions
•
Many contrib/ improvements
E.164.2. Migration to Version 8.2 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: •
Set escape_string_warning to on by default (Bruce) This issues a warning if backslash escapes are used in non-escape (non-E”) strings.
•
Change the row constructor syntax (ROW(...)) so that list elements foo.* will be expanded to a list of their member fields, rather than creating a nested row type field as formerly (Tom) The new behavior is substantially more useful since it allows, for example, triggers to check for data changes with IF row(new.*) IS DISTINCT FROM row(old.*). The old behavior is still available by omitting .*.
•
Make row comparisons follow SQL standard semantics and allow them to be used in index scans (Tom) Previously, row = and comparisons followed the standard but < >= did not. A row comparison can now be used as an index constraint for a multicolumn index matching the row value.
•
Make row IS [NOT] NULL tests follow SQL standard semantics (Tom) The former behavior conformed to the standard for simple cases with IS NULL, but IS NOT NULL would return true if any row field was non-null, whereas the standard says it should return true only when all fields are non-null.
•
Make SET CONSTRAINT affect only one constraint (Kris Jurka) In previous releases, SET CONSTRAINT modified all constraints with a matching name. In this release, the schema search path is used to modify only the first matching constraint. A schema specification is also supported. This more nearly conforms to the SQL standard.
•
Remove RULE permission for tables, for security reasons (Tom) As of this release, only a table’s owner can create or modify rules for the table. For backwards compatibility, GRANT/REVOKE RULE is still accepted, but it does nothing.
2445
Appendix E. Release Notes •
Array comparison improvements (Tom) Now array dimensions are also compared.
•
Change array concatenation to match documented behavior (Tom) This changes the previous behavior where concatenation would modify the array lower bound.
•
Make command-line options of postmaster and postgres identical (Peter) This allows the postmaster to pass arguments to each backend without using -o. Note that some options are now only available as long-form options, because there were conflicting single-letter options.
•
Deprecate use of postmaster symbolic link (Peter) postmaster and postgres commands now act identically, with the behavior determined by commandline options. The postmaster symbolic link is kept for compatibility, but is not really needed.
•
Change log_duration to output even if the query is not output (Tom) In prior releases, log_duration only printed if the query appeared earlier in the log.
•
Make to_char(time) and to_char(interval) treat HH and HH12 as 12-hour intervals Most applications should use HH24 unless they want a 12-hour display.
•
Zero unmasked bits in conversion from INET to CIDR (Tom) This ensures that the converted value is actually valid for CIDR.
•
Remove australian_timezones configuration variable (Joachim Wieland) This variable has been superseded by a more general facility for configuring timezone abbreviations.
•
Improve cost estimation for nested-loop index scans (Tom) This might eliminate the need to set unrealistically small values of random_page_cost. If you have been using a very small random_page_cost, please recheck your test cases.
•
Change behavior of pg_dump -n and -t options. (Greg Sabino Mullane) See the pg_dump manual page for details.
•
Change libpq PQdsplen() to return a useful value (Martijn van Oosterhout)
•
Declare libpq PQgetssl() as returning void *, rather than SSL * (Martijn van Oosterhout) This allows applications to use the function without including the OpenSSL headers.
•
C-language loadable modules must now include a PG_MODULE_MAGIC macro call for version compatibility checking (Martijn van Oosterhout)
•
For security’s sake, modules used by a PL/PerlU function are no longer available to PL/Perl functions (Andrew) Note: This also implies that data can no longer be shared between a PL/Perl function and a PL/PerlU function. Some Perl installations have not been compiled with the correct flags to allow multiple interpreters to exist within a single process. In this situation PL/Perl and PL/PerlU cannot both be used in a single backend. The solution is to get a Perl installation which supports multiple interpreters.
•
In contrib/xml2/, rename xml_valid() to xml_is_well_formed() (Tom)
2446
Appendix E. Release Notes xml_valid() will remain for backward compatibility, but its behavior will change to do schema
checking in a future release. • •
Remove contrib/ora2pg/, now at http://www.samse.fr/GPL/ora2pg Remove contrib modules that have been migrated to PgFoundry: adddepend, dbase, dbmirror, fulltextindex, mac, userlock
Remove QNX and BEOS ports (Bruce) These ports no longer had active maintainers.
E.164.3. Changes Below you will find a detailed account of the changes between PostgreSQL 8.2 and the previous major release.
E.164.3.1. Performance Improvements •
Allow the planner to reorder outer joins in some circumstances (Tom) In previous releases, outer joins would always be evaluated in the order written in the query. This change allows the query optimizer to consider reordering outer joins, in cases where it can determine that the join order can be changed without altering the meaning of the query. This can make a considerable performance difference for queries involving multiple outer joins or mixed inner and outer joins.
•
Improve efficiency of IN (list-of-expressions) clauses (Tom)
•
Improve sorting speed and reduce memory usage (Simon, Tom)
Add FILLFACTOR to table and index creation (ITAGAKI Takahiro) This leaves extra free space in each table or index page, allowing improved performance as the database grows. This is particularly valuable to maintain clustering.
•
Increase default values for shared_buffers and max_fsm_pages (Andrew)
•
Improve locking performance by breaking the lock manager tables into sections (Tom) This allows locking to be more fine-grained, reducing contention.
•
Reduce locking requirements of sequential scans (Qingqing Zhou)
•
Reduce locking required for database creation and destruction (Tom)
•
Improve the optimizer’s selectivity estimates for LIKE, ILIKE, and regular expression operations (Tom)
•
Improve planning of joins to inherited tables and UNION ALL views (Tom)
•
Allow constraint exclusion to be applied to inherited UPDATE and DELETE queries (Tom) SELECT already honored constraint exclusion.
•
Improve planning of constant WHERE clauses, such as a condition that depends only on variables inherited from an outer query level (Tom)
•
Protocol-level unnamed prepared statements are re-planned for each set of BIND values (Tom)
2447
Appendix E. Release Notes This improves performance because the exact parameter values can be used in the plan. •
Speed up vacuuming of B-Tree indexes (Heikki Linnakangas, Tom)
•
Avoid extra scan of tables without indexes during VACUUM (Greg Stark)
•
Improve multicolumn GiST indexing (Oleg, Teodor)
•
Remove dead index entries before B-Tree page split (Junji Teramoto)
E.164.3.2. Server Changes •
Allow a forced switch to a new transaction log file (Simon, Tom) This is valuable for keeping warm standby slave servers in sync with the master. Transaction log file switching now also happens automatically during pg_stop_backup(). This ensures that all transaction log files needed for recovery can be archived immediately.
•
Add WAL informational functions (Simon) Add functions for interrogating the current transaction log insertion point and determining WAL filenames from the hex WAL locations displayed by pg_stop_backup() and related functions.
•
Improve recovery from a crash during WAL replay (Simon) The server now does periodic checkpoints during WAL recovery, so if there is a crash, future WAL recovery is shortened. This also eliminates the need for warm standby servers to replay the entire log since the base backup if they crash.
•
Improve reliability of long-term WAL replay (Heikki, Simon, Tom) Formerly, trying to roll forward through more than 2 billion transactions would not work due to XID wraparound. This meant warm standby servers had to be reloaded from fresh base backups periodically.
•
Add archive_timeout to force transaction log file switches at a given interval (Simon) This enforces a maximum replication delay for warm standby servers.
•
Add native LDAP authentication (Magnus Hagander) This is particularly useful for platforms that do not support PAM, such as Windows.
•
Add GRANT CONNECT ON DATABASE (Gevik Babakhani) This gives SQL-level control over database access. It works as an additional filter on top of the existing pg_hba.conf controls.
•
Add support for SSL Certificate Revocation List (CRL) files (Libor Hohoš) The server and libpq both recognize CRL files now.
•
GiST indexes are now clusterable (Teodor)
•
Remove routine autovacuum server log entries (Bruce) pg_stat_activity now shows autovacuum activity.
•
Track maximum XID age within individual tables, instead of whole databases (Alvaro) This reduces the overhead involved in preventing transaction ID wraparound, by avoiding unnecessary VACUUMs.
•
Add last vacuum and analyze timestamp columns to the stats collector (Larry Rosenman) These values now appear in the pg_stat_*_tables system views.
2448
Appendix E. Release Notes •
Improve performance of statistics monitoring, especially stats_command_string (Tom, Bruce) This release enables stats_command_string by default, now that its overhead is minimal. This means pg_stat_activity will now show all active queries by default.
•
Add a waiting column to pg_stat_activity (Tom) This allows pg_stat_activity to show all the information included in the ps display.
•
Add configuration parameter update_process_title to control whether the ps display is updated for every command (Bruce) On platforms where it is expensive to update the ps display, it might be worthwhile to turn this off and rely solely on pg_stat_activity for status information.
•
Allow units to be specified in configuration settings (Peter) For example, you can now set shared_buffers to 32MB rather than mentally converting sizes.
•
Add support for include directives in postgresql.conf (Joachim Wieland)
•
Improve logging of protocol-level prepare/bind/execute messages (Bruce, Tom) Such logging now shows statement names, bind parameter values, and the text of the query being executed. Also, the query text is properly included in logged error messages when enabled by log_min_error_statement.
•
Prevent max_stack_depth from being set to unsafe values On platforms where we can determine the actual kernel stack depth limit (which is most), make sure that the initial default value of max_stack_depth is safe, and reject attempts to set it to unsafely large values.
•
Enable highlighting of error location in query in more cases (Tom) The server is now able to report a specific error location for some semantic errors (such as unrecognized column name), rather than just for basic syntax errors as before.
•
Fix “failed to re-find parent key” errors in VACUUM (Tom)
•
Clean out pg_internal.init cache files during server restart (Simon) This avoids a hazard that the cache files might contain stale data after PITR recovery.
•
Fix race condition for truncation of a large relation across a gigabyte boundary by VACUUM (Tom)
•
Fix bug causing needless deadlock errors on row-level locks (Tom)
Each backend process is now its own process group leader (Tom) This allows query cancel to abort subprocesses invoked from a backend or archive/recovery process.
E.164.3.3. Query Changes •
Add INSERT/UPDATE/DELETE RETURNING (Jonah Harris, Tom) This allows these commands to return values, such as the computed serial key for a new row. In the UPDATE case, values from the updated version of the row are returned.
•
Add support for multiple-row VALUES clauses, per SQL standard (Joe, Tom)
2449
Appendix E. Release Notes This allows INSERT to insert multiple rows of constants, or queries to generate result sets using constants. For example, INSERT ... VALUES (...), (...), ...., and SELECT * FROM (VALUES (...), (...), ....) AS alias(f1, ...). •
Allow UPDATE and DELETE to use an alias for the target table (Atsushi Ogawa) The SQL standard does not permit an alias in these commands, but many database systems allow one anyway for notational convenience.
•
Allow UPDATE to set multiple columns with a list of values (Susanne Ebrecht) This is basically a short-hand for assigning the columns and values in pairs. The syntax is UPDATE tab SET (column, ...) = (val, ...).
•
Make row comparisons work per standard (Tom) The forms = now compare rows lexicographically, that is, compare the first elements, if equal compare the second elements, and so on. Formerly they expanded to an AND condition across all the elements, which was neither standard nor very useful.
•
Add CASCADE option to TRUNCATE (Joachim Wieland) This causes TRUNCATE to automatically include all tables that reference the specified table(s) via foreign keys. While convenient, this is a dangerous tool — use with caution!
•
Support FOR UPDATE and FOR SHARE in the same SELECT command (Tom)
•
Add IS NOT DISTINCT FROM (Pavel Stehule) This operator is similar to equality (=), but evaluates to true when both left and right operands are NULL, and to false when just one is, rather than yielding NULL in these cases.
•
Improve the length output used by UNION/INTERSECT/EXCEPT (Tom) When all corresponding columns are of the same defined length, that length is used for the result, rather than a generic length.
•
Allow ILIKE to work for multi-byte encodings (Tom) Internally, ILIKE now calls lower() and then uses LIKE. Locale-specific regular expression patterns still do not work in these encodings.
•
Enable standard_conforming_strings to be turned on (Kevin Grittner) This allows backslash escaping in strings to be disabled, making PostgreSQL more standardscompliant. The default is off for backwards compatibility, but future releases will default this to on.
•
Do not flatten subqueries that contain volatile functions in their target lists (Jaime Casanova) This prevents surprising behavior due to multiple evaluation of a volatile function (such as random() or nextval()). It might cause performance degradation in the presence of functions that are unnecessarily marked as volatile.
•
Add system views pg_prepared_statements and pg_cursors to show prepared statements and open cursors (Joachim Wieland, Neil) These are very useful in pooled connection setups.
•
Support portal parameters in EXPLAIN and EXECUTE (Tom) This allows, for example, JDBC ? parameters to work in these commands.
•
If SQL-level PREPARE parameters are unspecified, infer their types from the content of the query (Neil) Protocol-level PREPARE already did this.
2450
Appendix E. Release Notes •
Allow LIMIT and OFFSET to exceed two billion (Dhanaraj M)
E.164.3.4. Object Manipulation Changes •
Add TABLESPACE clause to CREATE TABLE AS (Neil) This allows a tablespace to be specified for the new table.
•
Add ON COMMIT clause to CREATE TABLE AS (Neil) This allows temporary tables to be truncated or dropped on transaction commit. The default behavior is for the table to remain until the session ends.
•
Add INCLUDING CONSTRAINTS to CREATE TABLE LIKE (Greg Stark) This allows easy copying of CHECK constraints to a new table.
•
Allow the creation of placeholder (shell) types (Martijn van Oosterhout) A shell type declaration creates a type name, without specifying any of the details of the type. Making a shell type is useful because it allows cleaner declaration of the type’s input/output functions, which must exist before the type can be defined “for real”. The syntax is CREATE TYPE typename.
•
Aggregate functions now support multiple input parameters (Sergey Koposov, Tom)
•
Add new aggregate creation syntax (Tom) The new syntax is CREATE AGGREGATE aggname (input_type) (parameter_list). This more naturally supports the new multi-parameter aggregate functionality. The previous syntax is still supported.
•
Add ALTER ROLE PASSWORD NULL to remove a previously set role password (Peter)
•
Add DROP object IF EXISTS for many object types (Andrew) This allows DROP operations on non-existent objects without generating an error.
•
Add DROP OWNED to drop all objects owned by a role (Alvaro)
•
Add REASSIGN OWNED to reassign ownership of all objects owned by a role (Alvaro) This, and DROP OWNED above, facilitate dropping roles.
•
Add GRANT ON SEQUENCE syntax (Bruce) This was added for setting sequence-specific permissions. GRANT ON TABLE for sequences is still supported for backward compatibility.
•
Add USAGE permission for sequences that allows only currval() and nextval(), not setval() (Bruce) USAGE permission allows more fine-grained control over sequence access. Granting USAGE allows users to increment a sequence, but prevents them from setting the sequence to an arbitrary value using setval().
•
Add ALTER TABLE [ NO ] INHERIT (Greg Stark) This allows inheritance to be adjusted dynamically, rather than just at table creation and destruction. This is very valuable when using inheritance to implement table partitioning.
•
Allow comments on global objects to be stored globally (Kris Jurka) Previously, comments attached to databases were stored in individual databases, making them ineffective, and there was no provision at all for comments on roles or tablespaces. This change adds a
2451
Appendix E. Release Notes new shared catalog pg_shdescription and stores comments on databases, roles, and tablespaces therein.
E.164.3.5. Utility Command Changes •
Add option to allow indexes to be created without blocking concurrent writes to the table (Greg Stark, Tom) The new syntax is CREATE INDEX CONCURRENTLY. The default behavior is still to block table modification while an index is being created.
•
Provide advisory locking functionality (Abhijit Menon-Sen, Tom) This is a new locking API designed to replace what used to be in /contrib/userlock. The userlock code is now on pgfoundry.
•
Allow COPY to dump a SELECT query (Zoltan Boszormenyi, Karel Zak) This allows COPY to dump arbitrary SQL queries. The syntax is COPY (SELECT ...) TO.
•
Make the COPY command return a command tag that includes the number of rows copied (Volkan YAZICI)
•
Allow VACUUM to expire rows without being affected by other concurrent VACUUM operations (Hannu Krossing, Alvaro, Tom)
•
Make initdb detect the operating system locale and set the default DateStyle accordingly (Peter) This makes it more likely that the installed postgresql.conf DateStyle value will be as desired.
•
Reduce number of progress messages displayed by initdb (Tom)
E.164.3.6. Date/Time Changes •
Allow full timezone names in timestamp input values (Joachim Wieland) For example, ’2006-05-24 21:11 America/New_York’::timestamptz.
•
Support configurable timezone abbreviations (Joachim Wieland) A desired set of timezone abbreviations can be chosen via the configuration parameter timezone_abbreviations.
•
Add pg_timezone_abbrevs and pg_timezone_names views to show supported timezones (Magnus Hagander)
•
Add clock_timestamp(), statement_timestamp(), and transaction_timestamp() (Bruce) clock_timestamp() is the current wall-clock time, statement_timestamp() is the time the current statement arrived at the server, and transaction_timestamp() is an alias for now().
•
Allow to_char() to print localized month and day names (Euler Taveira de Oliveira)
•
Allow to_char(time) and to_char(interval) to output AM/PM specifications (Bruce) Intervals and times are treated as 24-hour periods, e.g. 25 hours is considered AM.
•
Add new function justify_interval() to adjust interval units (Mark Dilger)
2452
Appendix E. Release Notes •
Allow timezone offsets up to 14:59 away from GMT Kiribati uses GMT+14, so we’d better accept that.
Allow assignment to array elements not contiguous with the existing entries (Tom) The intervening array positions will be filled with nulls. This is per SQL standard.
•
New built-in operators for array-subset comparisons (@>, and < comparisons (Bruce)
•
Add XLOG_BLCKSZ as independent from BLCKSZ (Mark Wong)
•
Add LWLOCK_STATS define to report locking activity (Tom)
•
Emit warnings for unknown configure options (Martijn van Oosterhout)
•
Add server support for “plugin” libraries that can be used for add-on tasks such as debugging and performance measurement (Korry Douglas) This consists of two features: a table of “rendezvous variables” that allows separately-loaded shared libraries to communicate, and a new configuration parameter local_preload_libraries that allows libraries to be loaded into specific sessions without explicit cooperation from the client application. This allows external add-ons to implement features such as a PL/pgSQL debugger.
•
Rename
existing
configuration
parameter
preload_libraries
to
shared_preload_libraries (Tom)
This was done for clarity in comparison to local_preload_libraries. •
Add new configuration parameter server_version_num (Greg Sabino Mullane)
2456
Appendix E. Release Notes This is like server_version, but is an integer, e.g. 80200. This allows applications to make version checks more easily. •
Add a configuration parameter seq_page_cost (Tom)
•
Re-implement the regression test script as a C program (Magnus, Tom)
•
Allow loadable modules to allocate shared memory and lightweight locks (Marc Munro)
•
Add automatic initialization and finalization of dynamically loaded libraries (Ralf Engelschall, Tom) New functions _PG_init() and _PG_fini() are called if the library defines such symbols. Hence we no longer need to specify an initialization function in shared_preload_libraries; we can assume that the library used the _PG_init() convention instead.
•
Add PG_MODULE_MAGIC header block to all shared object files (Martijn van Oosterhout) The magic block prevents version mismatches between loadable object files and servers.
•
Add shared library support for AIX (Laurenz Albe)
•
New XML documentation section (Bruce)
E.164.3.17. Contrib Changes •
•
Major tsearch2 improvements (Oleg, Teodor) •
multibyte encoding support, including UTF8
•
query rewriting support
•
improved ranking functions
•
thesaurus dictionary support
•
Ispell dictionaries now recognize MySpell format, used by OpenOffice
•
GIN support
Add adminpack module containing Pgadmin administration functions (Dave) These functions provide additional file system access routines not present in the default PostgreSQL server.
•
Add sslinfo module (Victor Wagner) Reports information about the current connection’s SSL certificate.
•
Add pgrowlocks module (Tatsuo) This shows row locking information for a specified table.
•
Add hstore module (Oleg, Teodor)
•
Add isn module, replacing isbn_issn (Jeremy Kronuz) This new implementation supports EAN13, UPC, ISBN (books), ISMN (music), and ISSN (serials).
•
Add index information functions to pgstattuple (ITAGAKI Takahiro, Satoshi Nagayasu)
•
Add pg_freespacemap module to display free space map information (Mark Kirkwood)
•
pgcrypto now has all planned functionality (Marko Kreen)
2457
Appendix E. Release Notes
•
•
Include iMath library in pgcrypto to have the public-key encryption functions always available.
•
Add SHA224 algorithm that was missing in OpenBSD code.
•
Activate builtin code for SHA224/256/384/512 hashes on older OpenSSL to have those algorithms always available.
•
New function gen_random_bytes() that returns cryptographically strong randomness. Useful for generating encryption keys.
•
Remove digest_exists(), hmac_exists() and cipher_exists() functions.
Improvements to cube module (Joshua Reich) New functions are cube(float[]), cube(float[], float[]), and cube_subset(cube, int4[]).
•
Add async query capability to dblink (Kai Londenberg, Joe Conway)
•
New operators for array-subset comparisons (@>, for polygons consistent with the box "over" operators (Tom)
• CREATE LANGUAGE
can ignore the provided arguments in favor of information from
pg_pltemplate (Tom)
A new system catalog pg_pltemplate has been defined to carry information about the preferred definitions of procedural languages (such as whether they have validator functions). When an entry exists in this catalog for the language being created, CREATE LANGUAGE will ignore all its parameters except the language name and instead use the catalog information. This measure was taken because of increasing problems with obsolete language definitions being loaded by old dump files. As of 8.1, pg_dump will dump procedural language definitions as just CREATE LANGUAGE name, relying on a template entry to exist at load time. We expect this will be a more future-proof representation. •
Make pg_cancel_backend(int) return a boolean rather than an integer (Neil)
•
Some users are having problems loading UTF-8 data into 8.1.X. This is because previous versions allowed invalid UTF-8 byte sequences to be entered into the database, and this release properly accepts only valid UTF-8 sequences. One way to correct a dumpfile is to run the command iconv -c -f UTF-8 -t UTF-8 -o cleanfile.sql dumpfile.sql. The -c option removes invalid character sequences. A diff of the two files will show the sequences that are invalid. iconv reads the entire input file into memory so it might be necessary to use split to break up the dump into multiple smaller files for processing.
E.188.3. Additional Changes Below you will find a detailed account of the additional changes between PostgreSQL 8.1 and the previous major release.
E.188.3.1. Performance Improvements •
Improve GiST and R-tree index performance (Neil)
•
Improve the optimizer, including auto-resizing of hash joins (Tom)
•
Overhaul internal API in several areas
•
Change WAL record CRCs from 64-bit to 32-bit (Tom) We determined that the extra cost of computing 64-bit CRCs was significant, and the gain in reliability too marginal to justify it.
•
Prevent writing large empty gaps in WAL pages (Tom)
•
Improve spinlock behavior on SMP machines, particularly Opterons (Tom)
•
Allow nonconsecutive index columns to be used in a multicolumn index (Tom) For example, this allows an index on columns a,b,c to be used in a query with WHERE a = 4 and c = 10.
•
Skip WAL logging for CREATE TABLE AS / SELECT INTO (Simon) Since a crash during CREATE TABLE AS would cause the table to be dropped during recovery, there is no reason to WAL log as the table is loaded. (Logging still happens if WAL archiving is enabled, however.)
2488
Appendix E. Release Notes •
Allow concurrent GiST index access (Teodor, Oleg)
•
Add configuration parameter full_page_writes to control writing full pages to WAL (Bruce) To prevent partial disk writes from corrupting the database, PostgreSQL writes a complete copy of each database disk page to WAL the first time it is modified after a checkpoint. This option turns off that functionality for more speed. This is safe to use with battery-backed disk caches where partial page writes cannot happen.
•
Use O_DIRECT if available when using O_SYNC for wal_sync_method (Itagaki Takahiro) O_DIRECT causes disk writes to bypass the kernel cache, and for WAL writes, this improves per-
formance. •
Improve COPY FROM performance (Alon Goldshuv) This was accomplished by reading COPY input in larger chunks, rather than character by character.
•
Improve the performance of COUNT(), SUM, AVG(), STDDEV(), and VARIANCE() (Neil, Tom)
E.188.3.2. Server Changes •
Prevent problems due to transaction ID (XID) wraparound (Tom) The server will now warn when the transaction counter approaches the wraparound point. If the counter becomes too close to wraparound, the server will stop accepting queries. This ensures that data is not lost before needed vacuuming is performed.
•
•
Fix problems with object IDs (OIDs) conflicting with existing system objects after the OID counter has wrapped around (Tom) Add warning about the need to increase max_fsm_relations and max_fsm_pages during VACUUM (Ron Mayer)
•
Add temp_buffers configuration parameter to allow users to determine the size of the local buffer area for temporary table access (Tom)
•
Add session start time and client IP address to pg_stat_activity (Magnus)
•
Adjust pg_stat views for bitmap scans (Tom) The meanings of some of the fields have changed slightly.
•
Enhance pg_locks view (Tom)
•
Log queries for client-side PREPARE and EXECUTE (Simon)
•
Allow Kerberos name and user name case sensitivity to be specified in postgresql.conf (Magnus)
•
Add configuration parameter krb_server_hostname so that the server host name can be specified as part of service principal (Todd Kover) If not set, any service principal matching an entry in the keytab can be used. This is new Kerberos matching behavior in this release.
•
Add log_line_prefix options for millisecond timestamps (%m) and remote host (%h) (Ed L.)
•
Add WAL logging for GiST indexes (Teodor, Oleg) GiST indexes are now safe for crash and point-in-time recovery.
•
Remove old *.backup files when we do pg_stop_backup() (Bruce) This prevents a large number of *.backup files from existing in pg_xlog/.
2489
Appendix E. Release Notes •
Add configuration parameters to control TCP/IP keep-alive times for idle, interval, and count (Oliver Jowett) These values can be changed to allow more rapid detection of lost client connections.
•
Add per-user and per-database connection limits (Petr Jelinek) Using ALTER USER and ALTER DATABASE, limits can now be enforced on the maximum number of sessions that can concurrently connect as a specific user or to a specific database. Setting the limit to zero disables user or database connections.
•
Allow more than two gigabytes of shared memory and per-backend work memory on 64-bit machines (Koichi Suzuki)
•
New system catalog pg_pltemplate allows overriding obsolete procedural-language definitions in dump files (Tom)
E.188.3.3. Query Changes •
Add temporary views (Koju Iijima, Neil)
•
Fix HAVING without any aggregate functions or GROUP BY so that the query returns a single group (Tom) Previously, such a case would treat the HAVING clause the same as a WHERE clause. This was not per spec.
•
Add USING clause to allow additional tables to be specified to DELETE (Euler Taveira de Oliveira, Neil) In prior releases, there was no clear method for specifying additional tables to be used for joins in a DELETE statement. UPDATE already has a FROM clause for this purpose.
•
Add support for \x hex escapes in backend and ecpg strings (Bruce) This is just like the standard C \x escape syntax. Octal escapes were already supported.
•
Add BETWEEN SYMMETRIC query syntax (Pavel Stehule) This feature allows BETWEEN comparisons without requiring the first value to be less than the second. For example, 2 BETWEEN [ASYMMETRIC] 3 AND 1 returns false, while 2 BETWEEN SYMMETRIC 3 AND 1 returns true. BETWEEN ASYMMETRIC was already supported.
•
Add NOWAIT option to SELECT ... FOR UPDATE/SHARE (Hans-Juergen Schoenig) While the statement_timeout configuration parameter allows a query taking more than a certain amount of time to be canceled, the NOWAIT option allows a query to be canceled as soon as a SELECT ... FOR UPDATE/SHARE command cannot immediately acquire a row lock.
E.188.3.4. Object Manipulation Changes •
Track dependencies of shared objects (Alvaro) PostgreSQL allows global tables (users, databases, tablespaces) to reference information in multiple databases. This addition adds dependency information for global tables, so, for example, user ownership can be tracked across databases, so a user who owns something in any database can no longer be removed. Dependency tracking already existed for database-local objects.
•
Allow limited ALTER OWNER commands to be performed by the object owner (Stephen Frost)
2490
Appendix E. Release Notes Prior releases allowed only superusers to change object owners. Now, ownership can be transferred if the user executing the command owns the object and would be able to create it as the new owner (that is, the user is a member of the new owning role and that role has the CREATE permission that would be needed to create the object afresh). •
Add ALTER object SET SCHEMA capability for some object types (tables, functions, types) (Bernd Helmle) This allows objects to be moved to different schemas.
•
Add ALTER TABLE ENABLE/DISABLE TRIGGER to disable triggers (Satoshi Nagayasu)
E.188.3.5. Utility Command Changes •
Allow TRUNCATE to truncate multiple tables in a single command (Alvaro) Because of referential integrity checks, it is not allowed to truncate a table that is part of a referential integrity constraint. Using this new functionality, TRUNCATE can be used to truncate such tables, if both tables involved in a referential integrity constraint are truncated in a single TRUNCATE command.
•
Properly process carriage returns and line feeds in COPY CSV mode (Andrew) In release 8.0, carriage returns and line feeds in CSV COPY TO were processed in an inconsistent manner. (This was documented on the TODO list.)
•
Add COPY WITH CSV HEADER to allow a header line as the first line in COPY (Andrew) This allows handling of the common CSV usage of placing the column names on the first line of the data file. For COPY TO, the first line contains the column names, and for COPY FROM, the first line is ignored.
•
On Windows, display better sub-second precision in EXPLAIN ANALYZE (Magnus)
•
Add trigger duration display to EXPLAIN ANALYZE (Tom) Prior releases included trigger execution time as part of the total execution time, but did not show it separately. It is now possible to see how much time is spent in each trigger.
•
Add support for \x hex escapes in COPY (Sergey Ten) Previous releases only supported octal escapes.
•
Make SHOW ALL include variable descriptions (Matthias Schmidt) SHOW varname still only displays the variable’s value and does not include the description.
•
Make initdb create a new standard database called postgres, and convert utilities to use postgres rather than template1 for standard lookups (Dave) In prior releases, template1 was used both as a default connection for utilities like createuser, and as a template for new databases. This caused CREATE DATABASE to sometimes fail, because a new database cannot be created if anyone else is in the template database. With this change, the default connection database is now postgres, meaning it is much less likely someone will be using template1 during CREATE DATABASE.
•
Create new reindexdb command-line utility by moving /contrib/reindexdb into the server (Euler Taveira de Oliveira)
2491
Appendix E. Release Notes
E.188.3.6. Data Type and Function Changes •
Add MAX() and MIN() aggregates for array types (Koju Iijima)
•
Fix to_date() and to_timestamp() to behave reasonably when CC and YY fields are both used (Karel Zak) If the format specification contains CC and a year specification is YYY or longer, ignore the CC. If the year specification is YY or shorter, interpret CC as the previous century.
Add support for numeric ^ numeric based on power(numeric, numeric) The function already existed, but there was no operator assigned to it.
•
Fix NUMERIC modulus by properly truncating the quotient during computation (Bruce) In previous releases, modulus for large values sometimes returned negative results due to rounding of the quotient.
•
Add a function lastval() (Dennis Björklund) lastval() is a simplified version of currval(). It automatically determines the proper sequence name based on the most recent nextval() or setval() call performed by the current session.
•
Add to_timestamp(DOUBLE PRECISION) (Michael Glaesemann) Converts Unix seconds since 1970 to a TIMESTAMP WITH TIMEZONE.
•
Add pg_postmaster_start_time() function (Euler Taveira de Oliveira, Matthias Schmidt)
•
Allow the full use of time zone names in AT TIME ZONE, not just the short list previously available (Magnus) Previously, only a predefined list of time zone names were supported by AT TIME ZONE. Now any supported time zone name can be used, e.g.: SELECT CURRENT_TIMESTAMP AT TIME ZONE ’Europe/London’;
In the above query, the time zone used is adjusted based on the daylight saving time rules that were in effect on the supplied date. •
Add GREATEST() and LEAST() variadic functions (Pavel Stehule) These functions take a variable number of arguments and return the greatest or least value among the arguments.
•
Add pg_column_size() (Mark Kirkwood) This returns storage size of a column, which might be compressed.
•
Add regexp_replace() (Atsushi Ogawa) This allows regular expression replacement, like sed. An optional flag argument allows selection of global (replace all) and case-insensitive modes.
•
Fix interval division and multiplication (Bruce) Previous versions sometimes returned unjustified results, like ’4 months’::interval / 5 returning ’1 mon -6 days’.
•
Fix roundoff behavior in timestamp, time, and interval output (Tom) This fixes some cases in which the seconds field would be shown as 60 instead of incrementing the higher-order fields.
2492
Appendix E. Release Notes •
Add a separate day field to type interval so a one day interval can be distinguished from a 24 hour interval (Michael Glaesemann) Days that contain a daylight saving time adjustment are not 24 hours long, but typically 23 or 25 hours. This change creates a conceptual distinction between intervals of “so many days” and intervals of “so many hours”. Adding 1 day to a timestamp now gives the same local time on the next day even if a daylight saving time adjustment occurs between, whereas adding 24 hours will give a different local time when this happens. For example, under US DST rules: ’2005-04-03 00:00:00-05’ + ’1 day’ = ’2005-04-04 00:00:00-04’ ’2005-04-03 00:00:00-05’ + ’24 hours’ = ’2005-04-04 01:00:00-04’
•
Add justify_days() and justify_hours() (Michael Glaesemann) These functions, respectively, adjust days to an appropriate number of full months and days, and adjust hours to an appropriate number of full days and hours.
•
Move /contrib/dbsize into the backend, and rename some of the functions (Dave Page, Andreas Pflug) •
pg_tablespace_size()
•
pg_database_size()
•
pg_relation_size()
•
pg_total_relation_size()
•
pg_size_pretty()
pg_total_relation_size() includes indexes and TOAST tables. •
Add functions for read-only file access to the cluster directory (Dave Page, Andreas Pflug) •
pg_stat_file()
•
pg_read_file()
•
pg_ls_dir()
•
Add pg_reload_conf() to force reloading of the configuration files (Dave Page, Andreas Pflug)
•
Add pg_rotate_logfile() to force rotation of the server log file (Dave Page, Andreas Pflug)
•
Change pg_stat_* views to include TOAST tables (Tom)
E.188.3.7. Encoding and Locale Changes •
Rename some encodings to be more consistent and to follow international standards (Bruce) •
UNICODE is now UTF8
•
ALT is now WIN866
•
WIN is now WIN1251
•
TCVN is now WIN1258
The original names still work.
2493
Appendix E. Release Notes •
Add support for WIN1252 encoding (Roland Volkmann)
•
Add support for four-byte UTF8 characters (John Hansen) Previously only one, two, and three-byte UTF8 characters were supported. This is particularly important for support for some Chinese character sets.
•
Allow direct conversion between EUC_JP and SJIS to improve performance (Atsushi Ogawa)
•
Allow the UTF8 encoding to work on Windows (Magnus) This is done by mapping UTF8 to the Windows-native UTF16 implementation.
E.188.3.8. General Server-Side Language Changes •
Fix ALTER LANGUAGE RENAME (Sergey Yatskevich)
•
Allow function characteristics, like strictness and volatility, to be modified via ALTER FUNCTION (Neil)
•
Increase the maximum number of function arguments to 100 (Tom)
•
Allow SQL and PL/pgSQL functions to use OUT and INOUT parameters (Tom) OUT is an alternate way for a function to return values. Instead of using RETURN, values can be returned by assigning to parameters declared as OUT or INOUT. This is notationally simpler in some
cases, particularly so when multiple values need to be returned. While returning multiple values from a function was possible in previous releases, this greatly simplifies the process. (The feature will be extended to other server-side languages in future releases.) •
Move language handler functions into the pg_catalog schema This makes it easier to drop the public schema if desired.
•
Add SPI_getnspname() to SPI (Neil)
E.188.3.9. PL/pgSQL Server-Side Language Changes •
Overhaul the memory management of PL/pgSQL functions (Neil) The parsetree of each function is now stored in a separate memory context. This allows this memory to be easily reclaimed when it is no longer needed.
•
Check function syntax at CREATE FUNCTION time, rather than at runtime (Neil) Previously, most syntax errors were reported only when the function was executed.
•
Allow OPEN to open non-SELECT queries like EXPLAIN and SHOW (Tom)
•
No longer require functions to issue a RETURN statement (Tom) This is a byproduct of the newly added OUT and INOUT functionality. RETURN can be omitted when it is not needed to provide the function’s return value.
•
Add support for an optional INTO clause to PL/pgSQL’s EXECUTE statement (Pavel Stehule, Neil)
•
Make CREATE TABLE AS set ROW_COUNT (Tom)
•
Define SQLSTATE and SQLERRM to return the SQLSTATE and error message of the current exception (Pavel Stehule, Neil) These variables are only defined inside exception blocks.
2494
Appendix E. Release Notes •
Allow the parameters to the RAISE statement to be expressions (Pavel Stehule, Neil)
•
Add a loop CONTINUE statement (Pavel Stehule, Neil)
•
Allow block and loop labels (Pavel Stehule)
E.188.3.10. PL/Perl Server-Side Language Changes •
Allow large result sets to be returned efficiently (Abhijit Menon-Sen) This allows functions to use return_next() to avoid building the entire result set in memory.
•
Allow one-row-at-a-time retrieval of query results (Abhijit Menon-Sen) This allows functions to use spi_query() and spi_fetchrow() to avoid accumulating the entire result set in memory.
•
Force PL/Perl to handle strings as UTF8 if the server encoding is UTF8 (David Kamholz)
•
Add a validator function for PL/Perl (Andrew) This allows syntax errors to be reported at definition time, rather than execution time.
•
Allow PL/Perl to return a Perl array when the function returns an array type (Andrew) This basically maps PostgreSQL arrays to Perl arrays.
•
Allow Perl nonfatal warnings to generate NOTICE messages (Andrew)
•
Allow Perl’s strict mode to be enabled (Andrew)
E.188.3.11. psql Changes •
Add \set ON_ERROR_ROLLBACK to allow statements in a transaction to error without affecting the rest of the transaction (Greg Sabino Mullane) This is basically implemented by wrapping every statement in a sub-transaction.
•
Add support for \x hex strings in psql variables (Bruce) Octal escapes were already supported.
•
Add support for troff -ms output format (Roger Leigh)
•
Allow the history file location to be controlled by HISTFILE (Andreas Seltenreich) This allows configuration of per-database history storage.
•
Prevent \x (expanded mode) from affecting the output of \d tablename (Neil)
•
Add -L option to psql to log sessions (Lorne Sunley) This option was added because some operating systems do not have simple command-line activity logging functionality.
•
Make \d show the tablespaces of indexes (Qingqing Zhou)
•
Allow psql help (\h) to make a best guess on the proper help information (Greg Sabino Mullane) This allows the user to just add \h to the front of the syntax error query and get help on the supported syntax. Previously any additional query text beyond the command name had to be removed to use \h.
2495
Appendix E. Release Notes •
Add \pset numericlocale to allow numbers to be output in a locale-aware format (Eugen Nedelcu) For example, using C locale 100000 would be output as 100,000.0 while a European locale might output this value as 100.000,0.
•
Make startup banner show both server version number and psql’s version number, when they are different (Bruce) Also, a warning will be shown if the server and psql are from different major releases.
E.188.3.12. pg_dump Changes •
Add -n / --schema switch to pg_restore (Richard van den Berg) This allows just the objects in a specified schema to be restored.
•
Allow pg_dump to dump large objects even in text mode (Tom) With this change, large objects are now always dumped; the former -b switch is a no-op.
•
Allow pg_dump to dump a consistent snapshot of large objects (Tom)
•
Dump comments for large objects (Tom)
•
Add --encoding to pg_dump (Magnus Hagander) This allows a database to be dumped in an encoding that is different from the server’s encoding. This is valuable when transferring the dump to a machine with a different encoding.
•
Rely on pg_pltemplate for procedural languages (Tom) If the call handler for a procedural language is in the pg_catalog schema, pg_dump does not dump the handler. Instead, it dumps the language using just CREATE LANGUAGE name, relying on the pg_pltemplate catalog to provide the language’s creation parameters at load time.
E.188.3.13. libpq Changes •
Add a PGPASSFILE environment variable to specify the password file’s filename (Andrew)
•
Add lo_create(), that is similar to lo_creat() but allows the OID of the large object to be specified (Tom)
•
Make libpq consistently return an error to the client application on malloc() failure (Neil)
E.188.3.14. Source Code Changes •
Fix pgxs to support building against a relocated installation
•
Add spinlock support for the Itanium processor using Intel compiler (Vikram Kalsi)
•
Add Kerberos 5 support for Windows (Magnus)
•
Add Chinese FAQ ([email protected])
•
Rename Rendezvous to Bonjour to match OS/X feature renaming (Bruce)
•
Add support for fsync_writethrough on Darwin (Chris Campbell)
2496
Appendix E. Release Notes •
Streamline the passing of information within the server, the optimizer, and the lock system (Tom)
•
Allow pg_config to be compiled using MSVC (Andrew) This is required to build DBD::Pg using MSVC.
•
Remove support for Kerberos V4 (Magnus) Kerberos 4 had security vulnerabilities and is no longer maintained.
•
Code cleanups (Coverity static analysis performed by EnterpriseDB)
•
Modify postgresql.conf to use documentation defaults on/off rather than true/false (Bruce)
•
Enhance pg_config to be able to report more build-time values (Tom)
•
Allow libpq to be built thread-safe on Windows (Dave Page)
•
Allow IPv6 connections to be used on Windows (Andrew)
•
Add Server Administration documentation about I/O subsystem reliability (Bruce)
•
Move private declarations from gist.h to gist_private.h (Neil) In previous releases, gist.h contained both the public GiST API (intended for use by authors of GiST index implementations) as well as some private declarations used by the implementation of GiST itself. The latter have been moved to a separate file, gist_private.h. Most GiST index implementations should be unaffected.
•
Overhaul GiST memory management (Neil) GiST methods are now always invoked in a short-lived memory context. Therefore, memory allocated via palloc() will be reclaimed automatically, so GiST index implementations do not need to manually release allocated memory via pfree().
E.188.3.15. Contrib Changes •
Add /contrib/pg_buffercache contrib module (Mark Kirkwood) This displays the contents of the buffer cache, for debugging and performance tuning purposes.
•
Remove /contrib/array because it is obsolete (Tom)
•
Clean up the /contrib/lo module (Tom)
•
Move /contrib/findoidjoins to /src/tools (Tom)
•
Remove the , & operators from /contrib/cube These operators were not useful.
•
Improve /contrib/btree_gist (Janko Richter)
•
Improve /contrib/pgbench (Tomoaki Sato, Tatsuo) There is now a facility for testing with SQL command scripts given by the user, instead of only a hard-wired command sequence.
•
Improve /contrib/pgcrypto (Marko Kreen) •
Implementation of OpenPGP symmetric-key and public-key encryption Both RSA and Elgamal public-key algorithms are supported.
•
Stand alone build: include SHA256/384/512 hashes, Fortuna PRNG
2497
Appendix E. Release Notes •
OpenSSL build: support 3DES, use internal AES with OpenSSL < 0.9.7
•
Take build parameters (OpenSSL, zlib) from configure result There is no need to edit the Makefile anymore.
•
Remove support for libmhash and libmcrypt
E.189. Release 8.0.26 Release date: 2010-10-04 This release contains a variety of fixes from 8.0.25. For information about new features in the 8.0 major release, see Section E.215. This is expected to be the last PostgreSQL release in the 8.0.X series. Users are encouraged to update to a newer release branch soon.
E.189.1. Migration to Version 8.0.26 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.22, see Section E.193.
E.189.2. Changes •
Use a separate interpreter for each calling SQL userid in PL/Perl and PL/Tcl (Tom Lane) This change prevents security problems that can be caused by subverting Perl or Tcl code that will be executed later in the same session under another SQL user identity (for example, within a SECURITY DEFINER function). Most scripting languages offer numerous ways that that might be done, such as redefining standard functions or operators called by the target function. Without this change, any SQL user with Perl or Tcl language usage rights can do essentially anything with the SQL privileges of the target function’s owner. The cost of this change is that intentional communication among Perl and Tcl functions becomes more difficult. To provide an escape hatch, PL/PerlU and PL/TclU functions continue to use only one interpreter per session. This is not considered a security issue since all such functions execute at the trust level of a database superuser already. It is likely that third-party procedural languages that claim to offer trusted execution have similar security issues. We advise contacting the authors of any PL you are depending on for securitycritical purposes. Our thanks to Tim Bunce for pointing out this issue (CVE-2010-3433).
•
Prevent possible crashes in pg_get_expr() by disallowing it from being called with an argument that is not one of the system catalog columns it’s intended to be used with (Heikki Linnakangas, Tom Lane)
Appendix E. Release Notes This occurred when a sub-select contains a join alias reference that expands into an expression containing another sub-select. •
Defend against functions returning setof record where not all the returned rows are actually of the same rowtype (Tom Lane)
•
Take care to fsync the contents of lockfiles (both postmaster.pid and the socket lockfile) while writing them (Tom Lane) This omission could result in corrupted lockfile contents if the machine crashes shortly after postmaster start. That could in turn prevent subsequent attempts to start the postmaster from succeeding, until the lockfile is manually removed.
•
Avoid recursion while assigning XIDs to heavily-nested subtransactions (Andres Freund, Robert Haas) The original coding could result in a crash if there was limited stack space.
•
Fix log_line_prefix’s %i escape, which could produce junk early in backend startup (Tom Lane)
•
Fix possible data corruption in ALTER TABLE ... SET TABLESPACE when archiving is enabled (Jeff Davis)
•
Allow CREATE DATABASE and ALTER DATABASE ... SET TABLESPACE to be interrupted by query-cancel (Guillaume Lelarge)
•
In PL/Python, defend against null pointer results from PyCObject_AsVoidPtr and PyCObject_FromVoidPtr (Peter Eisentraut)
•
Improve contrib/dblink’s handling of tables containing dropped columns (Tom Lane)
•
Fix connection leak after “duplicate connection name” errors in contrib/dblink (Itagaki Takahiro)
•
Fix contrib/dblink to handle connection names longer than 62 bytes correctly (Itagaki Takahiro)
•
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
•
Update time zone data files to tzdata release 2010l for DST law changes in Egypt and Palestine; also historical corrections for Finland. This change also adds new names for two Micronesian timezones: Pacific/Chuuk is now preferred over Pacific/Truk (and the preferred abbreviation is CHUT not TRUT) and Pacific/Pohnpei is preferred over Pacific/Ponape.
E.190. Release 8.0.25 Release date: 2010-05-17 This release contains a variety of fixes from 8.0.24. For information about new features in the 8.0 major release, see Section E.215. The PostgreSQL community will stop releasing updates for the 8.0.X release series in July 2010. Users are encouraged to update to a newer release branch soon.
2499
Appendix E. Release Notes
E.190.1. Migration to Version 8.0.25 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.22, see Section E.193.
E.190.2. Changes •
Enforce restrictions in plperl using an opmask applied to the whole interpreter, instead of using Safe.pm (Tim Bunce, Andrew Dunstan) Recent developments have convinced us that Safe.pm is too insecure to rely on for making plperl trustable. This change removes use of Safe.pm altogether, in favor of using a separate interpreter with an opcode mask that is always applied. Pleasant side effects of the change include that it is now possible to use Perl’s strict pragma in a natural way in plperl, and that Perl’s $a and $b variables work as expected in sort routines, and that function compilation is significantly faster. (CVE-2010-1169)
•
Prevent PL/Tcl from executing untrustworthy code from pltcl_modules (Tom) PL/Tcl’s feature for autoloading Tcl code from a database table could be exploited for trojan-horse attacks, because there was no restriction on who could create or insert into that table. This change disables the feature unless pltcl_modules is owned by a superuser. (However, the permissions on the table are not checked, so installations that really need a less-than-secure modules table can still grant suitable privileges to trusted non-superusers.) Also, prevent loading code into the unrestricted “normal” Tcl interpreter unless we are really going to execute a pltclu function. (CVE-2010-1170)
•
Do not allow an unprivileged user to reset superuser-only parameter settings (Alvaro) Previously, if an unprivileged user ran ALTER USER ... RESET ALL for himself, or ALTER DATABASE ... RESET ALL for a database he owns, this would remove all special parameter settings for the user or database, even ones that are only supposed to be changeable by a superuser. Now, the ALTER will only remove the parameters that the user has permission to change.
•
Avoid possible crash during backend shutdown if shutdown occurs when a CONTEXT addition would be made to log entries (Tom) In some cases the context-printing function would fail because the current transaction had already been rolled back when it came time to print a log message.
•
Update pl/perl’s ppport.h for modern Perl versions (Andrew)
•
Fix assorted memory leaks in pl/python (Andreas Freund, Tom)
•
Prevent infinite recursion in psql when expanding a variable that refers to itself (Tom)
•
Ensure that contrib/pgstattuple functions respond to cancel interrupts promptly (Tatsuhito Kasahara)
•
Make server startup deal properly with the case that shmget() returns EINVAL for an existing shared memory segment (Tom) This behavior has been observed on BSD-derived kernels including OS X. It resulted in an entirelymisleading startup failure complaining that the shared memory request size was too large.
•
Update time zone data files to tzdata release 2010j for DST law changes in Argentina, Australian Antarctic, Bangladesh, Mexico, Morocco, Pakistan, Palestine, Russia, Syria, Tunisia; also historical corrections for Taiwan.
2500
Appendix E. Release Notes
E.191. Release 8.0.24 Release date: 2010-03-15 This release contains a variety of fixes from 8.0.23. For information about new features in the 8.0 major release, see Section E.215. The PostgreSQL community will stop releasing updates for the 8.0.X release series in July 2010. Users are encouraged to update to a newer release branch soon.
E.191.1. Migration to Version 8.0.24 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.22, see Section E.193.
E.191.2. Changes •
Add new configuration parameter ssl_renegotiation_limit to control how often we do session key renegotiation for an SSL connection (Magnus) This can be set to zero to disable renegotiation completely, which may be required if a broken SSL library is used. In particular, some vendors are shipping stopgap patches for CVE-2009-3555 that cause renegotiation attempts to fail.
•
Fix possible crashes when trying to recover from a failure in subtransaction start (Tom)
•
Fix server memory leak associated with use of savepoints and a client encoding different from server’s encoding (Tom)
•
Make substring() for bit types treat any negative length as meaning “all the rest of the string” (Tom) The previous coding treated only -1 that way, and would produce an invalid result value for other negative values, possibly leading to a crash (CVE-2010-0442).
•
Fix integer-to-bit-string conversions to handle the first fractional byte correctly when the output bit width is wider than the given integer by something other than a multiple of 8 bits (Tom)
•
Fix some cases of pathologically slow regular expression matching (Tom)
•
Fix the STOP WAL LOCATION entry in backup history files to report the next WAL segment’s name when the end location is exactly at a segment boundary (Itagaki Takahiro)
•
When reading pg_hba.conf and related files, do not treat @something as a file inclusion request if the @ appears inside quote marks; also, never treat @ by itself as a file inclusion request (Tom) This prevents erratic behavior if a role or database name starts with @. If you need to include a file whose path name contains spaces, you can still do so, but you must write @"/path to/file" rather than putting the quotes around the whole construct.
•
Prevent infinite loop on some platforms if a directory is named as an inclusion target in pg_hba.conf and related files (Tom)
•
Fix plpgsql failure in one case where a composite column is set to NULL (Tom)
•
Add volatile markings in PL/Python to avoid possible compiler-specific misbehavior (Zdenek Kotala)
•
Ensure PL/Tcl initializes the Tcl interpreter fully (Tom)
2501
Appendix E. Release Notes The only known symptom of this oversight is that the Tcl clock command misbehaves if using Tcl 8.5 or later. •
Prevent crash in contrib/dblink when too many key columns are specified to a dblink_build_sql_* function (Rushabh Lathia, Joe Conway)
•
Fix assorted crashes in contrib/xml2 caused by sloppy memory management (Tom)
•
Update time zone data files to tzdata release 2010e for DST law changes in Bangladesh, Chile, Fiji, Mexico, Paraguay, Samoa.
E.192. Release 8.0.23 Release date: 2009-12-14 This release contains a variety of fixes from 8.0.22. For information about new features in the 8.0 major release, see Section E.215.
E.192.1. Migration to Version 8.0.23 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.22, see Section E.193.
E.192.2. Changes •
Protect against indirect security threats caused by index functions changing session-local state (Gurjeet Singh, Tom) This change prevents allegedly-immutable index functions from possibly subverting a superuser’s session (CVE-2009-4136).
•
Reject SSL certificates containing an embedded null byte in the common name (CN) field (Magnus) This prevents unintended matching of a certificate to a server or client name during SSL validation (CVE-2009-4034).
•
Fix possible crash during backend-startup-time cache initialization (Tom)
•
Prevent signals from interrupting VACUUM at unsafe times (Alvaro) This fix prevents a PANIC if a VACUUM FULL is canceled after it’s already committed its tuple movements, as well as transient errors if a plain VACUUM is interrupted after having truncated the table.
•
Fix possible crash due to integer overflow in hash table size calculation (Tom) This could occur with extremely large planner estimates for the size of a hashjoin’s result.
•
Fix very rare crash in inet/cidr comparisons (Chris Mikkelson)
•
Fix premature drop of temporary files used for a cursor that is accessed within a subtransaction (Heikki)
•
Fix PAM password processing to be more robust (Tom)
2502
Appendix E. Release Notes The previous code is known to fail with the combination of the Linux pam_krb5 PAM module with Microsoft Active Directory as the domain controller. It might have problems elsewhere too, since it was making unjustified assumptions about what arguments the PAM stack would pass to it. •
Fix rare crash in exception processing in PL/Python (Peter)
•
Ensure psql’s flex module is compiled with the correct system header definitions (Tom) This fixes build failures on platforms where --enable-largefile causes incompatible changes in the generated code.
•
Make the postmaster ignore any application_name parameter in connection request packets, to improve compatibility with future libpq versions (Tom)
•
Update time zone data files to tzdata release 2009s for DST law changes in Antarctica, Argentina, Bangladesh, Fiji, Novokuznetsk, Pakistan, Palestine, Samoa, Syria; also historical corrections for Hong Kong.
E.193. Release 8.0.22 Release date: 2009-09-09 This release contains a variety of fixes from 8.0.21. For information about new features in the 8.0 major release, see Section E.215.
E.193.1. Migration to Version 8.0.22 A dump/restore is not required for those running 8.0.X. However, if you have any hash indexes on interval columns, you must REINDEX them after updating to 8.0.22. Also, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.193.2. Changes •
Disallow RESET ROLE and RESET SESSION AUTHORIZATION inside security-definer functions (Tom, Heikki) This covers a case that was missed in the previous patch that disallowed SET ROLE and SET SESSION AUTHORIZATION inside security-definer functions. (See CVE-2007-6600)
•
Fix handling of sub-SELECTs appearing in the arguments of an outer-level aggregate function (Tom)
•
Fix hash calculation for data type interval (Tom) This corrects wrong results for hash joins on interval values. It also changes the contents of hash indexes on interval columns. If you have any such indexes, you must REINDEX them after updating.
•
Treat to_char(..., ’TH’) as an uppercase ordinal suffix with ’HH’/’HH12’ (Heikki) It was previously handled as ’th’ (lowercase).
•
Fix overflow for INTERVAL ’x ms’ when x is more than 2 million and integer datetimes are in use (Alex Hunsaker)
•
Fix calculation of distance between a point and a line segment (Tom)
2503
Appendix E. Release Notes This led to incorrect results from a number of geometric operators. •
Fix money data type to work in locales where currency amounts have no fractional digits, e.g. Japan (Itagaki Takahiro)
•
Properly round datetime input like 00:12:57.9999999999999999999999999999 (Tom)
•
Fix poor choice of page split point in GiST R-tree operator classes (Teodor)
•
Fix portability issues in plperl initialization (Andrew Dunstan)
•
Fix pg_ctl to not go into an infinite loop if postgresql.conf is empty (Jeff Davis)
•
Fix contrib/xml2’s xslt_process() to properly handle the maximum number of parameters (twenty) (Tom)
•
Improve robustness of libpq’s code to recover from errors during COPY FROM STDIN (Tom)
•
Avoid including conflicting readline and editline header files when both libraries are installed (Zdenek Kotala)
•
Update time zone data files to tzdata release 2009l for DST law changes in Bangladesh, Egypt, Jordan, Pakistan, Argentina/San_Luis, Cuba, Jordan (historical correction only), Mauritius, Morocco, Palestine, Syria, Tunisia.
E.194. Release 8.0.21 Release date: 2009-03-16 This release contains a variety of fixes from 8.0.20. For information about new features in the 8.0 major release, see Section E.215.
E.194.1. Migration to Version 8.0.21 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.194.2. Changes •
Prevent error recursion crashes when encoding conversion fails (Tom) This change extends fixes made in the last two minor releases for related failure scenarios. The previous fixes were narrowly tailored for the original problem reports, but we have now recognized that any error thrown by an encoding conversion function could potentially lead to infinite recursion while trying to report the error. The solution therefore is to disable translation and encoding conversion and report the plain-ASCII form of any error message, if we find we have gotten into a recursive error reporting situation. (CVE-2009-0922)
•
Disallow CREATE CONVERSION with the wrong encodings for the specified conversion function (Heikki) This prevents one possible scenario for encoding conversion failure. The previous change is a backstop to guard against other kinds of failures in the same area.
2504
Appendix E. Release Notes •
Fix core dump when to_char() is given format codes that are inappropriate for the type of the data argument (Tom)
•
Add MUST (Mauritius Island Summer Time) to the default list of known timezone abbreviations (Xavier Bugaud)
E.195. Release 8.0.20 Release date: 2009-02-02 This release contains a variety of fixes from 8.0.19. For information about new features in the 8.0 major release, see Section E.215.
E.195.1. Migration to Version 8.0.20 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.195.2. Changes •
Improve handling of URLs in headline() function (Teodor)
•
Improve handling of overlength headlines in headline() function (Teodor)
•
Prevent possible Assert failure or misconversion if an encoding conversion is created with the wrong conversion function for the specified pair of encodings (Tom, Heikki)
•
Avoid unnecessary locking of small tables in VACUUM (Heikki)
•
Fix uninitialized variables in contrib/tsearch2’s get_covers() function (Teodor)
•
Make all documentation reference pgsql-bugs and/or pgsql-hackers as appropriate, instead of the now-decommissioned pgsql-ports and pgsql-patches mailing lists (Tom)
•
Update time zone data files to tzdata release 2009a (for Kathmandu and historical DST corrections in Switzerland, Cuba)
E.196. Release 8.0.19 Release date: 2008-11-03 This release contains a variety of fixes from 8.0.18. For information about new features in the 8.0 major release, see Section E.215.
E.196.1. Migration to Version 8.0.19 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
2505
Appendix E. Release Notes
E.196.2. Changes •
Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the “character has no equivalent” message itself couldn’t be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation.
•
Fix possible crash when deeply nested functions are invoked from a trigger (Tom)
•
Ensure an error is reported when a newly-defined PL/pgSQL trigger function is invoked as a normal function (Tom)
•
Fix incorrect tsearch2 headline generation when single query item matches first word of text (Sushant Sinha)
•
Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an --enable-integer-datetimes build (Ron Mayer)
•
Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn’t handle it properly. The only likely consequence is an incorrect error indication.
•
Fix ecpg’s parsing of CREATE USER (Michael)
•
Fix recent breakage of pg_ctl restart (Tom)
•
Update time zone data files to tzdata release 2008i (for DST law changes in Argentina, Brazil, Mauritius, Syria)
E.197. Release 8.0.18 Release date: 2008-09-22 This release contains a variety of fixes from 8.0.17. For information about new features in the 8.0 major release, see Section E.215.
E.197.1. Migration to Version 8.0.18 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.197.2. Changes •
Widen local lock counters from 32 to 64 bits (Tom) This responds to reports that the counters could overflow in sufficiently long transactions, leading to unexpected “lock is already held” errors.
•
Add checks in executor startup to ensure that the tuples produced by an INSERT or UPDATE will match the target table’s current rowtype (Tom)
2506
Appendix E. Release Notes ALTER COLUMN TYPE, followed by re-use of a previously cached plan, could produce this type of
situation. The check protects against data corruption and/or crashes that could ensue. •
Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom)
•
Improve performance of writing very long log messages to syslog (Tom)
•
Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom)
•
Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions’ contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like col IS NULL.
•
Fix PL/Tcl to behave correctly with Tcl 8.5, and to be more careful about the encoding of data sent to or from Tcl (Tom)
•
Fix PL/Python to work with Python 2.5 This is a back-port of fixes made during the 8.2 development cycle.
•
Improve pg_dump and pg_restore’s error reporting after failure to send a SQL command (Tom)
•
Fix pg_ctl to properly preserve postmaster command-line arguments across a restart (Bruce)
•
Update time zone data files to tzdata release 2008f (for DST law changes in Argentina, Bahamas, Brazil, Mauritius, Morocco, Pakistan, Palestine, and Paraguay)
E.198. Release 8.0.17 Release date: 2008-06-12 This release contains one serious bug fix over 8.0.16. For information about new features in the 8.0 major release, see Section E.215.
E.198.1. Migration to Version 8.0.17 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.198.2. Changes •
Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid SELECT DISTINCT view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7.
2507
Appendix E. Release Notes
E.199. Release 8.0.16 Release date: never released This release contains a variety of fixes from 8.0.15. For information about new features in the 8.0 major release, see Section E.215.
E.199.1. Migration to Version 8.0.16 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.199.2. Changes •
Fix ALTER TABLE ADD COLUMN ... PRIMARY KEY so that the new column is correctly checked to see if it’s been initialized to all non-nulls (Brendan Jurd) Previous versions neglected to check this requirement at all.
•
Fix possible CREATE TABLE failure when inheriting the “same” constraint from multiple parent relations that inherited that constraint from a common ancestor (Tom)
•
Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic “Yo” characters (e and E with two dots) (Sergey Burladyan)
•
Fix a few datatype input functions that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions.
•
Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn’t got a match. An example is substring(’foo’ from ’foo(bar)?’). This should return NULL, since (bar) isn’t matched, but it was mistakenly returning the whole-pattern match instead (ie, foo).
•
Update time zone data files to tzdata release 2008c (for DST law changes in Morocco, Iraq, Choibalsan, Pakistan, Syria, Cuba, Argentina/San_Luis, and Chile)
•
Fix incorrect result from ecpg’s PGTYPEStimestamp_sub() function (Michael)
•
Fix core dump in contrib/xml2’s xpath_table() function when the input query returns a NULL value (Tom)
•
Fix contrib/xml2’s makefile to not override CFLAGS (Tom)
•
Fix DatumGetBool macro to not fail with gcc 4.3 (Tom) This problem affects “old style” (V0) C functions that return boolean. The fix is already in 8.3, but the need to back-patch it was not realized at the time.
•
Fix longstanding LISTEN/NOTIFY race condition (Tom) In rare cases a session that had just executed a LISTEN might not get a notification, even though one would be expected because the concurrent transaction executing NOTIFY was observed to commit later.
2508
Appendix E. Release Notes A side effect of the fix is that a transaction that has executed a not-yet-committed LISTEN command will not see any row in pg_listener for the LISTEN, should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior. •
Fix rare crash when an error occurs during a query using a hash index (Heikki)
•
Fix input of datetime values for February 29 in years BC (Tom) The former coding was mistaken about which years were leap years.
•
Fix “unrecognized node type” error in some variants of ALTER OWNER (Tom)
•
Fix pg_ctl to correctly extract the postmaster’s port number from command-line options (Itagaki Takahiro, Tom) Previously, pg_ctl start -w could try to contact the postmaster on the wrong port, leading to bogus reports of startup failure.
•
Use -fwrapv to defend against possible misoptimization in recent gcc versions (Tom) This is known to be necessary when building PostgreSQL with gcc 4.3 or later.
•
Fix display of constant expressions in ORDER BY and GROUP BY (Tom) An explicitly casted constant would be shown incorrectly. This could for example lead to corruption of a view definition during dump and reload.
•
Fix libpq to handle NOTICE messages correctly during COPY OUT (Tom) This failure has only been observed to occur when a user-defined datatype’s output routine issues a NOTICE, but there is no guarantee it couldn’t happen due to other causes.
E.200. Release 8.0.15 Release date: 2008-01-07 This release contains a variety of fixes from 8.0.14, including fixes for significant security issues. For information about new features in the 8.0 major release, see Section E.215. This is the last 8.0.X release for which the PostgreSQL community will produce binary packages for Windows. Windows users are encouraged to move to 8.2.X or later, since there are Windows-specific fixes in 8.2.X that are impractical to back-port. 8.0.X will continue to be supported on other platforms.
E.200.1. Migration to Version 8.0.15 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.200.2. Changes •
Prevent functions in indexes from executing with the privileges of the user running VACUUM, ANALYZE, etc (Tom) Functions used in index expressions and partial-index predicates are evaluated whenever a new table entry is made. It has long been understood that this poses a risk of trojan-horse code execution
2509
Appendix E. Release Notes if one modifies a table owned by an untrustworthy user. (Note that triggers, defaults, check constraints, etc. pose the same type of risk.) But functions in indexes pose extra danger because they will be executed by routine maintenance operations such as VACUUM FULL, which are commonly performed automatically under a superuser account. For example, a nefarious user can execute code with superuser privileges by setting up a trojan-horse index definition and waiting for the next routine vacuum. The fix arranges for standard maintenance operations (including VACUUM, ANALYZE, REINDEX, and CLUSTER) to execute as the table owner rather than the calling user, using the same privilege-switching mechanism already used for SECURITY DEFINER functions. To prevent bypassing this security measure, execution of SET SESSION AUTHORIZATION and SET ROLE is now forbidden within a SECURITY DEFINER context. (CVE-2007-6600) •
Repair assorted bugs in the regular-expression package (Tom, Will Drewry) Suitably crafted regular-expression patterns could cause crashes, infinite or near-infinite looping, and/or massive memory consumption, all of which pose denial-of-service hazards for applications that accept regex search patterns from untrustworthy sources. (CVE-2007-4769, CVE-2007-4772, CVE-2007-6067)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe) The fix that appeared for this in 8.0.14 was incomplete, as it plugged the hole for only some dblink functions. (CVE-2007-6601, CVE-2007-3278)
•
Update time zone data files to tzdata release 2007k (in particular, recent Argentina changes) (Tom)
•
Fix planner failure in some cases of WHERE false AND var IN (SELECT ...) (Tom)
•
Preserve the tablespace of indexes that are rebuilt by ALTER TABLE ... ALTER COLUMN TYPE (Tom)
•
Make archive recovery always start a new WAL timeline, rather than only when a recovery stop time was used (Simon) This avoids a corner-case risk of trying to overwrite an existing archived copy of the last WAL segment, and seems simpler and cleaner than the original definition.
•
Make VACUUM not use all of maintenance_work_mem when the table is too small for it to be useful (Alvaro)
•
Fix potential crash in translate() when using a multibyte database encoding (Tom)
•
Fix PL/Perl to cope when platform’s Perl defines type bool as int rather than char (Tom) While this could theoretically happen anywhere, no standard build of Perl did things this way ... until Mac OS X 10.5.
•
Fix PL/Python to not crash on long exception messages (Alvaro)
•
Fix pg_dump to correctly handle inheritance child tables that have default expressions different from their parent’s (Tom)
•
ecpg parser fixes (Michael)
•
Make contrib/tablefunc’s crosstab() handle NULL rowid as a category in its own right, rather than crashing (Joe)
•
Fix tsvector and tsquery output routines to escape backslashes correctly (Teodor, Bruce)
•
Fix crash of to_tsvector() on huge input strings (Teodor)
•
Require a specific version of Autoconf to be used when re-generating the configure script (Peter)
2510
Appendix E. Release Notes This affects developers and packagers only. The change was made to prevent accidental use of untested combinations of Autoconf and PostgreSQL versions. You can remove the version check if you really want to use a different Autoconf version, but it’s your responsibility whether the result works or not.
E.201. Release 8.0.14 Release date: 2007-09-17 This release contains a variety of fixes from 8.0.13. For information about new features in the 8.0 major release, see Section E.215.
E.201.1. Migration to Version 8.0.14 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.201.2. Changes •
Prevent index corruption when a transaction inserts rows and then aborts close to the end of a concurrent VACUUM on the same table (Tom)
•
Make CREATE DOMAIN ... DEFAULT NULL work properly (Tom)
•
Fix excessive logging of SSL error messages (Tom)
•
Fix logging so that log messages are never interleaved when using the syslogger process (Andrew)
•
Fix crash when log_min_error_statement logging runs out of memory (Tom)
•
Fix incorrect handling of some foreign-key corner cases (Tom)
•
Prevent CLUSTER from failing due to attempting to process temporary tables of other sessions (Alvaro)
•
Update the time zone database rules, particularly New Zealand’s upcoming changes (Tom)
•
Windows socket improvements (Magnus)
•
Suppress timezone name (%Z) in log timestamps on Windows because of possible encoding mismatches (Tom)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe)
E.202. Release 8.0.13 Release date: 2007-04-23 This release contains a variety of fixes from 8.0.12, including a security fix. For information about new features in the 8.0 major release, see Section E.215.
2511
Appendix E. Release Notes
E.202.1. Migration to Version 8.0.13 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.202.2. Changes •
Support explicit placement of the temporary-table schema within search_path, and disable searching it for functions and operators (Tom) This is needed to allow a security-definer function to set a truly secure value of search_path. Without it, an unprivileged SQL user can use temporary objects to execute code with the privileges of the security-definer function (CVE-2007-2138). See CREATE FUNCTION for more information.
• /contrib/tsearch2
crash fixes (Teodor)
•
Fix potential-data-corruption bug in how VACUUM FULL handles UPDATE chains (Tom, Pavan Deolasee)
•
Fix PANIC during enlargement of a hash index (bug introduced in 8.0.10) (Tom)
•
Fix POSIX-style timezone specs to follow new USA DST rules (Tom)
E.203. Release 8.0.12 Release date: 2007-02-07 This release contains one fix from 8.0.11. For information about new features in the 8.0 major release, see Section E.215.
E.203.1. Migration to Version 8.0.12 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.203.2. Changes •
Remove overly-restrictive check for type length in constraints and functional indexes(Tom)
E.204. Release 8.0.11 Release date: 2007-02-05 This release contains a variety of fixes from 8.0.10, including a security fix. For information about new features in the 8.0 major release, see Section E.215.
2512
Appendix E. Release Notes
E.204.1. Migration to Version 8.0.11 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.204.2. Changes •
Remove security vulnerabilities that allowed connected users to read backend memory (Tom) The vulnerabilities involve suppressing the normal check that a SQL function returns the data type it’s declared to, and changing the data type of a table column (CVE-2007-0555, CVE-2007-0556). These errors can easily be exploited to cause a backend crash, and in principle might be used to read database content that the user should not be able to access.
•
Fix rare bug wherein btree index page splits could fail due to choosing an infeasible split point (Heikki Linnakangas)
•
Fix for rare Assert() crash triggered by UNION (Tom)
•
Tighten security of multi-byte character processing for UTF8 sequences over three bytes long (Tom)
E.205. Release 8.0.10 Release date: 2007-01-08 This release contains a variety of fixes from 8.0.9. For information about new features in the 8.0 major release, see Section E.215.
E.205.1. Migration to Version 8.0.10 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.205.2. Changes •
Improve handling of getaddrinfo() on AIX (Tom) This fixes a problem with starting the statistics collector, among other things.
•
Fix “failed to re-find parent key” errors in VACUUM (Tom)
•
Fix race condition for truncation of a large relation across a gigabyte boundary by VACUUM (Tom)
Fix possible deadlock in Windows signal handling (Teodor)
•
Fix error when constructing an ARRAY[] made up of multiple empty elements (Tom)
•
Fix ecpg memory leak during connection (Michael)
2513
Appendix E. Release Notes • to_number() and to_char(numeric) are now STABLE, not IMMUTABLE, for new initdb installs
(Tom) This is because lc_numeric can potentially change the output of these functions. •
Improve index usage of regular expressions that use parentheses (Tom) This improves psql \d performance also.
•
Update timezone database This affects Australian and Canadian daylight-savings rules in particular.
E.206. Release 8.0.9 Release date: 2006-10-16 This release contains a variety of fixes from 8.0.8. For information about new features in the 8.0 major release, see Section E.215.
E.206.1. Migration to Version 8.0.9 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.206.2. Changes •
Fix crash when referencing NEW row values in rule WHERE expressions (Tom)
•
Fix core dump when an untyped literal is taken as ANYARRAY
•
Fix mishandling of AFTER triggers when query contains a SQL function returning multiple rows (Tom)
•
Fix ALTER TABLE ... TYPE to recheck NOT NULL for USING clause (Tom)
•
Fix string_to_array() to handle overlapping matches for the separator string For example, string_to_array(’123xx456xxx789’, ’xx’).
•
Fix corner cases in pattern matching for psql’s \d commands
•
Fix index-corrupting bugs in /contrib/ltree (Teodor)
•
Numerous robustness fixes in ecpg (Joachim Wieland)
•
Fix backslash escaping in /contrib/dbmirror
•
Fix instability of statistics collection on Win32 (Tom, Andrew)
•
Fixes for AIX and Intel compilers (Tom)
E.207. Release 8.0.8 Release date: 2006-05-23
2514
Appendix E. Release Notes This release contains a variety of fixes from 8.0.7, including patches for extremely serious security issues. For information about new features in the 8.0 major release, see Section E.215.
E.207.1. Migration to Version 8.0.8 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209. Full security against the SQL-injection attacks described in CVE-2006-2313 and CVE-2006-2314 might require changes in application code. If you have applications that embed untrustworthy strings into SQL commands, you should examine them as soon as possible to ensure that they are using recommended escaping techniques. In most cases, applications should be using subroutines provided by libraries or drivers (such as libpq’s PQescapeStringConn()) to perform string escaping, rather than relying on ad hoc code to do it.
E.207.2. Changes •
Change the server to reject invalidly-encoded multibyte characters in all cases (Tatsuo, Tom) While PostgreSQL has been moving in this direction for some time, the checks are now applied uniformly to all encodings and all textual input, and are now always errors not merely warnings. This change defends against SQL-injection attacks of the type described in CVE-2006-2313.
•
Reject unsafe uses of \’ in string literals As a server-side defense against SQL-injection attacks of the type described in CVE-2006-2314, the server now only accepts ” and not \’ as a representation of ASCII single quote in SQL string literals. By default, \’ is rejected only when client_encoding is set to a client-only encoding (SJIS, BIG5, GBK, GB18030, or UHC), which is the scenario in which SQL injection is possible. A new configuration parameter backslash_quote is available to adjust this behavior when needed. Note that full security against CVE-2006-2314 might require client-side changes; the purpose of backslash_quote is in part to make it obvious that insecure clients are insecure.
•
Modify libpq’s string-escaping routines to be aware of encoding considerations and standard_conforming_strings
This fixes libpq-using applications for the security issues described in CVE-2006-2313 and CVE2006-2314, and also future-proofs them against the planned changeover to SQL-standard string literal syntax. Applications that use multiple PostgreSQL connections concurrently should migrate to PQescapeStringConn() and PQescapeByteaConn() to ensure that escaping is done correctly for the settings in use in each database connection. Applications that do string escaping “by hand” should be modified to rely on library routines instead. •
Fix some incorrect encoding conversion functions win1251_to_iso, alt_to_iso, euc_tw_to_big5, euc_tw_to_mic, mic_to_euc_tw were all broken to varying extents.
•
Clean up stray remaining uses of \’ in strings (Bruce, Jan)
•
Fix bug that sometimes caused OR’d index scans to miss rows they should have returned
•
Fix WAL replay for case where a btree index has been truncated
•
Fix SIMILAR TO for patterns involving | (Tom)
2515
Appendix E. Release Notes •
Fix SELECT INTO and CREATE TABLE AS to create tables in the default tablespace, not the base directory (Kris Jurka)
•
Fix server to use custom DH SSL parameters correctly (Michael Fuhr)
•
Fix for Bonjour on Intel Macs (Ashley Clark)
•
Fix various minor memory leaks
•
Fix problem with password prompting on some Win32 systems (Robert Kinberg)
E.208. Release 8.0.7 Release date: 2006-02-14 This release contains a variety of fixes from 8.0.6. For information about new features in the 8.0 major release, see Section E.215.
E.208.1. Migration to Version 8.0.7 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.6, see Section E.209.
E.208.2. Changes •
Fix potential crash in SET SESSION AUTHORIZATION (CVE-2006-0553) An unprivileged user could crash the server process, resulting in momentary denial of service to other users, if the server has been compiled with Asserts enabled (which is not the default). Thanks to Akio Ishida for reporting this problem.
•
Fix bug with row visibility logic in self-inserted rows (Tom) Under rare circumstances a row inserted by the current command could be seen as already valid, when it should not be. Repairs bug created in 8.0.4, 7.4.9, and 7.3.11 releases.
•
Fix race condition that could lead to “file already exists” errors during pg_clog and pg_subtrans file creation (Tom)
•
Fix cases that could lead to crashes if a cache-invalidation message arrives at just the wrong time (Tom)
•
Properly check DOMAIN constraints for UNKNOWN parameters in prepared statements (Neil)
•
Ensure ALTER COLUMN TYPE will process FOREIGN KEY, UNIQUE, and PRIMARY KEY constraints in the proper order (Nakano Yoshihisa)
•
Fixes to allow restoring dumps that have cross-schema references to custom operators or operator classes (Tom)
•
Allow pg_restore to continue properly after a COPY failure; formerly it tried to treat the remaining COPY data as SQL commands (Stephen Frost)
•
Fix pg_ctl unregister crash when the data directory is not specified (Magnus)
•
Fix ecpg crash on AMD64 and PPC (Neil)
2516
Appendix E. Release Notes •
Recover properly if error occurs during argument passing in PL/python (Neil)
•
Fix PL/perl’s handling of locales on Win32 to match the backend (Andrew)
•
Fix crash when log_min_messages is set to DEBUG3 or above in postgresql.conf on Win32 (Bruce)
•
Fix pgxs -L library path specification for Win32, Cygwin, OS X, AIX (Bruce)
•
Check that SID is enabled while checking for Win32 admin privileges (Magnus)
•
Properly reject out-of-range date inputs (Kris Jurka)
•
Portability fix for testing presence of finite and isinf during configure (Tom)
E.209. Release 8.0.6 Release date: 2006-01-09 This release contains a variety of fixes from 8.0.5. For information about new features in the 8.0 major release, see Section E.215.
E.209.1. Migration to Version 8.0.6 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.3, see Section E.212. Also, you might need to REINDEX indexes on textual columns after updating, if you are affected by the locale or plperl issues described below.
E.209.2. Changes •
Fix Windows code so that postmaster will continue rather than exit if there is no more room in ShmemBackendArray (Magnus) The previous behavior could lead to a denial-of-service situation if too many connection requests arrive close together. This applies only to the Windows port.
•
Fix bug introduced in 8.0 that could allow ReadBuffer to return an already-used page as new, potentially causing loss of recently-committed data (Tom)
•
Fix for protocol-level Describe messages issued outside a transaction or in a failed transaction (Tom)
•
Fix character string comparison for locales that consider different character combinations as equal, such as Hungarian (Tom) This might require REINDEX to fix existing indexes on textual columns.
•
Set locale environment variables during postmaster startup to ensure that plperl won’t change the locale later This fixes a problem that occurred if the postmaster was started with environment variables specifying a different locale than what initdb had been told. Under these conditions, any use of plperl was likely to lead to corrupt indexes. You might need REINDEX to fix existing indexes on textual columns if this has happened to you.
•
Allow more flexible relocation of installation directories (Tom)
2517
Appendix E. Release Notes Previous releases supported relocation only if all installation directory paths were the same except for the last component. •
Fix longstanding bug in strpos() and regular expression handling in certain rarely used Asian multibyte character sets (Tatsuo)
•
Various fixes for functions returning RECORDs (Tom)
•
Fix bug in /contrib/pgcrypto gen_salt, which caused it not to use all available salt space for MD5 and XDES algorithms (Marko Kreen, Solar Designer) Salts for Blowfish and standard DES are unaffected.
•
Fix /contrib/dblink to throw an error, rather than crashing, when the number of columns specified is different from what’s actually returned by the query (Joe)
E.210. Release 8.0.5 Release date: 2005-12-12 This release contains a variety of fixes from 8.0.4. For information about new features in the 8.0 major release, see Section E.215.
E.210.1. Migration to Version 8.0.5 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.3, see Section E.212.
E.210.2. Changes •
Fix race condition in transaction log management There was a narrow window in which an I/O operation could be initiated for the wrong page, leading to an Assert failure or data corruption.
•
Fix bgwriter problems after recovering from errors (Tom) The background writer was found to leak buffer pins after write errors. While not fatal in itself, this might lead to mysterious blockages of later VACUUM commands.
•
Prevent failure if client sends Bind protocol message when current transaction is already aborted
• /contrib/ltree
fixes (Teodor)
•
AIX and HPUX compile fixes (Tom)
•
Retry file reads and writes after Windows NO_SYSTEM_RESOURCES error (Qingqing Zhou)
•
Fix intermittent failure when log_line_prefix includes %i
•
Fix psql performance issue with long scripts on Windows (Merlin Moncure)
•
Fix missing updates of pg_group flat file
•
Fix longstanding planning error for outer joins This bug sometimes caused a bogus error “RIGHT JOIN is only supported with merge-joinable join conditions”.
2518
Appendix E. Release Notes •
Postpone timezone initialization until after postmaster.pid is created This avoids confusing startup scripts that expect the pid file to appear quickly.
•
Prevent core dump in pg_autovacuum when a table has been dropped
•
Fix problems with whole-row references (foo.*) to subquery results
E.211. Release 8.0.4 Release date: 2005-10-04 This release contains a variety of fixes from 8.0.3. For information about new features in the 8.0 major release, see Section E.215.
E.211.1. Migration to Version 8.0.4 A dump/restore is not required for those running 8.0.X. However, if you are upgrading from a version earlier than 8.0.3, see Section E.212.
E.211.2. Changes •
Fix error that allowed VACUUM to remove ctid chains too soon, and add more checking in code that follows ctid links This fixes a long-standing problem that could cause crashes in very rare circumstances.
•
Fix CHAR() to properly pad spaces to the specified length when using a multiple-byte character set (Yoshiyuki Asaba) In prior releases, the padding of CHAR() was incorrect because it only padded to the specified number of bytes without considering how many characters were stored.
•
Force a checkpoint before committing CREATE DATABASE This should fix recent reports of “index is not a btree” failures when a crash occurs shortly after CREATE DATABASE.
•
Fix the sense of the test for read-only transaction in COPY The code formerly prohibited COPY TO, where it should prohibit COPY FROM.
•
Handle consecutive embedded newlines in COPY CSV-mode input
•
Fix date_trunc(week) for dates near year end
•
Fix planning problem with outer-join ON clauses that reference only the inner-side relation
•
Further fixes for x FULL JOIN y ON true corner cases
•
Fix overenthusiastic optimization of x IN (SELECT DISTINCT ...) and related cases
•
Fix mis-planning of queries with small LIMIT values due to poorly thought out “fuzzy” cost comparison
•
Make array_in and array_recv more paranoid about validating their OID parameter
•
Fix missing rows in queries like UPDATE a=... WHERE a... with GiST index on column a
2519
Appendix E. Release Notes •
Improve robustness of datetime parsing
•
Improve checking for partially-written WAL pages
•
Improve robustness of signal handling when SSL is enabled
•
Improve MIPS and M68K spinlock code
•
Don’t try to open more than max_files_per_process files during postmaster startup
•
Various memory leakage fixes
•
Various portability improvements
•
Update timezone data files
•
Improve handling of DLL load failures on Windows
•
Improve random-number generation on Windows
•
Make psql -f filename return a nonzero exit code when opening the file fails
•
Change pg_dump to handle inherited check constraints more reliably
•
Fix password prompting in pg_restore on Windows
•
Fix PL/pgSQL to handle var := var correctly when the variable is of pass-by-reference type
•
Fix PL/Perl %_SHARED so it’s actually shared
•
Fix contrib/pg_autovacuum to allow sleep intervals over 2000 sec
•
Update contrib/tsearch2 to use current Snowball code
E.212. Release 8.0.3 Release date: 2005-05-09 This release contains a variety of fixes from 8.0.2, including several security-related issues. For information about new features in the 8.0 major release, see Section E.215.
E.212.1. Migration to Version 8.0.3 A dump/restore is not required for those running 8.0.X. However, it is one possible way of handling two significant security problems that have been found in the initial contents of 8.0.X system catalogs. A dump/initdb/reload sequence using 8.0.3’s initdb will automatically correct these problems. The larger security problem is that the built-in character set encoding conversion functions can be invoked from SQL commands by unprivileged users, but the functions were not designed for such use and are not secure against malicious choices of arguments. The fix involves changing the declared parameter list of these functions so that they can no longer be invoked from SQL commands. (This does not affect their normal use by the encoding conversion machinery.) The lesser problem is that the contrib/tsearch2 module creates several functions that are improperly declared to return internal when they do not accept internal arguments. This breaks type safety for all functions using internal arguments. It is strongly recommended that all installations repair these errors, either by initdb or by following the manual repair procedure given below. The errors at least allow unprivileged database users to crash their server process, and might allow unprivileged users to gain the privileges of a database superuser.
2520
Appendix E. Release Notes If you wish not to do an initdb, perform the same manual repair procedures shown in the 7.4.8 release notes.
E.212.2. Changes •
Change encoding function signature to prevent misuse
•
Change contrib/tsearch2 to avoid unsafe use of INTERNAL function results
•
Guard against incorrect second parameter to record_out
•
Repair ancient race condition that allowed a transaction to be seen as committed for some purposes (eg SELECT FOR UPDATE) slightly sooner than for other purposes This is an extremely serious bug since it could lead to apparent data inconsistencies being briefly visible to applications.
•
Repair race condition between relation extension and VACUUM This could theoretically have caused loss of a page’s worth of freshly-inserted data, although the scenario seems of very low probability. There are no known cases of it having caused more than an Assert failure.
•
Fix comparisons of TIME WITH TIME ZONE values The comparison code was wrong in the case where the --enable-integer-datetimes configuration switch had been used. NOTE: if you have an index on a TIME WITH TIME ZONE column, it will need to be REINDEXed after installing this update, because the fix corrects the sort order of column values.
•
Fix EXTRACT(EPOCH) for TIME WITH TIME ZONE values
•
Fix mis-display of negative fractional seconds in INTERVAL values This error only occurred when the --enable-integer-datetimes configuration switch had been used.
•
Fix pg_dump to dump trigger names containing % correctly (Neil)
•
Still more 64-bit fixes for contrib/intagg
•
Prevent incorrect optimization of functions returning RECORD
•
Prevent crash on COALESCE(NULL,NULL)
•
Fix Borland makefile for libpq
•
Fix contrib/btree_gist for timetz type (Teodor)
•
Make pg_ctl check the PID found in postmaster.pid to see if it is still a live process
•
Fix pg_dump/pg_restore problems caused by addition of dump timestamps
•
Fix interaction between materializing holdable cursors and firing deferred triggers during transaction commit
•
Fix memory leak in SQL functions returning pass-by-reference data types
2521
Appendix E. Release Notes
E.213. Release 8.0.2 Release date: 2005-04-07 This release contains a variety of fixes from 8.0.1. For information about new features in the 8.0 major release, see Section E.215.
E.213.1. Migration to Version 8.0.2 A dump/restore is not required for those running 8.0.*. This release updates the major version number of the PostgreSQL libraries, so it might be necessary to re-link some user applications if they cannot find the properly-numbered shared library.
E.213.2. Changes •
Increment the major version number of all interface libraries (Bruce) This should have been done in 8.0.0. It is required so 7.4.X versions of PostgreSQL client applications, like psql, can be used on the same machine as 8.0.X applications. This might require re-linking user applications that use these libraries.
•
Add Windows-only wal_sync_method setting of fsync_writethrough (Magnus, Bruce) This setting causes PostgreSQL to write through any disk-drive write cache when writing to WAL. This behavior was formerly called fsync, but was renamed because it acts quite differently from fsync on other platforms.
•
Enable the wal_sync_method setting of open_datasync on Windows, and make it the default for that platform (Magnus, Bruce) Because the default is no longer fsync_writethrough, data loss is possible during a power failure if the disk drive has write caching enabled. To turn off the write cache on Windows, from the Device Manager, choose the drive properties, then Policies.
•
New cache management algorithm 2Q replaces ARC (Tom) This was done to avoid a pending US patent on ARC. The 2Q code might be a few percentage points slower than ARC for some work loads. A better cache management algorithm will appear in 8.1.
•
Planner adjustments to improve behavior on freshly-created tables (Tom)
•
Allow plpgsql to assign to an element of an array that is initially NULL (Tom) Formerly the array would remain NULL, but now it becomes a single-element array. The main SQL engine was changed to handle UPDATE of a null array value this way in 8.0, but the similar case in plpgsql was overlooked.
•
Convert \r\n and \r to \n in plpython function bodies (Michael Fuhr) This prevents syntax errors when plpython code is written on a Windows or Mac client.
•
Allow SPI cursors to handle utility commands that return rows, such as EXPLAIN (Tom)
•
Fix CLUSTER failure after ALTER TABLE SET WITHOUT OIDS (Tom)
•
Reduce memory usage of ALTER TABLE ADD COLUMN (Neil)
•
Fix ALTER LANGUAGE RENAME (Tom)
2522
Appendix E. Release Notes •
Document the Windows-only register and unregister options of pg_ctl (Magnus)
•
Ensure operations done during backend shutdown are counted by statistics collector This is expected to resolve reports of pg_autovacuum not vacuuming the system catalogs often enough — it was not being told about catalog deletions caused by temporary table removal during backend exit.
•
Change the Windows default for configuration parameter log_destination to eventlog (Magnus) By default, a server running on Windows will now send log output to the Windows event logger rather than standard error.
•
Make Kerberos authentication work on Windows (Magnus)
•
Allow ALTER DATABASE RENAME by superusers who aren’t flagged as having CREATEDB privilege (Tom)
•
Modify WAL log entries for CREATE and DROP DATABASE to not specify absolute paths (Tom) This allows point-in-time recovery on a different machine with possibly different database location. Note that CREATE TABLESPACE still poses a hazard in such situations.
•
Fix crash from a backend exiting with an open transaction that created a table and opened a cursor on it (Tom)
•
Fix array_map() so it can call PL functions (Tom)
•
Several contrib/tsearch2 and contrib/btree_gist fixes (Teodor)
•
Fix crash of some contrib/pgcrypto functions on some platforms (Marko Kreen)
•
Fix contrib/intagg for 64-bit platforms (Tom)
•
Fix ecpg bugs in parsing of CREATE statement (Michael)
•
Work around gcc bug on powerpc and amd64 causing problems in ecpg (Christof Petig)
•
Do not use locale-aware versions of upper(), lower(), and initcap() when the locale is C (Bruce) This allows these functions to work on platforms that generate errors for non-7-bit data when the locale is C.
•
Fix quote_ident() to quote names that match keywords (Tom)
•
Fix to_date() to behave reasonably when CC and YY fields are both used (Karel)
•
Prevent to_char(interval) from failing when given a zero-month interval (Tom)
•
Fix wrong week returned by date_trunc(’week’) (Bruce) date_trunc(’week’) returned the wrong year for the first few days of January in some years.
•
Use the correct default mask length for class D addresses in INET data types (Tom)
E.214. Release 8.0.1 Release date: 2005-01-31 This release contains a variety of fixes from 8.0.0, including several security-related issues. For information about new features in the 8.0 major release, see Section E.215.
2523
Appendix E. Release Notes
E.214.1. Migration to Version 8.0.1 A dump/restore is not required for those running 8.0.0.
E.214.2. Changes •
Disallow LOAD to non-superusers On platforms that will automatically execute initialization functions of a shared library (this includes at least Windows and ELF-based Unixen), LOAD can be used to make the server execute arbitrary code. Thanks to NGS Software for reporting this.
•
Check that creator of an aggregate function has the right to execute the specified transition functions This oversight made it possible to bypass denial of EXECUTE permission on a function.
•
Fix security and 64-bit issues in contrib/intagg
•
Add needed STRICT marking to some contrib functions (Kris Jurka)
•
Avoid buffer overrun when plpgsql cursor declaration has too many parameters (Neil)
•
Make ALTER TABLE ADD COLUMN enforce domain constraints in all cases
•
Fix planning error for FULL and RIGHT outer joins The result of the join was mistakenly supposed to be sorted the same as the left input. This could not only deliver mis-sorted output to the user, but in case of nested merge joins could give outright wrong answers.
•
Improve planning of grouped aggregate queries
• ROLLBACK TO savepoint
closes cursors created since the savepoint
•
Fix inadequate backend stack size on Windows
•
Avoid SHGetSpecialFolderPath() on Windows (Magnus)
•
Fix some problems in running pg_autovacuum as a Windows service (Dave Page)
•
Multiple minor bug fixes in pg_dump/pg_restore
•
Fix ecpg segfault with named structs used in typedefs (Michael)
E.215. Release 8.0 Release date: 2005-01-19
E.215.1. Overview Major changes in this release: Microsoft Windows Native Server This is the first PostgreSQL release to run natively on Microsoft Windows® as a server. It can run as a Windows service. This release supports NT-based Windows releases like Windows 2000 SP4, Windows XP, and Windows 2003. Older releases like Windows 95, Windows 98, and Windows ME are not supported because these operating systems do not have the infrastructure to
2524
Appendix E. Release Notes support PostgreSQL. A separate installer project has been created to ease installation on Windows — see http://www.postgresql.org/ftp/win32/. Although tested throughout our release cycle, the Windows port does not have the benefit of years of use in production environments that PostgreSQL has on Unix platforms. Therefore it should be treated with the same level of caution as you would a new product. Previous releases required the Unix emulation toolkit Cygwin in order to run the server on Windows operating systems. PostgreSQL has supported native clients on Windows for many years. Savepoints Savepoints allow specific parts of a transaction to be aborted without affecting the remainder of the transaction. Prior releases had no such capability; there was no way to recover from a statement failure within a transaction except by aborting the whole transaction. This feature is valuable for application writers who require error recovery within a complex transaction. Point-In-Time Recovery In previous releases there was no way to recover from disk drive failure except to restore from a previous backup or use a standby replication server. Point-in-time recovery allows continuous backup of the server. You can recover either to the point of failure or to some transaction in the past. Tablespaces Tablespaces allow administrators to select different file systems for storage of individual tables, indexes, and databases. This improves performance and control over disk space usage. Prior releases used initlocation and manual symlink management for such tasks. Improved Buffer Management, CHECKPOINT, VACUUM This release has a more intelligent buffer replacement strategy, which will make better use of available shared buffers and improve performance. The performance impact of vacuum and checkpoints is also lessened. Change Column Types A column’s data type can now be changed with ALTER TABLE. New Perl Server-Side Language A new version of the plperl server-side language now supports a persistent shared storage area, triggers, returning records and arrays of records, and SPI calls to access the database. Comma-separated-value (CSV) support in COPY COPY can now read and write comma-separated-value files. It has the flexibility to interpret non-
standard quoting and separation characters too.
E.215.2. Migration to Version 8.0 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: •
In READ COMMITTED serialization mode, volatile functions now see the results of concurrent transactions committed up to the beginning of each statement within the function, rather than up to the beginning of the interactive command that called the function.
2525
Appendix E. Release Notes •
Functions declared STABLE or IMMUTABLE always use the snapshot of the calling query, and therefore do not see the effects of actions taken after the calling query starts, whether in their own transaction or other transactions. Such a function must be read-only, too, meaning that it cannot use any SQL commands other than SELECT.
•
Nondeferred AFTER triggers are now fired immediately after completion of the triggering query, rather than upon finishing the current interactive command. This makes a difference when the triggering query occurred within a function: the trigger is invoked before the function proceeds to its next operation.
•
Server configuration parameters virtual_host and tcpip_socket have been replaced with a more general parameter listen_addresses. Also, the server now listens on localhost by default, which eliminates the need for the -i postmaster switch in many scenarios.
•
Server configuration parameters SortMem and VacuumMem have been renamed to work_mem and maintenance_work_mem to better reflect their use. The original names are still supported in SET and SHOW.
•
Server configuration parameters log_pid, log_timestamp, and log_source_port have been replaced with a more general parameter log_line_prefix.
•
Server configuration parameter syslog has been replaced with a more logical log_destination variable to control the log output destination.
•
Server configuration parameter log_statement has been changed so it can selectively log just database modification or data definition statements. Server configuration parameter log_duration now prints only when log_statement prints the query.
•
Server configuration parameter max_expr_depth parameter has been replaced with max_stack_depth which measures the physical stack size rather than the expression nesting depth. This helps prevent session termination due to stack overflow caused by recursive functions.
•
The length() function no longer counts trailing spaces in CHAR(n) values.
•
Casting an integer to BIT(N) selects the rightmost N bits of the integer, not the leftmost N bits as before.
•
Updating an element or slice of a NULL array value now produces a nonnull array result, namely an array containing just the assigned-to positions.
•
Syntax checking of array input values has been tightened up considerably. Junk that was previously allowed in odd places with odd results now causes an error. Empty-string element values must now be written as "", rather than writing nothing. Also changed behavior with respect to whitespace surrounding array elements: trailing whitespace is now ignored, for symmetry with leading whitespace (which has always been ignored).
•
Overflow in integer arithmetic operations is now detected and reported as an error.
•
The arithmetic operators associated with the single-byte "char" data type have been removed.
•
The extract() function (also called date_part) now returns the proper year for BC dates. It previously returned one less than the correct year. The function now also returns the proper values for millennium and century.
• CIDR
values now must have their nonmasked bits be zero. For example, we no longer allow
204.248.199.1/31 as a CIDR value. Such values should never have been accepted by Post-
greSQL and will now be rejected. • EXECUTE •
now returns a completion tag that matches the executed statement.
psql’s \copy command now reads or writes to the query’s stdin/stdout, rather than psql’s stdin/stdout. The previous behavior can be accessed via new pstdin/pstdout parameters.
2526
Appendix E. Release Notes •
The JDBC client interface has been removed from the core distribution, and is now hosted at http://jdbc.postgresql.org.
•
The Tcl client interface has also been removed. There are several Tcl interfaces now hosted at http://gborg.postgresql.org.
•
The server now uses its own time zone database, rather than the one supplied by the operating system. This will provide consistent behavior across all platforms. In most cases, there should be little noticeable difference in time zone behavior, except that the time zone names used by SET/SHOW TimeZone might be different from what your platform provides.
•
Configure’s threading option no longer requires users to run tests or edit configuration files; threading options are now detected automatically.
•
Now that tablespaces have been implemented, initlocation has been removed.
•
The API for user-defined GiST indexes has been changed. The Union and PickSplit methods are now passed a pointer to a special GistEntryVector structure, rather than a bytea.
E.215.3. Deprecated Features Some aspects of PostgreSQL’s behavior have been determined to be suboptimal. For the sake of backward compatibility these have not been removed in 8.0, but they are considered deprecated and will be removed in the next major release. •
The 8.1 release will remove the to_char() function for intervals.
•
The server now warns of empty strings passed to oid/float4/float8 data types, but continues to interpret them as zeroes as before. In the next major release, empty strings will be considered invalid input for these data types.
•
By default, tables in PostgreSQL 8.0 and earlier are created with OIDs. In the next release, this will not be the case: to create a table that contains OIDs, the WITH OIDS clause must be specified or the default_with_oids configuration parameter must be set. Users are encouraged to explicitly specify WITH OIDS if their tables require OIDs for compatibility with future releases of PostgreSQL.
E.215.4. Changes Below you will find a detailed account of the changes between release 8.0 and the previous major release.
E.215.4.1. Performance Improvements •
Support cross-data-type index usage (Tom) Before this change, many queries would not use an index if the data types did not match exactly. This improvement makes index usage more intuitive and consistent.
•
New buffer replacement strategy that improves caching (Jan) Prior releases used a least-recently-used (LRU) cache to keep recently referenced pages in memory. The LRU algorithm did not consider the number of times a specific cache entry was accessed, so large table scans could force out useful cache pages. The new cache algorithm uses four separate lists to track most recently used and most frequently used cache pages and dynamically optimize
2527
Appendix E. Release Notes their replacement based on the work load. This should lead to much more efficient use of the shared buffer cache. Administrators who have tested shared buffer sizes in the past should retest with this new cache replacement policy. •
Add subprocess to write dirty buffers periodically to reduce checkpoint writes (Jan) In previous releases, the checkpoint process, which runs every few minutes, would write all dirty buffers to the operating system’s buffer cache then flush all dirty operating system buffers to disk. This resulted in a periodic spike in disk usage that often hurt performance. The new code uses a background writer to trickle disk writes at a steady pace so checkpoints have far fewer dirty pages to write to disk. Also, the new code does not issue a global sync() call, but instead fsync()s just the files written since the last checkpoint. This should improve performance and minimize degradation during checkpoints.
•
Add ability to prolong vacuum to reduce performance impact (Jan) On busy systems, VACUUM performs many I/O requests which can hurt performance for other users. This release allows you to slow down VACUUM to reduce its impact on other users, though this increases the total duration of VACUUM.
•
Improve B-tree index performance for duplicate keys (Dmitry Tkach, Tom) This improves the way indexes are scanned when many duplicate values exist in the index.
•
Use dynamically-generated table size estimates while planning (Tom) Formerly the planner estimated table sizes using the values seen by the last VACUUM or ANALYZE, both as to physical table size (number of pages) and number of rows. Now, the current physical table size is obtained from the kernel, and the number of rows is estimated by multiplying the table size by the row density (rows per page) seen by the last VACUUM or ANALYZE. This should produce more reliable estimates in cases where the table size has changed significantly since the last housekeeping command.
•
Improved index usage with OR clauses (Tom) This allows the optimizer to use indexes in statements with many OR clauses that would not have been indexed in the past. It can also use multi-column indexes where the first column is specified and the second column is part of an OR clause.
•
Improve matching of partial index clauses (Tom) The server is now smarter about using partial indexes in queries involving complex WHERE clauses.
•
Improve performance of the GEQO optimizer (Tom) The GEQO optimizer is used to plan queries involving many tables (by default, twelve or more). This release speeds up the way queries are analyzed to decrease time spent in optimization.
•
Miscellaneous optimizer improvements There is not room here to list all the minor improvements made, but numerous special cases work better than in prior releases.
•
Improve lookup speed for C functions (Tom) This release uses a hash table to lookup information for dynamically loaded C functions. This improves their speed so they perform nearly as quickly as functions that are built into the server executable.
•
Add type-specific ANALYZE statistics capability (Mark Cave-Ayland) This feature allows more flexibility in generating statistics for nonstandard data types.
• ANALYZE
now collects statistics for expression indexes (Tom)
2528
Appendix E. Release Notes Expression indexes (also called functional indexes) allow users to index not just columns but the results of expressions and function calls. With this release, the optimizer can gather and use statistics about the contents of expression indexes. This will greatly improve the quality of planning for queries in which an expression index is relevant. •
New two-stage sampling method for ANALYZE (Manfred Koizar) This gives better statistics when the density of valid rows is very different in different regions of a table.
•
Speed up TRUNCATE (Tom) This buys back some of the performance loss observed in 7.4, while still keeping TRUNCATE transaction-safe.
E.215.4.2. Server Changes •
Add WAL file archiving and point-in-time recovery (Simon Riggs)
•
Add tablespaces so admins can control disk layout (Gavin)
•
Add a built-in log rotation program (Andreas Pflug) It is now possible to log server messages conveniently without relying on either syslog or an external log rotation program.
•
Add new read-only server configuration parameters to show server compile-time settings: block_size, integer_datetimes, max_function_args, max_identifier_length, max_index_keys (Joe)
•
Make quoting of sameuser, samegroup, and all remove special meaning of these terms in pg_hba.conf (Andrew)
•
Use clearer IPv6 name ::1/128 for localhost in default pg_hba.conf (Andrew)
•
Use CIDR format in pg_hba.conf examples (Andrew)
•
Rename server configuration parameters SortMem and VacuumMem to work_mem and maintenance_work_mem (Old names still supported) (Tom) This change was made to clarify that bulk operations such as index and foreign key creation use maintenance_work_mem, while work_mem is for workspaces used during query execution.
•
Allow logging of session disconnections using server configuration log_disconnections (Andrew)
•
Add new server configuration parameter log_line_prefix to allow control of information emitted in each log line (Andrew) Available information includes user name, database name, remote IP address, and session start time.
•
Remove server configuration parameters log_pid, log_timestamp, log_source_port; functionality superseded by log_line_prefix (Andrew)
•
Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter (Andrew, Tom) virtual_host could only specify a single IP address to listen on. listen_addresses allows
multiple addresses to be specified.
2529
Appendix E. Release Notes •
Listen on localhost by default, which eliminates the need for the -i postmaster switch in many scenarios (Andrew) Listening on localhost (127.0.0.1) opens no new security holes but allows configurations like Windows and JDBC, which do not support local sockets, to work without special adjustments.
•
Remove syslog server configuration parameter, and add more logical log_destination variable to control log output location (Magnus)
•
Change server configuration parameter log_statement to take values all, mod, ddl, or none to select which queries are logged (Bruce) This allows administrators to log only data definition changes or only data modification statements.
•
Some logging-related configuration parameters could formerly be adjusted by ordinary users, but only in the “more verbose” direction. They are now treated more strictly: only superusers can set them. However, a superuser can use ALTER USER to provide per-user settings of these values for non-superusers. Also, it is now possible for superusers to set values of superuser-only configuration parameters via PGOPTIONS.
•
Allow configuration files to be placed outside the data directory (mlw) By default, configuration files are kept in the cluster’s top directory. With this addition, configuration files can be placed outside the data directory, easing administration.
•
Plan prepared queries only when first executed so constants can be used for statistics (Oliver Jowett) Prepared statements plan queries once and execute them many times. While prepared queries avoid the overhead of re-planning on each use, the quality of the plan suffers from not knowing the exact parameters to be used in the query. In this release, planning of unnamed prepared statements is delayed until the first execution, and the actual parameter values of that execution are used as optimization hints. This allows use of out-of-line parameter passing without incurring a performance penalty.
•
Allow DECLARE CURSOR to take parameters (Oliver Jowett) It is now useful to issue DECLARE CURSOR in a Parse message with parameters. The parameter values sent at Bind time will be substituted into the execution of the cursor’s query.
•
Fix hash joins and aggregates of inet and cidr data types (Tom) Release 7.4 handled hashing of mixed inet and cidr values incorrectly. (This bug did not exist in prior releases because they wouldn’t try to hash either data type.)
•
Make log_duration print only when log_statement prints the query (Ed L.)
E.215.4.3. Query Changes •
Add savepoints (nested transactions) (Alvaro)
•
Unsupported isolation levels are now accepted and promoted to the nearest supported level (Peter) The SQL specification states that if a database doesn’t support a specific isolation level, it should use the next more restrictive level. This change complies with that recommendation.
•
Allow BEGIN WORK to specify transaction isolation levels like START TRANSACTION does (Bruce)
•
Fix table permission checking for cases in which rules generate a query type different from the originally submitted query (Tom)
•
Implement dollar quoting to simplify single-quote usage (Andrew, Tom, David Fetter)
2530
Appendix E. Release Notes In previous releases, because single quotes had to be used to quote a function’s body, the use of single quotes inside the function text required use of two single quotes or other error-prone notations. With this release we add the ability to use "dollar quoting" to quote a block of text. The ability to use different quoting delimiters at different nesting levels greatly simplifies the task of quoting correctly, especially in complex functions. Dollar quoting can be used anywhere quoted text is needed. •
Make CASE val WHEN compval1 THEN ... evaluate val only once (Tom) CASE no longer evaluates the tested expression multiple times. This has benefits when the expres-
sion is complex or is volatile. •
Test HAVING before computing target list of an aggregate query (Tom) Fixes improper failure of cases such as SELECT SUM(win)/SUM(lose) ... GROUP BY ... HAVING SUM(lose) > 0. This should work but formerly could fail with divide-by-zero.
•
Replace max_expr_depth parameter with max_stack_depth parameter, measured in kilobytes of stack size (Tom) This gives us a fairly bulletproof defense against crashing due to runaway recursive functions. Instead of measuring the depth of expression nesting, we now directly measure the size of the execution stack.
•
Allow arbitrary row expressions (Tom) This release allows SQL expressions to contain arbitrary composite types, that is, row values. It also allows functions to more easily take rows as arguments and return row values.
•
Allow LIKE/ILIKE to be used as the operator in row and subselect comparisons (Fabien Coelho)
•
Avoid locale-specific case conversion of basic ASCII letters in identifiers and keywords (Tom) This solves the “Turkish problem” with mangling of words containing I and i. Folding of characters outside the 7-bit-ASCII set is still locale-aware.
•
Improve syntax error reporting (Fabien, Tom) Syntax error reports are more useful than before.
•
Change EXECUTE to return a completion tag matching the executed statement (Kris Jurka) Previous releases return an EXECUTE tag for any EXECUTE call. In this release, the tag returned will reflect the command executed.
•
Avoid emitting NATURAL CROSS JOIN in rule listings (Tom) Such a clause makes no logical sense, but in some cases the rule decompiler formerly produced this syntax.
E.215.4.4. Object Manipulation Changes •
Add COMMENT ON for casts, conversions, languages, operator classes, and large objects (Christopher)
•
Add new server configuration parameter default_with_oids to control whether tables are created with OIDs by default (Neil) This allows administrators to control whether CREATE TABLE commands create tables with or without OID columns by default. (Note: the current factory default setting for default_with_oids is TRUE, but the default will become FALSE in future releases.)
2531
Appendix E. Release Notes •
Add WITH / WITHOUT OIDS clause to CREATE TABLE AS (Neil)
•
Allow ALTER TABLE DROP COLUMN to drop an OID column (ALTER TABLE SET WITHOUT OIDS still works) (Tom)
•
Allow composite types as table columns (Tom)
•
Allow ALTER ... ADD COLUMN with defaults and NOT NULL constraints; works per SQL spec (Rod) It is now possible for ADD COLUMN to create a column that is not initially filled with NULLs, but with a specified default value.
•
Add ALTER COLUMN TYPE to change column’s type (Rod) It is now possible to alter a column’s data type without dropping and re-adding the column.
•
Allow multiple ALTER actions in a single ALTER TABLE command (Rod) This is particularly useful for ALTER commands that rewrite the table (which include ALTER COLUMN TYPE and ADD COLUMN with a default). By grouping ALTER commands together, the table need be rewritten only once.
•
Allow ALTER TABLE to add SERIAL columns (Tom) This falls out from the new capability of specifying defaults for new columns.
•
Allow changing the owners of aggregates, conversions, databases, functions, operators, operator classes, schemas, types, and tablespaces (Christopher, Euler Taveira de Oliveira) Previously this required modifying the system tables directly.
•
Allow temporary object creation to be limited to SECURITY DEFINER functions (Sean Chittenden)
•
Add ALTER TABLE ... SET WITHOUT CLUSTER (Christopher) Prior to this release, there was no way to clear an auto-cluster specification except to modify the system tables.
•
Constraint/Index/SERIAL names are now table_column_type with numbers appended to guarantee uniqueness within the schema (Tom) The SQL specification states that such names should be unique within a schema.
•
Add pg_get_serial_sequence() to return a SERIAL column’s sequence name (Christopher) This allows automated scripts to reliably find the SERIAL sequence name.
•
Warn when primary/foreign key data type mismatch requires costly lookup
•
New ALTER INDEX command to allow moving of indexes between tablespaces (Gavin)
•
Make ALTER TABLE OWNER change dependent sequence ownership too (Alvaro)
E.215.4.5. Utility Command Changes •
Allow CREATE SCHEMA to create triggers, indexes, and sequences (Neil)
•
Add ALSO keyword to CREATE RULE (Fabien Coelho) This allows ALSO to be added to rule creation to contrast it with INSTEAD rules.
•
Add NOWAIT option to LOCK (Tatsuo) This allows the LOCK command to fail if it would have to wait for the requested lock.
•
Allow COPY to read and write comma-separated-value (CSV) files (Andrew, Bruce)
2532
Appendix E. Release Notes •
Generate error if the COPY delimiter and NULL string conflict (Bruce)
• GRANT/REVOKE •
behavior follows the SQL spec more closely
Avoid locking conflict between CREATE INDEX and CHECKPOINT (Tom) In 7.3 and 7.4, a long-running B-tree index build could block concurrent CHECKPOINTs from completing, thereby causing WAL bloat because the WAL log could not be recycled.
•
Database-wide ANALYZE does not hold locks across tables (Tom) This reduces the potential for deadlocks against other backends that want exclusive locks on tables. To get the benefit of this change, do not execute database-wide ANALYZE inside a transaction block (BEGIN block); it must be able to commit and start a new transaction for each table.
• REINDEX
does not exclusively lock the index’s parent table anymore
The index itself is still exclusively locked, but readers of the table can continue if they are not using the particular index being rebuilt. •
Erase MD5 user passwords when a user is renamed (Bruce) PostgreSQL uses the user name as salt when encrypting passwords via MD5. When a user’s name is changed, the salt will no longer match the stored MD5 password, so the stored password becomes useless. In this release a notice is generated and the password is cleared. A new password must then be assigned if the user is to be able to log in with a password.
•
New pg_ctl kill option for Windows (Andrew) Windows does not have a kill command to send signals to backends so this capability was added to pg_ctl.
•
Information schema improvements
•
Add --pwfile option to initdb so the initial password can be set by GUI tools (Magnus)
•
Detect locale/encoding mismatch in initdb (Peter)
•
Add register command to pg_ctl to register Windows operating system service (Dave Page)
E.215.4.6. Data Type and Function Changes •
More complete support for composite types (row types) (Tom) Composite values can be used in many places where only scalar values worked before.
•
Reject nonrectangular array values as erroneous (Joe) Formerly, array_in would silently build a surprising result.
•
Overflow in integer arithmetic operations is now detected (Tom)
•
The arithmetic operators associated with the single-byte "char" data type have been removed. Formerly, the parser would select these operators in many situations where an “unable to select an operator” error would be more appropriate, such as null * null. If you actually want to do arithmetic on a "char" column, you can cast it to integer explicitly.
•
Syntax checking of array input values considerably tightened up (Joe) Junk that was previously allowed in odd places with odd results now causes an ERROR, for example, non-whitespace after the closing right brace.
•
Empty-string array element values must now be written as "", rather than writing nothing (Joe)
2533
Appendix E. Release Notes Formerly, both ways of writing an empty-string element value were allowed, but now a quoted empty string is required. The case where nothing at all appears will probably be considered to be a NULL element value in some future release. •
Array element trailing whitespace is now ignored (Joe) Formerly leading whitespace was ignored, but trailing whitespace between an element value and the delimiter or right brace was significant. Now trailing whitespace is also ignored.
•
Emit array values with explicit array bounds when lower bound is not one (Joe)
•
Accept YYYY-monthname-DD as a date string (Tom)
•
Make netmask and hostmask functions return maximum-length mask length (Tom)
•
Change factorial function to return numeric (Gavin) Returning numeric allows the factorial function to work for a wider range of input values.
• to_char/to_date() •
date conversion improvements (Kurt Roeckx, Fabien Coelho)
Make length() disregard trailing spaces in CHAR(n) (Gavin) This change was made to improve consistency: trailing spaces are semantically insignificant in CHAR(n) data, so they should not be counted by length().
•
Warn about empty string being passed to OID/float4/float8 data types (Neil) 8.1 will throw an error instead.
•
Allow leading or trailing whitespace in int2/int4/int8/float4/float8 input routines (Neil)
•
Better support for IEEE Infinity and NaN values in float4/float8 (Neil) These should now work on all platforms that support IEEE-compliant floating point arithmetic.
•
Add week option to date_trunc() (Robert Creager)
•
Fix to_char for 1 BC (previously it returned 1 AD) (Bruce)
•
Fix date_part(year) for BC dates (previously it returned one less than the correct year) (Bruce)
•
Fix date_part() to return the proper millennium and century (Fabien Coelho) In previous versions, the century and millennium results had a wrong number and started in the wrong year, as compared to standard reckoning of such things.
•
Add ceiling() as an alias for ceil(), and power() as an alias for pow() for standards compliance (Neil)
•
Change ln(), log(), power(), and sqrt() to emit the correct SQLSTATE error codes for certain error conditions, as specified by SQL:2003 (Neil)
•
Add width_bucket() function as defined by SQL:2003 (Neil)
•
Add generate_series() functions to simplify working with numeric sets (Joe)
•
Fix upper/lower/initcap() functions to work with multibyte encodings (Tom)
•
Add boolean and bitwise integer AND/OR aggregates (Fabien Coelho)
•
New session information functions to return network addresses for client and server (Sean Chittenden)
•
Add function to determine the area of a closed path (Sean Chittenden)
•
Add function to send cancel request to other backends (Magnus)
•
Add interval plus datetime operators (Tom)
2534
Appendix E. Release Notes The reverse ordering, datetime plus interval, was already supported, but both are required by the SQL standard. •
Casting an integer to BIT(N) selects the rightmost N bits of the integer (Tom) In prior releases, the leftmost N bits were selected, but this was deemed unhelpful, not to mention inconsistent with casting from bit to int.
•
Require CIDR values to have all nonmasked bits be zero (Kevin Brintnall)
E.215.4.7. Server-Side Language Changes •
In READ COMMITTED serialization mode, volatile functions now see the results of concurrent transactions committed up to the beginning of each statement within the function, rather than up to the beginning of the interactive command that called the function.
•
Functions declared STABLE or IMMUTABLE always use the snapshot of the calling query, and therefore do not see the effects of actions taken after the calling query starts, whether in their own transaction or other transactions. Such a function must be read-only, too, meaning that it cannot use any SQL commands other than SELECT. There is a considerable performance gain from declaring a function STABLE or IMMUTABLE rather than VOLATILE.
•
Nondeferred AFTER triggers are now fired immediately after completion of the triggering query, rather than upon finishing the current interactive command. This makes a difference when the triggering query occurred within a function: the trigger is invoked before the function proceeds to its next operation. For example, if a function inserts a new row into a table, any nondeferred foreign key checks occur before proceeding with the function.
•
Allow function parameters to be declared with names (Dennis Björklund) This allows better documentation of functions. Whether the names actually do anything depends on the specific function language being used.
•
Allow PL/pgSQL parameter names to be referenced in the function (Dennis Björklund) This basically creates an automatic alias for each named parameter.
•
Do minimal syntax checking of PL/pgSQL functions at creation time (Tom) This allows us to catch simple syntax errors sooner.
•
More support for composite types (row and record variables) in PL/pgSQL For example, it now works to pass a rowtype variable to another function as a single variable.
•
Default values for PL/pgSQL variables can now reference previously declared variables
•
Improve parsing of PL/pgSQL FOR loops (Tom) Parsing is now driven by presence of ".." rather than data type of FOR variable. This makes no difference for correct functions, but should result in more understandable error messages when a mistake is made.
•
Major overhaul of PL/Perl server-side language (Command Prompt, Andrew Dunstan)
•
In PL/Tcl, SPI commands are now run in subtransactions. If an error occurs, the subtransaction is cleaned up and the error is reported as an ordinary Tcl error, which can be trapped with catch. Formerly, it was not possible to catch such errors.
•
Accept ELSEIF in PL/pgSQL (Neil)
2535
Appendix E. Release Notes Previously PL/pgSQL only allowed ELSIF, but many people are accustomed to spelling this keyword ELSEIF.
E.215.4.8. psql Changes •
Improve psql information display about database objects (Christopher)
•
Allow psql to display group membership in \du and \dg (Markus Bertheau)
•
Prevent psql \dn from showing temporary schemas (Bruce)
•
Allow psql to handle tilde user expansion for file names (Zach Irmen)
•
Allow psql to display fancy prompts, including color, via readline (Reece Hart, Chet Ramey)
•
Make psql \copy match COPY command syntax fully (Tom)
•
Show the location of syntax errors (Fabien Coelho, Tom)
•
Add CLUSTER information to psql \d display (Bruce)
•
Change psql \copy stdin/stdout to read from command input/output (Bruce)
•
Add pstdin/pstdout to read from psql’s stdin/stdout (Mark Feit)
•
Add global psql configuration file, psqlrc.sample (Bruce) This allows a central file where global psql startup commands can be stored.
•
Have psql \d+ indicate if the table has an OID column (Neil)
•
On Windows, use binary mode in psql when reading files so control-Z is not seen as end-of-file
•
Have \dn+ show permissions and description for schemas (Dennis Björklund)
•
Improve tab completion support (Stefan Kaltenbrunn, Greg Sabino Mullane)
•
Allow boolean settings to be set using upper or lower case (Michael Paesold)
E.215.4.9. pg_dump Changes •
Use dependency information to improve the reliability of pg_dump (Tom) This should solve the longstanding problems with related objects sometimes being dumped in the wrong order.
•
Have pg_dump output objects in alphabetical order if possible (Tom) This should make it easier to identify changes between dump files.
•
Allow pg_restore to ignore some SQL errors (Fabien Coelho) This makes pg_restore’s behavior similar to the results of feeding a pg_dump output script to psql. In most cases, ignoring errors and plowing ahead is the most useful thing to do. Also added was a pg_restore option to give the old behavior of exiting on an error.
•
pg_restore -l display now includes objects’ schema names
•
New begin/end markers in pg_dump text output (Bruce)
•
Add start/stop times for pg_dump/pg_dumpall in verbose mode (Bruce)
•
Allow most pg_dump options in pg_dumpall (Christopher)
2536
Appendix E. Release Notes •
Have pg_dump use ALTER OWNER rather than SET SESSION AUTHORIZATION by default (Christopher)
E.215.4.10. libpq Changes •
Make libpq’s SIGPIPE handling thread-safe (Bruce)
•
Add PQmbdsplen() which returns the display length of a character (Tatsuo)
•
Add thread locking to SSL and Kerberos connections (Manfred Spraul)
•
Allow PQoidValue(), PQcmdTuples(), and PQoidStatus() to work on EXECUTE commands (Neil)
•
Add PQserverVersion() to provide more convenient access to the server version number (Greg Sabino Mullane)
•
Add PQprepare/PQsendPrepared() functions to support preparing statements without necessarily specifying the data types of their parameters (Abhijit Menon-Sen)
•
Many ECPG improvements, including SET DESCRIPTOR (Michael)
E.215.4.11. Source Code Changes •
Allow the database server to run natively on Windows (Claudio, Magnus, Andrew)
•
Shell script commands converted to C versions for Windows support (Andrew)
•
Create an extension makefile framework (Fabien Coelho, Peter) This simplifies the task of building extensions outside the original source tree.
•
Support relocatable installations (Bruce) Directory paths for installed files (such as the /share directory) are now computed relative to the actual location of the executables, so that an installation tree can be moved to another place without reconfiguring and rebuilding.
•
Use --with-docdir to choose installation location of documentation; also allow --infodir (Peter)
•
Add --without-docdir to prevent installation of documentation (Peter)
•
Upgrade to DocBook V4.2 SGML (Peter)
•
New PostgreSQL CVS tag (Marc) This was done to make it easier for organizations to manage their own copies of the PostgreSQL CVS repository. File version stamps from the master repository will not get munged by checking into or out of a copied repository.
•
Clarify locking code (Manfred Koizar)
•
Buffer manager cleanup (Neil)
•
Decouple platform tests from CPU spinlock code (Bruce, Tom)
•
Add inlined test-and-set code on PA-RISC for gcc (ViSolve, Tom)
•
Improve i386 spinlock code (Manfred Spraul)
•
Clean up spinlock assembly code to avoid warnings from newer gcc releases (Tom)
2537
Appendix E. Release Notes •
Remove JDBC from source tree; now a separate project
•
Remove the libpgtcl client interface; now a separate project
•
More accurately estimate memory and file descriptor usage (Tom)
•
Improvements to the Mac OS X startup scripts (Ray A.)
•
New fsync() test program (Bruce)
•
Major documentation improvements (Neil, Peter)
•
Remove pg_encoding; not needed anymore
•
Remove pg_id; not needed anymore
•
Remove initlocation; not needed anymore
•
Auto-detect thread flags (no more manual testing) (Bruce)
•
Use Olson’s public domain timezone library (Magnus)
•
With threading enabled, use thread flags on Unixware for backend executables too (Bruce) Unixware cannot mix threaded and nonthreaded object files in the same executable, so everything must be compiled as threaded.
•
psql now uses a flex-generated lexical analyzer to process command strings
•
Reimplement the linked list data structure used throughout the backend (Neil) This improves performance by allowing list append and length operations to be more efficient.
•
Allow dynamically loaded modules to create their own server configuration parameters (Thomas Hallgren)
•
New Brazilian version of FAQ (Euler Taveira de Oliveira)
•
Add French FAQ (Guillaume Lelarge)
•
New pgevent for Windows logging
•
Make libpq and ECPG build as proper shared libraries on OS X (Tom)
E.215.4.12. Contrib Changes •
Overhaul of contrib/dblink (Joe)
• contrib/dbmirror
improvements (Steven Singer)
•
New contrib/xml2 (John Gray, Torchbox)
•
Updated contrib/mysql
•
New version of contrib/btree_gist (Teodor)
•
New contrib/trgm, trigram matching for PostgreSQL (Teodor)
•
Many contrib/tsearch2 improvements (Teodor)
•
Add double metaphone to contrib/fuzzystrmatch (Andrew)
•
Allow contrib/pg_autovacuum to run as a Windows service (Dave Page)
•
Add functions to contrib/dbsize (Andreas Pflug)
•
Removed contrib/pg_logger: obsoleted by integrated logging subprocess
•
Removed contrib/rserv: obsoleted by various separate projects
2538
Appendix E. Release Notes
E.216. Release 7.4.30 Release date: 2010-10-04 This release contains a variety of fixes from 7.4.29. For information about new features in the 7.4 major release, see Section E.246. This is expected to be the last PostgreSQL release in the 7.4.X series. Users are encouraged to update to a newer release branch soon.
E.216.1. Migration to Version 7.4.30 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.26, see Section E.220.
E.216.2. Changes •
Use a separate interpreter for each calling SQL userid in PL/Perl and PL/Tcl (Tom Lane) This change prevents security problems that can be caused by subverting Perl or Tcl code that will be executed later in the same session under another SQL user identity (for example, within a SECURITY DEFINER function). Most scripting languages offer numerous ways that that might be done, such as redefining standard functions or operators called by the target function. Without this change, any SQL user with Perl or Tcl language usage rights can do essentially anything with the SQL privileges of the target function’s owner. The cost of this change is that intentional communication among Perl and Tcl functions becomes more difficult. To provide an escape hatch, PL/PerlU and PL/TclU functions continue to use only one interpreter per session. This is not considered a security issue since all such functions execute at the trust level of a database superuser already. It is likely that third-party procedural languages that claim to offer trusted execution have similar security issues. We advise contacting the authors of any PL you are depending on for securitycritical purposes. Our thanks to Tim Bunce for pointing out this issue (CVE-2010-3433).
•
Prevent possible crashes in pg_get_expr() by disallowing it from being called with an argument that is not one of the system catalog columns it’s intended to be used with (Heikki Linnakangas, Tom Lane)
•
Fix “cannot handle unplanned sub-select” error (Tom Lane) This occurred when a sub-select contains a join alias reference that expands into an expression containing another sub-select.
•
Take care to fsync the contents of lockfiles (both postmaster.pid and the socket lockfile) while writing them (Tom Lane) This omission could result in corrupted lockfile contents if the machine crashes shortly after postmaster start. That could in turn prevent subsequent attempts to start the postmaster from succeeding, until the lockfile is manually removed.
•
Improve contrib/dblink’s handling of tables containing dropped columns (Tom Lane)
•
Fix connection leak after “duplicate connection name” errors in contrib/dblink (Itagaki Takahiro)
2539
Appendix E. Release Notes •
Update build infrastructure and documentation to reflect the source code repository’s move from CVS to Git (Magnus Hagander and others)
E.217. Release 7.4.29 Release date: 2010-05-17 This release contains a variety of fixes from 7.4.28. For information about new features in the 7.4 major release, see Section E.246. The PostgreSQL community will stop releasing updates for the 7.4.X release series in July 2010. Users are encouraged to update to a newer release branch soon.
E.217.1. Migration to Version 7.4.29 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.26, see Section E.220.
E.217.2. Changes •
Enforce restrictions in plperl using an opmask applied to the whole interpreter, instead of using Safe.pm (Tim Bunce, Andrew Dunstan) Recent developments have convinced us that Safe.pm is too insecure to rely on for making plperl trustable. This change removes use of Safe.pm altogether, in favor of using a separate interpreter with an opcode mask that is always applied. Pleasant side effects of the change include that it is now possible to use Perl’s strict pragma in a natural way in plperl, and that Perl’s $a and $b variables work as expected in sort routines, and that function compilation is significantly faster. (CVE-2010-1169)
•
Prevent PL/Tcl from executing untrustworthy code from pltcl_modules (Tom) PL/Tcl’s feature for autoloading Tcl code from a database table could be exploited for trojan-horse attacks, because there was no restriction on who could create or insert into that table. This change disables the feature unless pltcl_modules is owned by a superuser. (However, the permissions on the table are not checked, so installations that really need a less-than-secure modules table can still grant suitable privileges to trusted non-superusers.) Also, prevent loading code into the unrestricted “normal” Tcl interpreter unless we are really going to execute a pltclu function. (CVE-2010-1170)
•
Do not allow an unprivileged user to reset superuser-only parameter settings (Alvaro) Previously, if an unprivileged user ran ALTER USER ... RESET ALL for himself, or ALTER DATABASE ... RESET ALL for a database he owns, this would remove all special parameter settings for the user or database, even ones that are only supposed to be changeable by a superuser. Now, the ALTER will only remove the parameters that the user has permission to change.
•
Avoid possible crash during backend shutdown if shutdown occurs when a CONTEXT addition would be made to log entries (Tom) In some cases the context-printing function would fail because the current transaction had already been rolled back when it came time to print a log message.
2540
Appendix E. Release Notes •
Update pl/perl’s ppport.h for modern Perl versions (Andrew)
•
Fix assorted memory leaks in pl/python (Andreas Freund, Tom)
•
Ensure that contrib/pgstattuple functions respond to cancel interrupts promptly (Tatsuhito Kasahara)
•
Make server startup deal properly with the case that shmget() returns EINVAL for an existing shared memory segment (Tom) This behavior has been observed on BSD-derived kernels including OS X. It resulted in an entirelymisleading startup failure complaining that the shared memory request size was too large.
E.218. Release 7.4.28 Release date: 2010-03-15 This release contains a variety of fixes from 7.4.27. For information about new features in the 7.4 major release, see Section E.246. The PostgreSQL community will stop releasing updates for the 7.4.X release series in July 2010. Users are encouraged to update to a newer release branch soon.
E.218.1. Migration to Version 7.4.28 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.26, see Section E.220.
E.218.2. Changes •
Add new configuration parameter ssl_renegotiation_limit to control how often we do session key renegotiation for an SSL connection (Magnus) This can be set to zero to disable renegotiation completely, which may be required if a broken SSL library is used. In particular, some vendors are shipping stopgap patches for CVE-2009-3555 that cause renegotiation attempts to fail.
•
Make substring() for bit types treat any negative length as meaning “all the rest of the string” (Tom) The previous coding treated only -1 that way, and would produce an invalid result value for other negative values, possibly leading to a crash (CVE-2010-0442).
•
Fix some cases of pathologically slow regular expression matching (Tom)
•
When reading pg_hba.conf and related files, do not treat @something as a file inclusion request if the @ appears inside quote marks; also, never treat @ by itself as a file inclusion request (Tom) This prevents erratic behavior if a role or database name starts with @. If you need to include a file whose path name contains spaces, you can still do so, but you must write @"/path to/file" rather than putting the quotes around the whole construct.
•
Prevent infinite loop on some platforms if a directory is named as an inclusion target in pg_hba.conf and related files (Tom)
2541
Appendix E. Release Notes •
Ensure PL/Tcl initializes the Tcl interpreter fully (Tom) The only known symptom of this oversight is that the Tcl clock command misbehaves if using Tcl 8.5 or later.
•
Prevent crash in contrib/dblink when too many key columns are specified to a dblink_build_sql_* function (Rushabh Lathia, Joe Conway)
E.219. Release 7.4.27 Release date: 2009-12-14 This release contains a variety of fixes from 7.4.26. For information about new features in the 7.4 major release, see Section E.246.
E.219.1. Migration to Version 7.4.27 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.26, see Section E.220.
E.219.2. Changes •
Protect against indirect security threats caused by index functions changing session-local state (Gurjeet Singh, Tom) This change prevents allegedly-immutable index functions from possibly subverting a superuser’s session (CVE-2009-4136).
•
Reject SSL certificates containing an embedded null byte in the common name (CN) field (Magnus) This prevents unintended matching of a certificate to a server or client name during SSL validation (CVE-2009-4034).
•
Fix possible crash during backend-startup-time cache initialization (Tom)
•
Prevent signals from interrupting VACUUM at unsafe times (Alvaro) This fix prevents a PANIC if a VACUUM FULL is canceled after it’s already committed its tuple movements, as well as transient errors if a plain VACUUM is interrupted after having truncated the table.
•
Fix possible crash due to integer overflow in hash table size calculation (Tom) This could occur with extremely large planner estimates for the size of a hashjoin’s result.
•
Fix very rare crash in inet/cidr comparisons (Chris Mikkelson)
•
Fix PAM password processing to be more robust (Tom) The previous code is known to fail with the combination of the Linux pam_krb5 PAM module with Microsoft Active Directory as the domain controller. It might have problems elsewhere too, since it was making unjustified assumptions about what arguments the PAM stack would pass to it.
•
Make the postmaster ignore any application_name parameter in connection request packets, to improve compatibility with future libpq versions (Tom)
2542
Appendix E. Release Notes
E.220. Release 7.4.26 Release date: 2009-09-09 This release contains a variety of fixes from 7.4.25. For information about new features in the 7.4 major release, see Section E.246.
E.220.1. Migration to Version 7.4.26 A dump/restore is not required for those running 7.4.X. However, if you have any hash indexes on interval columns, you must REINDEX them after updating to 7.4.26. Also, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.220.2. Changes •
Disallow RESET ROLE and RESET SESSION AUTHORIZATION inside security-definer functions (Tom, Heikki) This covers a case that was missed in the previous patch that disallowed SET ROLE and SET SESSION AUTHORIZATION inside security-definer functions. (See CVE-2007-6600)
•
Fix handling of sub-SELECTs appearing in the arguments of an outer-level aggregate function (Tom)
•
Fix hash calculation for data type interval (Tom) This corrects wrong results for hash joins on interval values. It also changes the contents of hash indexes on interval columns. If you have any such indexes, you must REINDEX them after updating.
•
Fix overflow for INTERVAL ’x ms’ when x is more than 2 million and integer datetimes are in use (Alex Hunsaker)
•
Fix calculation of distance between a point and a line segment (Tom) This led to incorrect results from a number of geometric operators.
•
Fix money data type to work in locales where currency amounts have no fractional digits, e.g. Japan (Itagaki Takahiro)
•
Properly round datetime input like 00:12:57.9999999999999999999999999999 (Tom)
•
Fix poor choice of page split point in GiST R-tree operator classes (Teodor)
•
Fix portability issues in plperl initialization (Andrew Dunstan)
•
Improve robustness of libpq’s code to recover from errors during COPY FROM STDIN (Tom)
•
Avoid including conflicting readline and editline header files when both libraries are installed (Zdenek Kotala)
E.221. Release 7.4.25 Release date: 2009-03-16 This release contains a variety of fixes from 7.4.24. For information about new features in the 7.4 major release, see Section E.246.
2543
Appendix E. Release Notes
E.221.1. Migration to Version 7.4.25 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.221.2. Changes •
Prevent error recursion crashes when encoding conversion fails (Tom) This change extends fixes made in the last two minor releases for related failure scenarios. The previous fixes were narrowly tailored for the original problem reports, but we have now recognized that any error thrown by an encoding conversion function could potentially lead to infinite recursion while trying to report the error. The solution therefore is to disable translation and encoding conversion and report the plain-ASCII form of any error message, if we find we have gotten into a recursive error reporting situation. (CVE-2009-0922)
•
Disallow CREATE CONVERSION with the wrong encodings for the specified conversion function (Heikki) This prevents one possible scenario for encoding conversion failure. The previous change is a backstop to guard against other kinds of failures in the same area.
•
Fix core dump when to_char() is given format codes that are inappropriate for the type of the data argument (Tom)
•
Add MUST (Mauritius Island Summer Time) to the default list of known timezone abbreviations (Xavier Bugaud)
E.222. Release 7.4.24 Release date: 2009-02-02 This release contains a variety of fixes from 7.4.23. For information about new features in the 7.4 major release, see Section E.246.
E.222.1. Migration to Version 7.4.24 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.222.2. Changes •
Improve handling of URLs in headline() function (Teodor)
•
Improve handling of overlength headlines in headline() function (Teodor)
•
Prevent possible Assert failure or misconversion if an encoding conversion is created with the wrong conversion function for the specified pair of encodings (Tom, Heikki)
•
Avoid unnecessary locking of small tables in VACUUM (Heikki)
2544
Appendix E. Release Notes •
Fix uninitialized variables in contrib/tsearch2’s get_covers() function (Teodor)
•
Fix bug in to_char()’s handling of TH format codes (Andreas Scherbaum)
•
Make all documentation reference pgsql-bugs and/or pgsql-hackers as appropriate, instead of the now-decommissioned pgsql-ports and pgsql-patches mailing lists (Tom)
E.223. Release 7.4.23 Release date: 2008-11-03 This release contains a variety of fixes from 7.4.22. For information about new features in the 7.4 major release, see Section E.246.
E.223.1. Migration to Version 7.4.23 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.223.2. Changes •
Fix backend crash when the client encoding cannot represent a localized error message (Tom) We have addressed similar issues before, but it would still fail if the “character has no equivalent” message itself couldn’t be converted. The fix is to disable localization and send the plain ASCII error message when we detect such a situation.
•
Fix incorrect tsearch2 headline generation when single query item matches first word of text (Sushant Sinha)
•
Fix improper display of fractional seconds in interval values when using a non-ISO datestyle in an --enable-integer-datetimes build (Ron Mayer)
•
Ensure SPI_getvalue and SPI_getbinval behave correctly when the passed tuple and tuple descriptor have different numbers of columns (Tom) This situation is normal when a table has had columns added or removed, but these two functions didn’t handle it properly. The only likely consequence is an incorrect error indication.
•
Fix ecpg’s parsing of CREATE USER (Michael)
E.224. Release 7.4.22 Release date: 2008-09-22 This release contains a variety of fixes from 7.4.21. For information about new features in the 7.4 major release, see Section E.246.
2545
Appendix E. Release Notes
E.224.1. Migration to Version 7.4.22 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.224.2. Changes •
Fix datetime input functions to correctly detect integer overflow when running on a 64-bit platform (Tom)
•
Improve performance of writing very long log messages to syslog (Tom)
•
Fix bug in backwards scanning of a cursor on a SELECT DISTINCT ON query (Tom)
•
Fix planner to estimate that GROUP BY expressions yielding boolean results always result in two groups, regardless of the expressions’ contents (Tom) This is very substantially more accurate than the regular GROUP BY estimate for certain boolean tests like col IS NULL.
•
Improve pg_dump and pg_restore’s error reporting after failure to send a SQL command (Tom)
E.225. Release 7.4.21 Release date: 2008-06-12 This release contains one serious bug fix over 7.4.20. For information about new features in the 7.4 major release, see Section E.246.
E.225.1. Migration to Version 7.4.21 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.225.2. Changes •
Make pg_get_ruledef() parenthesize negative constants (Tom) Before this fix, a negative constant in a view or rule might be dumped as, say, -42::integer, which is subtly incorrect: it should be (-42)::integer due to operator precedence rules. Usually this would make little difference, but it could interact with another recent patch to cause PostgreSQL to reject what had been a valid SELECT DISTINCT view query. Since this could result in pg_dump output failing to reload, it is being treated as a high-priority fix. The only released versions in which dump output is actually incorrect are 8.3.1 and 8.2.7.
2546
Appendix E. Release Notes
E.226. Release 7.4.20 Release date: never released This release contains a variety of fixes from 7.4.19. For information about new features in the 7.4 major release, see Section E.246.
E.226.1. Migration to Version 7.4.20 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.226.2. Changes •
Fix conversions between ISO-8859-5 and other encodings to handle Cyrillic “Yo” characters (e and E with two dots) (Sergey Burladyan)
•
Fix a few datatype input functions that were allowing unused bytes in their results to contain uninitialized, unpredictable values (Tom) This could lead to failures in which two apparently identical literal values were not seen as equal, resulting in the parser complaining about unmatched ORDER BY and DISTINCT expressions.
•
Fix a corner case in regular-expression substring matching (substring(string from pattern)) (Tom) The problem occurs when there is a match to the pattern overall but the user has specified a parenthesized subexpression and that subexpression hasn’t got a match. An example is substring(’foo’ from ’foo(bar)?’). This should return NULL, since (bar) isn’t matched, but it was mistakenly returning the whole-pattern match instead (ie, foo).
•
Fix incorrect result from ecpg’s PGTYPEStimestamp_sub() function (Michael)
•
Fix DatumGetBool macro to not fail with gcc 4.3 (Tom) This problem affects “old style” (V0) C functions that return boolean. The fix is already in 8.3, but the need to back-patch it was not realized at the time.
•
Fix longstanding LISTEN/NOTIFY race condition (Tom) In rare cases a session that had just executed a LISTEN might not get a notification, even though one would be expected because the concurrent transaction executing NOTIFY was observed to commit later. A side effect of the fix is that a transaction that has executed a not-yet-committed LISTEN command will not see any row in pg_listener for the LISTEN, should it choose to look; formerly it would have. This behavior was never documented one way or the other, but it is possible that some applications depend on the old behavior.
•
Fix display of constant expressions in ORDER BY and GROUP BY (Tom) An explicitly casted constant would be shown incorrectly. This could for example lead to corruption of a view definition during dump and reload.
•
Fix libpq to handle NOTICE messages correctly during COPY OUT (Tom) This failure has only been observed to occur when a user-defined datatype’s output routine issues a NOTICE, but there is no guarantee it couldn’t happen due to other causes.
2547
Appendix E. Release Notes
E.227. Release 7.4.19 Release date: 2008-01-07 This release contains a variety of fixes from 7.4.18, including fixes for significant security issues. For information about new features in the 7.4 major release, see Section E.246.
E.227.1. Migration to Version 7.4.19 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.227.2. Changes •
Prevent functions in indexes from executing with the privileges of the user running VACUUM, ANALYZE, etc (Tom)
Functions used in index expressions and partial-index predicates are evaluated whenever a new table entry is made. It has long been understood that this poses a risk of trojan-horse code execution if one modifies a table owned by an untrustworthy user. (Note that triggers, defaults, check constraints, etc. pose the same type of risk.) But functions in indexes pose extra danger because they will be executed by routine maintenance operations such as VACUUM FULL, which are commonly performed automatically under a superuser account. For example, a nefarious user can execute code with superuser privileges by setting up a trojan-horse index definition and waiting for the next routine vacuum. The fix arranges for standard maintenance operations (including VACUUM, ANALYZE, REINDEX, and CLUSTER) to execute as the table owner rather than the calling user, using the same privilege-switching mechanism already used for SECURITY DEFINER functions. To prevent bypassing this security measure, execution of SET SESSION AUTHORIZATION and SET ROLE is now forbidden within a SECURITY DEFINER context. (CVE-2007-6600) •
Repair assorted bugs in the regular-expression package (Tom, Will Drewry) Suitably crafted regular-expression patterns could cause crashes, infinite or near-infinite looping, and/or massive memory consumption, all of which pose denial-of-service hazards for applications that accept regex search patterns from untrustworthy sources. (CVE-2007-4769, CVE-2007-4772, CVE-2007-6067)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe) The fix that appeared for this in 7.4.18 was incomplete, as it plugged the hole for only some dblink functions. (CVE-2007-6601, CVE-2007-3278)
•
Fix planner failure in some cases of WHERE false AND var IN (SELECT ...) (Tom)
•
Fix potential crash in translate() when using a multibyte database encoding (Tom)
•
Fix PL/Python to not crash on long exception messages (Alvaro)
•
ecpg parser fixes (Michael)
•
Make contrib/tablefunc’s crosstab() handle NULL rowid as a category in its own right, rather than crashing (Joe)
•
Fix tsvector and tsquery output routines to escape backslashes correctly (Teodor, Bruce)
•
Fix crash of to_tsvector() on huge input strings (Teodor)
2548
Appendix E. Release Notes •
Require a specific version of Autoconf to be used when re-generating the configure script (Peter) This affects developers and packagers only. The change was made to prevent accidental use of untested combinations of Autoconf and PostgreSQL versions. You can remove the version check if you really want to use a different Autoconf version, but it’s your responsibility whether the result works or not.
E.228. Release 7.4.18 Release date: 2007-09-17 This release contains fixes from 7.4.17. For information about new features in the 7.4 major release, see Section E.246.
E.228.1. Migration to Version 7.4.18 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.228.2. Changes •
Prevent index corruption when a transaction inserts rows and then aborts close to the end of a concurrent VACUUM on the same table (Tom)
•
Make CREATE DOMAIN ... DEFAULT NULL work properly (Tom)
•
Fix excessive logging of SSL error messages (Tom)
•
Fix crash when log_min_error_statement logging runs out of memory (Tom)
•
Prevent CLUSTER from failing due to attempting to process temporary tables of other sessions (Alvaro)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe)
E.229. Release 7.4.17 Release date: 2007-04-23 This release contains fixes from 7.4.16, including a security fix. For information about new features in the 7.4 major release, see Section E.246.
E.229.1. Migration to Version 7.4.17 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
2549
Appendix E. Release Notes
E.229.2. Changes •
Support explicit placement of the temporary-table schema within search_path, and disable searching it for functions and operators (Tom) This is needed to allow a security-definer function to set a truly secure value of search_path. Without it, an unprivileged SQL user can use temporary objects to execute code with the privileges of the security-definer function (CVE-2007-2138). See CREATE FUNCTION for more information.
• /contrib/tsearch2
crash fixes (Teodor)
•
Fix potential-data-corruption bug in how VACUUM FULL handles UPDATE chains (Tom, Pavan Deolasee)
•
Fix PANIC during enlargement of a hash index (bug introduced in 7.4.15) (Tom)
E.230. Release 7.4.16 Release date: 2007-02-05 This release contains a variety of fixes from 7.4.15, including a security fix. For information about new features in the 7.4 major release, see Section E.246.
E.230.1. Migration to Version 7.4.16 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.230.2. Changes •
Remove security vulnerability that allowed connected users to read backend memory (Tom) The vulnerability involves suppressing the normal check that a SQL function returns the data type it’s declared to, or changing the data type of a table column used in a SQL function (CVE-20070555). This error can easily be exploited to cause a backend crash, and in principle might be used to read database content that the user should not be able to access.
•
Fix rare bug wherein btree index page splits could fail due to choosing an infeasible split point (Heikki Linnakangas)
•
Fix for rare Assert() crash triggered by UNION (Tom)
•
Tighten security of multi-byte character processing for UTF8 sequences over three bytes long (Tom)
E.231. Release 7.4.15 Release date: 2007-01-08
2550
Appendix E. Release Notes This release contains a variety of fixes from 7.4.14. For information about new features in the 7.4 major release, see Section E.246.
E.231.1. Migration to Version 7.4.15 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.231.2. Changes •
Improve handling of getaddrinfo() on AIX (Tom) This fixes a problem with starting the statistics collector, among other things.
•
Fix “failed to re-find parent key” errors in VACUUM (Tom)
Fix error when constructing an ARRAY[] made up of multiple empty elements (Tom)
• to_number() and to_char(numeric) are now STABLE, not IMMUTABLE, for new initdb installs
(Tom) This is because lc_numeric can potentially change the output of these functions. •
Improve index usage of regular expressions that use parentheses (Tom) This improves psql \d performance also.
E.232. Release 7.4.14 Release date: 2006-10-16 This release contains a variety of fixes from 7.4.13. For information about new features in the 7.4 major release, see Section E.246.
E.232.1. Migration to Version 7.4.14 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.232.2. Changes •
Fix core dump when an untyped literal is taken as ANYARRAY
•
Fix string_to_array() to handle overlapping matches for the separator string For example, string_to_array(’123xx456xxx789’, ’xx’).
•
Fix corner cases in pattern matching for psql’s \d commands
•
Fix index-corrupting bugs in /contrib/ltree (Teodor)
2551
Appendix E. Release Notes •
Fix backslash escaping in /contrib/dbmirror
•
Adjust regression tests for recent changes in US DST laws
E.233. Release 7.4.13 Release date: 2006-05-23 This release contains a variety of fixes from 7.4.12, including patches for extremely serious security issues. For information about new features in the 7.4 major release, see Section E.246.
E.233.1. Migration to Version 7.4.13 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235. Full security against the SQL-injection attacks described in CVE-2006-2313 and CVE-2006-2314 might require changes in application code. If you have applications that embed untrustworthy strings into SQL commands, you should examine them as soon as possible to ensure that they are using recommended escaping techniques. In most cases, applications should be using subroutines provided by libraries or drivers (such as libpq’s PQescapeStringConn()) to perform string escaping, rather than relying on ad hoc code to do it.
E.233.2. Changes •
Change the server to reject invalidly-encoded multibyte characters in all cases (Tatsuo, Tom) While PostgreSQL has been moving in this direction for some time, the checks are now applied uniformly to all encodings and all textual input, and are now always errors not merely warnings. This change defends against SQL-injection attacks of the type described in CVE-2006-2313.
•
Reject unsafe uses of \’ in string literals As a server-side defense against SQL-injection attacks of the type described in CVE-2006-2314, the server now only accepts ” and not \’ as a representation of ASCII single quote in SQL string literals. By default, \’ is rejected only when client_encoding is set to a client-only encoding (SJIS, BIG5, GBK, GB18030, or UHC), which is the scenario in which SQL injection is possible. A new configuration parameter backslash_quote is available to adjust this behavior when needed. Note that full security against CVE-2006-2314 might require client-side changes; the purpose of backslash_quote is in part to make it obvious that insecure clients are insecure.
•
Modify libpq’s string-escaping routines to be aware of encoding considerations and standard_conforming_strings
This fixes libpq-using applications for the security issues described in CVE-2006-2313 and CVE2006-2314, and also future-proofs them against the planned changeover to SQL-standard string literal syntax. Applications that use multiple PostgreSQL connections concurrently should migrate to PQescapeStringConn() and PQescapeByteaConn() to ensure that escaping is done correctly for the settings in use in each database connection. Applications that do string escaping “by hand” should be modified to rely on library routines instead. •
Fix some incorrect encoding conversion functions
2552
Appendix E. Release Notes win1251_to_iso, alt_to_iso, euc_tw_to_big5, euc_tw_to_mic, mic_to_euc_tw were all broken to varying extents. •
Clean up stray remaining uses of \’ in strings (Bruce, Jan)
•
Fix bug that sometimes caused OR’d index scans to miss rows they should have returned
•
Fix WAL replay for case where a btree index has been truncated
•
Fix SIMILAR TO for patterns involving | (Tom)
•
Fix server to use custom DH SSL parameters correctly (Michael Fuhr)
•
Fix for Bonjour on Intel Macs (Ashley Clark)
•
Fix various minor memory leaks
E.234. Release 7.4.12 Release date: 2006-02-14 This release contains a variety of fixes from 7.4.11. For information about new features in the 7.4 major release, see Section E.246.
E.234.1. Migration to Version 7.4.12 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.11, see Section E.235.
E.234.2. Changes •
Fix potential crash in SET SESSION AUTHORIZATION (CVE-2006-0553) An unprivileged user could crash the server process, resulting in momentary denial of service to other users, if the server has been compiled with Asserts enabled (which is not the default). Thanks to Akio Ishida for reporting this problem.
•
Fix bug with row visibility logic in self-inserted rows (Tom) Under rare circumstances a row inserted by the current command could be seen as already valid, when it should not be. Repairs bug created in 7.4.9 and 7.3.11 releases.
•
Fix race condition that could lead to “file already exists” errors during pg_clog file creation (Tom)
•
Properly check DOMAIN constraints for UNKNOWN parameters in prepared statements (Neil)
•
Fix to allow restoring dumps that have cross-schema references to custom operators (Tom)
•
Portability fix for testing presence of finite and isinf during configure (Tom)
E.235. Release 7.4.11 Release date: 2006-01-09
2553
Appendix E. Release Notes This release contains a variety of fixes from 7.4.10. For information about new features in the 7.4 major release, see Section E.246.
E.235.1. Migration to Version 7.4.11 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.8, see Section E.238. Also, you might need to REINDEX indexes on textual columns after updating, if you are affected by the locale or plperl issues described below.
E.235.2. Changes •
Fix for protocol-level Describe messages issued outside a transaction or in a failed transaction (Tom)
•
Fix character string comparison for locales that consider different character combinations as equal, such as Hungarian (Tom) This might require REINDEX to fix existing indexes on textual columns.
•
Set locale environment variables during postmaster startup to ensure that plperl won’t change the locale later This fixes a problem that occurred if the postmaster was started with environment variables specifying a different locale than what initdb had been told. Under these conditions, any use of plperl was likely to lead to corrupt indexes. You might need REINDEX to fix existing indexes on textual columns if this has happened to you.
•
Fix longstanding bug in strpos() and regular expression handling in certain rarely used Asian multibyte character sets (Tatsuo)
•
Fix bug in /contrib/pgcrypto gen_salt, which caused it not to use all available salt space for MD5 and XDES algorithms (Marko Kreen, Solar Designer) Salts for Blowfish and standard DES are unaffected.
•
Fix /contrib/dblink to throw an error, rather than crashing, when the number of columns specified is different from what’s actually returned by the query (Joe)
E.236. Release 7.4.10 Release date: 2005-12-12 This release contains a variety of fixes from 7.4.9. For information about new features in the 7.4 major release, see Section E.246.
E.236.1. Migration to Version 7.4.10 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.8, see Section E.238.
2554
Appendix E. Release Notes
E.236.2. Changes •
Fix race condition in transaction log management There was a narrow window in which an I/O operation could be initiated for the wrong page, leading to an Assert failure or data corruption.
•
Prevent failure if client sends Bind protocol message when current transaction is already aborted
• /contrib/ltree
fixes (Teodor)
•
AIX and HPUX compile fixes (Tom)
•
Fix longstanding planning error for outer joins This bug sometimes caused a bogus error “RIGHT JOIN is only supported with merge-joinable join conditions”.
•
Prevent core dump in pg_autovacuum when a table has been dropped
E.237. Release 7.4.9 Release date: 2005-10-04 This release contains a variety of fixes from 7.4.8. For information about new features in the 7.4 major release, see Section E.246.
E.237.1. Migration to Version 7.4.9 A dump/restore is not required for those running 7.4.X. However, if you are upgrading from a version earlier than 7.4.8, see Section E.238.
E.237.2. Changes •
Fix error that allowed VACUUM to remove ctid chains too soon, and add more checking in code that follows ctid links This fixes a long-standing problem that could cause crashes in very rare circumstances.
•
Fix CHAR() to properly pad spaces to the specified length when using a multiple-byte character set (Yoshiyuki Asaba) In prior releases, the padding of CHAR() was incorrect because it only padded to the specified number of bytes without considering how many characters were stored.
•
Fix the sense of the test for read-only transaction in COPY The code formerly prohibited COPY TO, where it should prohibit COPY FROM.
•
Fix planning problem with outer-join ON clauses that reference only the inner-side relation
•
Further fixes for x FULL JOIN y ON true corner cases
•
Make array_in and array_recv more paranoid about validating their OID parameter
•
Fix missing rows in queries like UPDATE a=... WHERE a... with GiST index on column a
2555
Appendix E. Release Notes •
Improve robustness of datetime parsing
•
Improve checking for partially-written WAL pages
•
Improve robustness of signal handling when SSL is enabled
•
Don’t try to open more than max_files_per_process files during postmaster startup
•
Various memory leakage fixes
•
Various portability improvements
•
Fix PL/pgSQL to handle var := var correctly when the variable is of pass-by-reference type
•
Update contrib/tsearch2 to use current Snowball code
E.238. Release 7.4.8 Release date: 2005-05-09 This release contains a variety of fixes from 7.4.7, including several security-related issues. For information about new features in the 7.4 major release, see Section E.246.
E.238.1. Migration to Version 7.4.8 A dump/restore is not required for those running 7.4.X. However, it is one possible way of handling two significant security problems that have been found in the initial contents of 7.4.X system catalogs. A dump/initdb/reload sequence using 7.4.8’s initdb will automatically correct these problems. The larger security problem is that the built-in character set encoding conversion functions can be invoked from SQL commands by unprivileged users, but the functions were not designed for such use and are not secure against malicious choices of arguments. The fix involves changing the declared parameter list of these functions so that they can no longer be invoked from SQL commands. (This does not affect their normal use by the encoding conversion machinery.) The lesser problem is that the contrib/tsearch2 module creates several functions that are misdeclared to return internal when they do not accept internal arguments. This breaks type safety for all functions using internal arguments. It is strongly recommended that all installations repair these errors, either by initdb or by following the manual repair procedures given below. The errors at least allow unprivileged database users to crash their server process, and might allow unprivileged users to gain the privileges of a database superuser. If you wish not to do an initdb, perform the following procedures instead. As the database superuser, do: BEGIN; UPDATE pg_proc SET proargtypes[3] = ’internal’::regtype WHERE pronamespace = 11 AND pronargs = 5 AND proargtypes[2] = ’cstring’::regtype; -- The command should report having updated 90 rows; -- if not, rollback and investigate instead of committing! COMMIT;
Next, if you have installed contrib/tsearch2, do: BEGIN;
2556
Appendix E. Release Notes UPDATE pg_proc SET proargtypes[0] = ’internal’::regtype WHERE oid IN ( ’dex_init(text)’::regprocedure, ’snb_en_init(text)’::regprocedure, ’snb_ru_init(text)’::regprocedure, ’spell_init(text)’::regprocedure, ’syn_init(text)’::regprocedure ); -- The command should report having updated 5 rows; -- if not, rollback and investigate instead of committing! COMMIT;
If this command fails with a message like “function "dex_init(text)" does not exist”, then either tsearch2 is not installed in this database, or you already did the update. The above procedures must be carried out in each database of an installation, including template1, and ideally including template0 as well. If you do not fix the template databases then any subsequently created databases will contain the same errors. template1 can be fixed in the same way as any other database, but fixing template0 requires additional steps. First, from any database issue: UPDATE pg_database SET datallowconn = true WHERE datname = ’template0’;
Next connect to template0 and perform the above repair procedures. Finally, do: -- re-freeze template0: VACUUM FREEZE; -- and protect it against future alterations: UPDATE pg_database SET datallowconn = false WHERE datname = ’template0’;
E.238.2. Changes •
Change encoding function signature to prevent misuse
•
Change contrib/tsearch2 to avoid unsafe use of INTERNAL function results
•
Repair ancient race condition that allowed a transaction to be seen as committed for some purposes (eg SELECT FOR UPDATE) slightly sooner than for other purposes This is an extremely serious bug since it could lead to apparent data inconsistencies being briefly visible to applications.
•
Repair race condition between relation extension and VACUUM This could theoretically have caused loss of a page’s worth of freshly-inserted data, although the scenario seems of very low probability. There are no known cases of it having caused more than an Assert failure.
•
Fix comparisons of TIME WITH TIME ZONE values The comparison code was wrong in the case where the --enable-integer-datetimes configuration switch had been used. NOTE: if you have an index on a TIME WITH TIME ZONE column, it will need to be REINDEXed after installing this update, because the fix corrects the sort order of column values.
•
Fix EXTRACT(EPOCH) for TIME WITH TIME ZONE values
2557
Appendix E. Release Notes •
Fix mis-display of negative fractional seconds in INTERVAL values This error only occurred when the --enable-integer-datetimes configuration switch had been used.
•
Ensure operations done during backend shutdown are counted by statistics collector This is expected to resolve reports of pg_autovacuum not vacuuming the system catalogs often enough — it was not being told about catalog deletions caused by temporary table removal during backend exit.
•
Additional buffer overrun checks in plpgsql (Neil)
•
Fix pg_dump to dump trigger names containing % correctly (Neil)
•
Fix contrib/pgcrypto for newer OpenSSL builds (Marko Kreen)
•
Still more 64-bit fixes for contrib/intagg
•
Prevent incorrect optimization of functions returning RECORD
•
Prevent to_char(interval) from dumping core for month-related formats
•
Prevent crash on COALESCE(NULL,NULL)
•
Fix array_map to call PL functions correctly
•
Fix permission checking in ALTER DATABASE RENAME
•
Fix ALTER LANGUAGE RENAME
•
Make RemoveFromWaitQueue clean up after itself This fixes a lock management error that would only be visible if a transaction was kicked out of a wait for a lock (typically by query cancel) and then the holder of the lock released it within a very narrow window.
•
Fix problem with untyped parameter appearing in INSERT ... SELECT
•
Fix CLUSTER failure after ALTER TABLE SET WITHOUT OIDS
E.239. Release 7.4.7 Release date: 2005-01-31 This release contains a variety of fixes from 7.4.6, including several security-related issues. For information about new features in the 7.4 major release, see Section E.246.
E.239.1. Migration to Version 7.4.7 A dump/restore is not required for those running 7.4.X.
E.239.2. Changes •
Disallow LOAD to non-superusers
2558
Appendix E. Release Notes On platforms that will automatically execute initialization functions of a shared library (this includes at least Windows and ELF-based Unixen), LOAD can be used to make the server execute arbitrary code. Thanks to NGS Software for reporting this. •
Check that creator of an aggregate function has the right to execute the specified transition functions This oversight made it possible to bypass denial of EXECUTE permission on a function.
•
Fix security and 64-bit issues in contrib/intagg
•
Add needed STRICT marking to some contrib functions (Kris Jurka)
•
Avoid buffer overrun when plpgsql cursor declaration has too many parameters (Neil)
•
Fix planning error for FULL and RIGHT outer joins The result of the join was mistakenly supposed to be sorted the same as the left input. This could not only deliver mis-sorted output to the user, but in case of nested merge joins could give outright wrong answers.
•
Fix plperl for quote marks in tuple fields
•
Fix display of negative intervals in SQL and GERMAN datestyles
•
Make age(timestamptz) do calculation in local timezone not GMT
E.240. Release 7.4.6 Release date: 2004-10-22 This release contains a variety of fixes from 7.4.5. For information about new features in the 7.4 major release, see Section E.246.
E.240.1. Migration to Version 7.4.6 A dump/restore is not required for those running 7.4.X.
E.240.2. Changes •
Repair possible failure to update hint bits on disk Under rare circumstances this oversight could lead to “could not access transaction status” failures, which qualifies it as a potential-data-loss bug.
•
Ensure that hashed outer join does not miss tuples Very large left joins using a hash join plan could fail to output unmatched left-side rows given just the right data distribution.
•
Disallow running pg_ctl as root This is to guard against any possible security issues.
•
Avoid using temp files in /tmp in make_oidjoins_check This has been reported as a security issue, though it’s hardly worthy of concern since there is no reason for non-developers to use this script anyway.
2559
Appendix E. Release Notes •
Prevent forced backend shutdown from re-emitting prior command result In rare cases, a client might think that its last command had succeeded when it really had been aborted by forced database shutdown.
•
Repair bug in pg_stat_get_backend_idset This could lead to misbehavior in some of the system-statistics views.
•
Fix small memory leak in postmaster
•
Fix “expected both swapped tables to have TOAST tables” bug This could arise in cases such as CLUSTER after ALTER TABLE DROP COLUMN.
•
Prevent pg_ctl restart from adding -D multiple times
•
Fix problem with NULL values in GiST indexes
• ::
is no longer interpreted as a variable in an ECPG prepare statement
E.241. Release 7.4.5 Release date: 2004-08-18 This release contains one serious bug fix over 7.4.4. For information about new features in the 7.4 major release, see Section E.246.
E.241.1. Migration to Version 7.4.5 A dump/restore is not required for those running 7.4.X.
E.241.2. Changes •
Repair possible crash during concurrent B-tree index insertions This patch fixes a rare case in which concurrent insertions into a B-tree index could result in a server panic. No permanent damage would result, but it’s still worth a re-release. The bug does not exist in pre-7.4 releases.
E.242. Release 7.4.4 Release date: 2004-08-16 This release contains a variety of fixes from 7.4.3. For information about new features in the 7.4 major release, see Section E.246.
E.242.1. Migration to Version 7.4.4 A dump/restore is not required for those running 7.4.X.
2560
Appendix E. Release Notes
E.242.2. Changes •
Prevent possible loss of committed transactions during crash Due to insufficient interlocking between transaction commit and checkpointing, it was possible for transactions committed just before the most recent checkpoint to be lost, in whole or in part, following a database crash and restart. This is a serious bug that has existed since PostgreSQL 7.1.
•
Check HAVING restriction before evaluating result list of an aggregate plan
•
Avoid crash when session’s current user ID is deleted
•
Fix hashed crosstab for zero-rows case (Joe)
•
Force cache update after renaming a column in a foreign key
•
Pretty-print UNION queries correctly
•
Make psql handle \r\n newlines properly in COPY IN
•
pg_dump handled ACLs with grant options incorrectly
•
Fix thread support for OS X and Solaris
•
Updated JDBC driver (build 215) with various fixes
•
ECPG fixes
•
Translation updates (various contributors)
E.243. Release 7.4.3 Release date: 2004-06-14 This release contains a variety of fixes from 7.4.2. For information about new features in the 7.4 major release, see Section E.246.
E.243.1. Migration to Version 7.4.3 A dump/restore is not required for those running 7.4.X.
E.243.2. Changes •
Fix temporary memory leak when using non-hashed aggregates (Tom)
•
ECPG fixes, including some for Informix compatibility (Michael)
•
Fixes for compiling with thread-safety, particularly Solaris (Bruce)
•
Fix error in COPY IN termination when using the old network protocol (ljb)
•
Several important fixes in pg_autovacuum, including fixes for large tables, unsigned oids, stability, temp tables, and debug mode (Matthew T. O’Connor)
•
Fix problem with reading tar-format dumps on NetBSD and BSD/OS (Bruce)
•
Several JDBC fixes
2561
Appendix E. Release Notes •
Fix ALTER SEQUENCE RESTART where last_value equals the restart value (Tom)
•
Repair failure to recalculate nested sub-selects (Tom)
•
Fix problems with non-constant expressions in LIMIT/OFFSET
•
Support FULL JOIN with no join clause, such as X FULL JOIN Y ON TRUE (Tom)
•
Fix another zero-column table bug (Tom)
•
Improve handling of non-qualified identifiers in GROUP BY clauses in sub-selects (Tom) Select-list aliases within the sub-select will now take precedence over names from outer query levels.
•
Do not generate “NATURAL CROSS JOIN” when decompiling rules (Tom)
•
Add checks for invalid field length in binary COPY (Tom) This fixes a difficult-to-exploit security hole.
•
Avoid locking conflict between ANALYZE and LISTEN/NOTIFY
E.244. Release 7.4.2 Release date: 2004-03-08 This release contains a variety of fixes from 7.4.1. For information about new features in the 7.4 major release, see Section E.246.
E.244.1. Migration to Version 7.4.2 A dump/restore is not required for those running 7.4.X. However, it might be advisable as the easiest method of incorporating fixes for two errors that have been found in the initial contents of 7.4.X system catalogs. A dump/initdb/reload sequence using 7.4.2’s initdb will automatically correct these problems. The more severe of the two errors is that data type anyarray has the wrong alignment label; this is a problem because the pg_statistic system catalog uses anyarray columns. The mislabeling can cause planner misestimations and even crashes when planning queries that involve WHERE clauses on double-aligned columns (such as float8 and timestamp). It is strongly recommended that all installations repair this error, either by initdb or by following the manual repair procedure given below. The lesser error is that the system view pg_settings ought to be marked as having public update access, to allow UPDATE pg_settings to be used as a substitute for SET. This can also be fixed either by initdb or manually, but it is not necessary to fix unless you want to use UPDATE pg_settings. If you wish not to do an initdb, the following procedure will work for fixing pg_statistic. As the database superuser, do: -- clear out old data in pg_statistic: DELETE FROM pg_statistic; VACUUM pg_statistic; -- this should update 1 row: UPDATE pg_type SET typalign = ’d’ WHERE oid = 2277; -- this should update 6 rows: UPDATE pg_attribute SET attalign = ’d’ WHERE atttypid = 2277;
2562
Appendix E. Release Notes --- At this point you MUST start a fresh backend to avoid a crash! --- repopulate pg_statistic: ANALYZE;
This can be done in a live database, but beware that all backends running in the altered database must be restarted before it is safe to repopulate pg_statistic. To repair the pg_settings error, simply do: GRANT SELECT, UPDATE ON pg_settings TO PUBLIC;
The above procedures must be carried out in each database of an installation, including template1, and ideally including template0 as well. If you do not fix the template databases then any subsequently created databases will contain the same errors. template1 can be fixed in the same way as any other database, but fixing template0 requires additional steps. First, from any database issue: UPDATE pg_database SET datallowconn = true WHERE datname = ’template0’;
Next connect to template0 and perform the above repair procedures. Finally, do: -- re-freeze template0: VACUUM FREEZE; -- and protect it against future alterations: UPDATE pg_database SET datallowconn = false WHERE datname = ’template0’;
E.244.2. Changes Release 7.4.2 incorporates all the fixes included in release 7.3.6, plus the following fixes: •
Fix pg_statistics alignment bug that could crash optimizer See above for details about this problem.
•
Allow non-super users to update pg_settings
•
Fix several optimizer bugs, most of which led to “variable not found in subplan target lists” errors
•
Avoid out-of-memory failure during startup of large multiple index scan
•
Fix multibyte problem that could lead to “out of memory” error during COPY IN
•
Fix problems with SELECT INTO / CREATE TABLE AS from tables without OIDs
•
Fix problems with alter_table regression test during parallel testing
•
Fix problems with hitting open file limit, especially on OS X (Tom)
•
Partial fix for Turkish-locale issues initdb will succeed now in Turkish locale, but there are still some inconveniences associated with the i/I problem.
•
Make pg_dump set client encoding on restore
•
Other minor pg_dump fixes
2563
Appendix E. Release Notes •
Allow ecpg to again use C keywords as column names (Michael)
•
Added ecpg WHENEVER NOT_FOUND to SELECT/INSERT/UPDATE/DELETE (Michael)
•
Fix ecpg crash for queries calling set-returning functions (Michael)
•
Various other ecpg fixes (Michael)
•
Fixes for Borland compiler
•
Thread build improvements (Bruce)
•
Various other build fixes
•
Various JDBC fixes
E.245. Release 7.4.1 Release date: 2003-12-22 This release contains a variety of fixes from 7.4. For information about new features in the 7.4 major release, see Section E.246.
E.245.1. Migration to Version 7.4.1 A dump/restore is not required for those running 7.4. If you want to install the fixes in the information schema you need to reload it into the database. This is either accomplished by initializing a new cluster by running initdb, or by running the following sequence of SQL commands in each database (ideally including template1) as a superuser in psql, after installing the new release: DROP SCHEMA information_schema CASCADE; \i /usr/local/pgsql/share/information_schema.sql
Substitute your installation path in the second command.
E.245.2. Changes •
Fixed bug in CREATE SCHEMA parsing in ECPG (Michael)
•
Fix compile error when --enable-thread-safety and --with-perl are used together (Peter)
•
Fix for subqueries that used hash joins (Tom) Certain subqueries that used hash joins would crash because of improperly shared structures.
•
Fix free space map compaction bug (Tom) This fixes a bug where compaction of the free space map could lead to a database server shutdown.
•
Fix for Borland compiler build of libpq (Bruce)
•
Fix netmask() and hostmask() to return the maximum-length masklen (Tom) Fix these functions to return values consistent with pre-7.4 releases.
•
Several contrib/pg_autovacuum fixes
2564
Appendix E. Release Notes Fixes include improper variable initialization, missing vacuum after TRUNCATE, and duration computation overflow for long vacuums. •
Allow compile of contrib/cube under Cygwin (Jason Tishler)
•
Fix Solaris use of password file when no passwords are defined (Tom) Fix crash on Solaris caused by use of any type of password authentication when no passwords were defined.
•
JDBC fix for thread problems, other fixes
•
Fix for bytea index lookups (Joe)
•
Fix information schema for bit data types (Peter)
•
Force zero_damaged_pages to be on during recovery from WAL
•
Prevent some obscure cases of “variable not in subplan target lists”
•
Make PQescapeBytea and byteaout consistent with each other (Joe)
•
Escape bytea output for bytes > 0x7e(Joe) If different client encodings are used for bytea output and input, it is possible for bytea values to be corrupted by the differing encodings. This fix escapes all bytes that might be affected.
•
Added missing SPI_finish() calls to dblink’s get_tuple_of_interest() (Joe)
•
New Czech FAQ
•
Fix information schema view constraint_column_usage for foreign keys (Peter)
•
ECPG fixes (Michael)
•
Fix bug with multiple IN subqueries and joins in the subqueries (Tom)
•
Allow COUNT(’x’) to work (Tom)
•
Install ECPG include files for Informix compatibility into separate directory (Peter) Some names of ECPG include files for Informix compatibility conflicted with operating system include files. By installing them in their own directory, name conflicts have been reduced.
•
Fix SSL memory leak (Neil) This release fixes a bug in 7.4 where SSL didn’t free all memory it allocated.
•
Prevent pg_service.conf from using service name as default dbname (Bruce)
•
Fix local ident authentication on FreeBSD (Tom)
E.246. Release 7.4 Release date: 2003-11-17
E.246.1. Overview Major changes in this release:
2565
Appendix E. Release Notes IN / NOT IN subqueries are now much more efficient
In previous releases, IN/NOT IN subqueries were joined to the upper query by sequentially scanning the subquery looking for a match. The 7.4 code uses the same sophisticated techniques used by ordinary joins and so is much faster. An IN will now usually be as fast as or faster than an equivalent EXISTS subquery; this reverses the conventional wisdom that applied to previous releases. Improved GROUP BY processing by using hash buckets In previous releases, rows to be grouped had to be sorted first. The 7.4 code can do GROUP BY without sorting, by accumulating results into a hash table with one entry per group. It will still use the sort technique, however, if the hash table is estimated to be too large to fit in sort_mem. New multikey hash join capability In previous releases, hash joins could only occur on single keys. This release allows multicolumn hash joins. Queries using the explicit JOIN syntax are now better optimized Prior releases evaluated queries using the explicit JOIN syntax only in the order implied by the syntax. 7.4 allows full optimization of these queries, meaning the optimizer considers all possible join orderings and chooses the most efficient. Outer joins, however, must still follow the declared ordering. Faster and more powerful regular expression code The entire regular expression module has been replaced with a new version by Henry Spencer, originally written for Tcl. The code greatly improves performance and supports several flavors of regular expressions. Function-inlining for simple SQL functions Simple SQL functions can now be inlined by including their SQL in the main query. This improves performance by eliminating per-call overhead. That means simple SQL functions now behave like macros. Full support for IPv6 connections and IPv6 address data types Previous releases allowed only IPv4 connections, and the IP data types only supported IPv4 addresses. This release adds full IPv6 support in both of these areas. Major improvements in SSL performance and reliability Several people very familiar with the SSL API have overhauled our SSL code to improve SSL key negotiation and error recovery. Make free space map efficiently reuse empty index pages, and other free space management improvements In previous releases, B-tree index pages that were left empty because of deleted rows could only be reused by rows with index values similar to the rows originally indexed on that page. In 7.4, VACUUM records empty index pages and allows them to be reused for any future index rows. SQL-standard information schema The information schema provides a standardized and stable way to access information about the schema objects defined in a database. Cursors conform more closely to the SQL standard The commands FETCH and MOVE have been overhauled to conform more closely to the SQL standard.
2566
Appendix E. Release Notes Cursors can exist outside transactions These cursors are also called holdable cursors. New client-to-server protocol The new protocol adds error codes, more status information, faster startup, better support for binary data transmission, parameter values separated from SQL commands, prepared statements available at the protocol level, and cleaner recovery from COPY failures. The older protocol is still supported by both server and clients. libpq and ECPG applications are now fully thread-safe While previous libpq releases already supported threads, this release improves thread safety by fixing some non-thread-safe code that was used during database connection startup. The configure option --enable-thread-safety must be used to enable this feature. New version of full-text indexing A new full-text indexing suite is available in contrib/tsearch2. New autovacuum tool The new autovacuum tool in contrib/autovacuum monitors the database statistics tables for INSERT/UPDATE/DELETE activity and automatically vacuums tables when needed. Array handling has been improved and moved into the server core Many array limitations have been removed, and arrays behave more like fully-supported data types.
E.246.2. Migration to Version 7.4 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: •
The server-side autocommit setting was removed and reimplemented in client applications and languages. Server-side autocommit was causing too many problems with languages and applications that wanted to control their own autocommit behavior, so autocommit was removed from the server and added to individual client APIs as appropriate.
•
Error message wording has changed substantially in this release. Significant effort was invested to make the messages more consistent and user-oriented. If your applications try to detect different error conditions by parsing the error message, you are strongly encouraged to use the new error code facility instead.
•
Inner joins using the explicit JOIN syntax might behave differently because they are now better optimized.
•
A number of server configuration parameters have been renamed for clarity, primarily those related to logging.
• FETCH 0 or MOVE 0 now does nothing. In prior releases, FETCH 0 would fetch all remaining rows,
and MOVE 0 would move to the end of the cursor. and MOVE now return the actual number of rows fetched/moved, or zero if at the beginning/end of the cursor. Prior releases would return the row count passed to the command, not the number of rows actually fetched or moved.
• FETCH
2567
Appendix E. Release Notes now can process files that use carriage-return or carriage-return/line-feed end-of-line sequences. Literal carriage-returns and line-feeds are no longer accepted in data values; use \r and \n instead.
• COPY
•
Trailing spaces are now trimmed when converting from type char(n) to varchar(n) or text. This is what most people always expected to happen anyway.
•
The data type float(p) now measures p in binary digits, not decimal digits. The new behavior follows the SQL standard.
•
Ambiguous date values now must match the ordering specified by the datestyle setting. In prior releases, a date specification of 10/20/03 was interpreted as a date in October even if datestyle specified that the day should be first. 7.4 will throw an error if a date specification is invalid for the current setting of datestyle.
•
The functions oidrand, oidsrand, and userfntest have been removed. These functions were determined to be no longer useful.
•
String literals specifying time-varying date/time values, such as ’now’ or ’today’ will no longer work as expected in column default expressions; they now cause the time of the table creation to be the default, not the time of the insertion. Functions such as now(), current_timestamp, or current_date should be used instead. In previous releases, there was special code so that strings such as ’now’ were interpreted at INSERT time and not at table creation time, but this work around didn’t cover all cases. Release 7.4 now requires that defaults be defined properly using functions such as now() or current_timestamp. These will work in all situations.
•
The dollar sign ($) is no longer allowed in operator names. It can instead be a non-first character in identifiers. This was done to improve compatibility with other database systems, and to avoid syntax problems when parameter placeholders ($n) are written adjacent to operators.
E.246.3. Changes Below you will find a detailed account of the changes between release 7.4 and the previous major release.
E.246.3.1. Server Operation Changes •
Allow IPv6 server connections (Nigel Kukard, Johan Jordaan, Bruce, Tom, Kurt Roeckx, Andrew Dunstan)
•
Fix SSL to handle errors cleanly (Nathan Mueller) In prior releases, certain SSL API error reports were not handled correctly. This release fixes those problems.
•
SSL protocol security and performance improvements (Sean Chittenden) SSL key renegotiation was happening too frequently, causing poor SSL performance. Also, initial key handling was improved.
•
Print lock information when a deadlock is detected (Tom) This allows easier debugging of deadlock situations.
•
Update /tmp socket modification times regularly to avoid their removal (Tom)
2568
Appendix E. Release Notes This should help prevent /tmp directory cleaner administration scripts from removing server socket files. •
Enable PAM for Mac OS X (Aaron Hillegass)
•
Make B-tree indexes fully WAL-safe (Tom) In prior releases, under certain rare cases, a server crash could cause B-tree indexes to become corrupt. This release removes those last few rare cases.
•
Allow B-tree index compaction and empty page reuse (Tom)
•
Fix inconsistent index lookups during split of first root page (Tom) In prior releases, when a single-page index split into two pages, there was a brief period when another database session could miss seeing an index entry. This release fixes that rare failure case.
•
Improve free space map allocation logic (Tom)
•
Preserve free space information between server restarts (Tom) In prior releases, the free space map was not saved when the postmaster was stopped, so newly started servers had no free space information. This release saves the free space map, and reloads it when the server is restarted.
•
Add start time to pg_stat_activity (Neil)
•
New code to detect corrupt disk pages; erase with zero_damaged_pages (Tom)
•
New client/server protocol: faster, no username length limit, allow clean exit from COPY (Tom)
•
Add transaction status, table ID, column ID to client/server protocol (Tom)
•
Add binary I/O to client/server protocol (Tom)
•
Remove autocommit server setting; move to client applications (Tom)
•
New error message wording, error codes, and three levels of error detail (Tom, Joe, Peter)
E.246.3.2. Performance Improvements •
Add hashing for GROUP BY aggregates (Tom)
•
Make nested-loop joins be smarter about multicolumn indexes (Tom)
•
Allow multikey hash joins (Tom)
•
Improve constant folding (Tom)
•
Add ability to inline simple SQL functions (Tom)
•
Reduce memory usage for queries using complex functions (Tom) In prior releases, functions returning allocated memory would not free it until the query completed. This release allows the freeing of function-allocated memory when the function call completes, reducing the total memory used by functions.
•
Improve GEQO optimizer performance (Tom) This release fixes several inefficiencies in the way the GEQO optimizer manages potential query paths.
•
Allow IN/NOT IN to be handled via hash tables (Tom)
•
Improve NOT IN (subquery ) performance (Tom)
•
Allow most IN subqueries to be processed as joins (Tom)
2569
Appendix E. Release Notes •
Pattern matching operations can use indexes regardless of locale (Peter) There is no way for non-ASCII locales to use the standard indexes for LIKE comparisons. This release adds a way to create a special index for LIKE.
•
Allow the postmaster to preload libraries using preload_libraries (Joe) For shared libraries that require a long time to load, this option is available so the library can be preloaded in the postmaster and inherited by all database sessions.
•
Improve optimizer cost computations, particularly for subqueries (Tom)
•
Avoid sort when subquery ORDER BY matches upper query (Tom)
•
Deduce that WHERE a.x = b.y AND b.y = 42 also means a.x = 42 (Tom)
•
Allow hash/merge joins on complex joins (Tom)
•
Allow hash joins for more data types (Tom)
•
Allow join optimization of explicit inner joins, disable with join_collapse_limit (Tom)
•
Add parameter from_collapse_limit to control conversion of subqueries to joins (Tom)
•
Use faster and more powerful regular expression code from Tcl (Henry Spencer, Tom)
•
Use bit-mapped relation sets in the optimizer (Tom)
•
Improve connection startup time (Tom) The new client/server protocol requires fewer network packets to start a database session.
•
Improve trigger/constraint performance (Stephan)
•
Improve speed of col IN (const, const, const, ...) (Tom)
•
Fix hash indexes which were broken in rare cases (Tom)
•
Improve hash index concurrency and speed (Tom) Prior releases suffered from poor hash index performance, particularly for high concurrency situations. This release fixes that, and the development group is interested in reports comparing B-tree and hash index performance.
•
Align shared buffers on 32-byte boundary for copy speed improvement (Manfred Spraul) Certain CPU’s perform faster data copies when addresses are 32-byte aligned.
•
Data type numeric reimplemented for better performance (Tom) numeric used to be stored in base 100. The new code uses base 10000, for significantly better
performance.
E.246.3.3. Server Configuration Changes •
Rename server parameter server_min_messages to log_min_messages (Bruce) This was done so most parameters that control the server logs begin with log_.
•
Rename show_*_stats to log_*_stats (Bruce)
•
Rename show_source_port to log_source_port (Bruce)
•
Rename hostname_lookup to log_hostname (Bruce)
•
Add checkpoint_warning to warn of excessive checkpointing (Bruce)
2570
Appendix E. Release Notes In prior releases, it was difficult to determine if checkpoint was happening too frequently. This feature adds a warning to the server logs when excessive checkpointing happens. •
New read-only server parameters for localization (Tom)
•
Change debug server log messages to output as DEBUG rather than LOG (Bruce)
•
Prevent server log variables from being turned off by non-superusers (Bruce) This is a security feature so non-superusers cannot disable logging that was enabled by the administrator.
• log_min_messages/client_min_messages
now controls debug_* output (Bruce)
This centralizes client debug information so all debug output can be sent to either the client or server logs. •
Add Mac OS X Rendezvous server support (Chris Campbell) This allows Mac OS X hosts to query the network for available PostgreSQL servers.
•
Add ability to print only slow statements using log_min_duration_statement (Christopher) This is an often requested debugging feature that allows administrators to see only slow queries in their server logs.
•
Allow pg_hba.conf to accept netmasks in CIDR format (Andrew Dunstan) This allows administrators to merge the host IP address and netmask fields into a single CIDR field in pg_hba.conf.
•
New read-only parameter is_superuser (Tom)
•
New parameter log_error_verbosity to control error detail (Tom) This works with the new error reporting feature to supply additional error information like hints, file names and line numbers.
• postgres --describe-config
now dumps server config variables (Aizaz Ahmed, Peter)
This option is useful for administration tools that need to know the configuration variable names and their minimums, maximums, defaults, and descriptions. •
Add new columns in pg_settings: context, type, source, min_val, max_val (Joe)
•
Make default shared_buffers 1000 and max_connections 100, if possible (Tom) Prior versions defaulted to 64 shared buffers so PostgreSQL would start on even very old systems. This release tests the amount of shared memory allowed by the platform and selects more reasonable default values if possible. Of course, users are still encouraged to evaluate their resource load and size shared_buffers accordingly.
•
New pg_hba.conf record type hostnossl to prevent SSL connections (Jon Jensen) In prior releases, there was no way to prevent SSL connections if both the client and server supported SSL. This option allows that capability.
•
Remove parameter geqo_random_seed (Tom)
•
Add server parameter regex_flavor to control regular expression processing (Tom)
•
Make pg_ctl better handle nonstandard ports (Greg)
2571
Appendix E. Release Notes
E.246.3.4. Query Changes •
New SQL-standard information schema (Peter)
•
Add read-only transactions (Peter)
•
Print key name and value in foreign-key violation messages (Dmitry Tkach)
•
Allow users to see their own queries in pg_stat_activity (Kevin Brown) In prior releases, only the superuser could see query strings using pg_stat_activity. Now ordinary users can see their own query strings.
•
Fix aggregates in subqueries to match SQL standard (Tom) The SQL standard says that an aggregate function appearing within a nested subquery belongs to the outer query if its argument contains only outer-query variables. Prior PostgreSQL releases did not handle this fine point correctly.
•
Add option to prevent auto-addition of tables referenced in query (Nigel J. Andrews) By default, tables mentioned in the query are automatically added to the FROM clause if they are not already there. This is compatible with historic POSTGRES behavior but is contrary to the SQL standard. This option allows selecting standard-compatible behavior.
•
Allow UPDATE ... SET col = DEFAULT (Rod) This allows UPDATE to set a column to its declared default value.
•
Allow expressions to be used in LIMIT/OFFSET (Tom) In prior releases, LIMIT/OFFSET could only use constants, not expressions.
•
Implement CREATE TABLE AS EXECUTE (Neil, Peter)
E.246.3.5. Object Manipulation Changes •
Make CREATE SEQUENCE grammar more conforming to SQL:2003 (Neil)
•
Add statement-level triggers (Neil) While this allows a trigger to fire at the end of a statement, it does not allow the trigger to access all rows modified by the statement. This capability is planned for a future release.
•
Add check constraints for domains (Rod) This greatly increases the usefulness of domains by allowing them to use check constraints.
•
Add ALTER DOMAIN (Rod) This allows manipulation of existing domains.
•
Fix several zero-column table bugs (Tom) PostgreSQL supports zero-column tables. This fixes various bugs that occur when using such tables.
•
Have ALTER TABLE ... ADD PRIMARY KEY add not-null constraint (Rod) In prior releases, ALTER TABLE ... ADD PRIMARY would add a unique index, but not a not-null constraint. That is fixed in this release.
•
Add ALTER TABLE ... WITHOUT OIDS (Rod) This allows control over whether new and updated rows will have an OID column. This is most useful for saving storage space.
2572
Appendix E. Release Notes •
Add ALTER SEQUENCE to modify minimum, maximum, increment, cache, cycle values (Rod)
•
Add ALTER TABLE ... CLUSTER ON (Alvaro Herrera) This command is used by pg_dump to record the cluster column for each table previously clustered. This information is used by database-wide cluster to cluster all previously clustered tables.
•
Improve automatic type casting for domains (Rod, Tom)
•
Allow dollar signs in identifiers, except as first character (Tom)
•
Disallow dollar signs in operator names, so x=$1 works (Tom)
•
Allow copying table schema using LIKE subtable, also SQL:2003 feature INCLUDING DEFAULTS (Rod)
•
Add WITH GRANT OPTION clause to GRANT (Peter) This enabled GRANT to give other users the ability to grant privileges on an object.
E.246.3.6. Utility Command Changes •
Add ON COMMIT clause to CREATE TABLE for temporary tables (Gavin) This adds the ability for a table to be dropped or all rows deleted on transaction commit.
•
Allow cursors outside transactions using WITH HOLD (Neil) In previous releases, cursors were removed at the end of the transaction that created them. Cursors can now be created with the WITH HOLD option, which allows them to continue to be accessed after the creating transaction has committed.
• FETCH 0
and MOVE 0 now do nothing (Bruce)
In previous releases, FETCH 0 fetched all remaining rows, and MOVE 0 moved to the end of the cursor. •
Cause FETCH and MOVE to return the number of rows fetched/moved, or zero if at the beginning/end of cursor, per SQL standard (Bruce) In prior releases, the row count returned by FETCH and MOVE did not accurately reflect the number of rows processed.
•
Properly handle SCROLL with cursors, or report an error (Neil) Allowing random access (both forward and backward scrolling) to some kinds of queries cannot be done without some additional work. If SCROLL is specified when the cursor is created, this additional work will be performed. Furthermore, if the cursor has been created with NO SCROLL, no random access is allowed.
•
Implement SQL-compatible options FIRST, LAST, ABSOLUTE n, RELATIVE n for FETCH and MOVE (Tom)
•
Allow EXPLAIN on DECLARE CURSOR (Tom)
•
Allow CLUSTER to use index marked as pre-clustered by default (Alvaro Herrera)
•
Allow CLUSTER to cluster all tables (Alvaro Herrera) This allows all previously clustered tables in a database to be reclustered with a single command.
•
Prevent CLUSTER on partial indexes (Tom)
•
Allow DOS and Mac line-endings in COPY files (Bruce)
2573
Appendix E. Release Notes •
Disallow literal carriage return as a data value, backslash-carriage-return and \r are still allowed (Bruce)
• COPY
changes (binary, \.) (Tom)
•
Recover from COPY failure cleanly (Tom)
•
Prevent possible memory leaks in COPY (Tom)
•
Make TRUNCATE transaction-safe (Rod) TRUNCATE can now be used inside a transaction. If the transaction aborts, the changes made by the TRUNCATE are automatically rolled back.
•
Allow prepare/bind of utility commands like FETCH and EXPLAIN (Tom)
•
Add EXPLAIN EXECUTE (Neil)
•
Improve VACUUM performance on indexes by reducing WAL traffic (Tom)
•
Functional indexes have been generalized into indexes on expressions (Tom) In prior releases, functional indexes only supported a simple function applied to one or more column names. This release allows any type of scalar expression.
•
Have SHOW TRANSACTION ISOLATION match input to SET TRANSACTION ISOLATION (Tom)
•
Have COMMENT ON DATABASE on nonlocal database generate a warning, rather than an error (Rod) Database comments are stored in database-local tables so comments on a database have to be stored in each database.
•
Improve reliability of LISTEN/NOTIFY (Tom)
•
Allow REINDEX to reliably reindex nonshared system catalog indexes (Tom) This allows system tables to be reindexed without the requirement of a standalone session, which was necessary in previous releases. The only tables that now require a standalone session for reindexing are the global system tables pg_database, pg_shadow, and pg_group.
E.246.3.7. Data Type and Function Changes •
New server parameter extra_float_digits to control precision display of floating-point numbers (Pedro Ferreira, Tom) This controls output precision which was causing regression testing problems.
•
Allow +1300 as a numeric time-zone specifier, for FJST (Tom)
•
Remove rarely used functions oidrand, oidsrand, and userfntest functions (Neil)
•
Add md5() function to main server, already in contrib/pgcrypto (Joe) An MD5 function was frequently requested. For more complex encryption capabilities, use contrib/pgcrypto.
•
Increase date range of timestamp (John Cochran)
•
Change EXTRACT(EPOCH FROM timestamp) so timestamp without time zone is assumed to be in local time, not GMT (Tom)
•
Trap division by zero in case the operating system doesn’t prevent it (Tom)
•
Change the numeric data type internally to base 10000 (Tom)
•
New hostmask() function (Greg Wickham)
2574
Appendix E. Release Notes •
Fixes for to_char() and to_timestamp() (Karel)
•
Allow functions that can take any argument data type and return any data type, using anyelement and anyarray (Joe) This allows the creation of functions that can work with any data type.
•
Arrays can now be specified as ARRAY[1,2,3], ARRAY[[’a’,’b’],[’c’,’d’]], or ARRAY[ARRAY[ARRAY[2]]] (Joe)
•
Allow proper comparisons for arrays, including ORDER BY and DISTINCT support (Joe)
•
Allow indexes on array columns (Joe)
•
Allow array concatenation with || (Joe)
•
Allow WHERE qualification expr op ANY/SOME/ALL (array_expr) (Joe) This allows arrays to behave like a list of values, for purposes like SELECT * FROM tab WHERE col IN (array_val).
Allow user defined aggregates to use polymorphic functions (Joe)
•
Allow assignments to empty arrays (Joe)
•
Allow 60 in seconds fields of time, timestamp, and interval input values (Tom) Sixty-second values are needed for leap seconds.
•
Allow cidr data type to be cast to text (Tom)
•
Disallow invalid time zone names in SET TIMEZONE
•
Trim trailing spaces when char is cast to varchar or text (Tom)
•
Make float(p) measure the precision p in binary digits, not decimal digits (Tom)
•
Add IPv6 support to the inet and cidr data types (Michael Graff)
•
Add family() function to report whether address is IPv4 or IPv6 (Michael Graff)
•
Have SHOW datestyle generate output similar to that used by SET datestyle (Tom)
•
Make EXTRACT(TIMEZONE) and SET/SHOW TIME ZONE follow the SQL convention for the sign of time zone offsets, i.e., positive is east from UTC (Tom)
•
Fix date_trunc(’quarter’, ...) (Böjthe Zoltán) Prior releases returned an incorrect value for this function call.
•
Make initcap() more compatible with Oracle (Mike Nolan) initcap() now uppercases a letter appearing after any non-alphanumeric character, rather than
only after whitespace. •
Allow only datestyle field order for date values not in ISO-8601 format (Greg)
•
Add new datestyle values MDY, DMY, and YMD to set input field order; honor US and European for backward compatibility (Tom)
•
String literals like ’now’ or ’today’ will no longer work as a column default. Use functions such as now(), current_timestamp instead. (change required for prepared statements) (Tom)
•
Treat NaN as larger than any other value in min()/max() (Tom) NaN was already sorted after ordinary numeric values for most purposes, but min() and max() didn’t get this right.
2575
Appendix E. Release Notes •
Prevent interval from suppressing :00 seconds display
•
New functions pg_get_triggerdef(prettyprint) and pg_conversion_is_visible() (Christopher)
•
Allow time to be specified as 040506 or 0405 (Tom)
•
Input date order must now be YYYY-MM-DD (with 4-digit year) or match datestyle
•
Make pg_get_constraintdef support unique, primary-key, and check constraints (Christopher)
E.246.3.8. Server-Side Language Changes •
Prevent PL/pgSQL crash when RETURN NEXT is used on a zero-row record variable (Tom)
•
Make PL/Python’s spi_execute interface handle null values properly (Andrew Bosma)
•
Allow PL/pgSQL to declare variables of composite types without %ROWTYPE (Tom)
•
Fix PL/Python’s _quote() function to handle big integers
•
Make PL/Python an untrusted language, now called plpythonu (Kevin Jacobs, Tom) The Python language no longer supports a restricted execution environment, so the trusted version of PL/Python was removed. If this situation changes, a version of PL/Python that can be used by non-superusers will be readded.
•
Allow polymorphic PL/pgSQL functions (Joe, Tom)
•
Allow polymorphic SQL functions (Joe)
•
Improved compiled function caching mechanism in PL/pgSQL with full support for polymorphism (Joe)
•
Add new parameter $0 in PL/pgSQL representing the function’s actual return type (Joe)
•
Allow PL/Tcl and PL/Python to use the same trigger on multiple tables (Tom)
•
Fixed PL/Tcl’s spi_prepare to accept fully qualified type names in the parameter type list (Jan)
E.246.3.9. psql Changes •
Add \pset pager always to always use pager (Greg) This forces the pager to be used even if the number of rows is less than the screen height. This is valuable for rows that wrap across several screen rows.
•
Improve tab completion (Rod, Ross Reedstrom, Ian Barwick)
•
Reorder \? help into groupings (Harald Armin Massa, Bruce)
•
Add backslash commands for listing schemas, casts, and conversions (Christopher)
• \encoding
now changes based on the server parameter client_encoding (Tom)
In previous versions, \encoding was not aware of encoding changes made using SET client_encoding. •
Save editor buffer into readline history (Ross) When \e is used to edit a query, the result is saved in the readline history for retrieval using the up arrow.
2576
Appendix E. Release Notes •
Improve \d display (Christopher)
•
Enhance HTML mode to be more standards-conforming (Greg)
•
New \set AUTOCOMMIT off capability (Tom) This takes the place of the removed server parameter autocommit.
•
New \set VERBOSITY to control error detail (Tom) This controls the new error reporting details.
•
New prompt escape sequence %x to show transaction status (Tom)
•
Long options for psql are now available on all platforms
E.246.3.10. pg_dump Changes •
Multiple pg_dump fixes, including tar format and large objects
•
Allow pg_dump to dump specific schemas (Neil)
•
Make pg_dump preserve column storage characteristics (Christopher) This preserves ALTER TABLE ... SET STORAGE information.
•
Make pg_dump preserve CLUSTER characteristics (Christopher)
•
Have pg_dumpall use GRANT/REVOKE to dump database-level privileges (Tom)
•
Allow pg_dumpall to support the options -a, -s, -x of pg_dump (Tom)
•
Prevent pg_dump from lowercasing identifiers specified on the command line (Tom)
•
pg_dump options --use-set-session-authorization and --no-reconnect now do nothing, all dumps use SET SESSION AUTHORIZATION pg_dump no longer reconnects to switch users, but instead always uses SET SESSION AUTHORIZATION. This will reduce password prompting during restores.
•
Long options for pg_dump are now available on all platforms PostgreSQL now includes its own long-option processing routines.
E.246.3.11. libpq Changes •
Add function PQfreemem for freeing memory on Windows, suggested for NOTIFY (Bruce) Windows requires that memory allocated in a library be freed by a function in the same library, hence free() doesn’t work for freeing memory allocated by libpq. PQfreemem is the proper way to free libpq memory, especially on Windows, and is recommended for other platforms as well.
•
Document service capability, and add sample file (Bruce) This allows clients to look up connection information in a central file on the client machine.
•
Make PQsetdbLogin have the same defaults as PQconnectdb (Tom)
•
Allow libpq to cleanly fail when result sets are too large (Tom)
•
Improve performance of function PQunescapeBytea (Ben Lamb)
•
Allow thread-safe libpq with configure option --enable-thread-safety (Lee Kindness, Philip Yarra)
2577
Appendix E. Release Notes •
Allow function pqInternalNotice to accept a format string and arguments instead of just a preformatted message (Tom, Sean Chittenden)
•
Control SSL negotiation with sslmode values disable, allow, prefer, and require (Jon Jensen)
•
Allow new error codes and levels of text (Tom)
•
Allow access to the underlying table and column of a query result (Tom) This is helpful for query-builder applications that want to know the underlying table and column names associated with a specific result set.
•
Allow access to the current transaction status (Tom)
•
Add ability to pass binary data directly to the server (Tom)
•
Add function PQexecPrepared and PQsendQueryPrepared functions which perform bind/execute of previously prepared statements (Tom)
E.246.3.12. JDBC Changes •
Allow setNull on updateable result sets
•
Allow executeBatch on a prepared statement (Barry)
•
Support SSL connections (Barry)
•
Handle schema names in result sets (Paul Sorenson)
•
Add refcursor support (Nic Ferrier)
E.246.3.13. Miscellaneous Interface Changes •
Prevent possible memory leak or core dump during libpgtcl shutdown (Tom)
•
Add Informix compatibility to ECPG (Michael) This allows ECPG to process embedded C programs that were written using certain Informix extensions.
•
Add type decimal to ECPG that is fixed length, for Informix (Michael)
•
Allow thread-safe embedded SQL programs with configure option --enable-thread-safety (Lee Kindness, Bruce) This allows multiple threads to access the database at the same time.
•
Moved Python client PyGreSQL to http://www.pygresql.org (Marc)
E.246.3.14. Source Code Changes •
Prevent need for separate platform geometry regression result files (Tom)
•
Improved PPC locking primitive (Reinhard Max)
•
New function palloc0 to allocate and clear memory (Bruce)
•
Fix locking code for s390x CPU (64-bit) (Tom)
2578
Appendix E. Release Notes •
Allow OpenBSD to use local ident credentials (William Ahern)
•
Make query plan trees read-only to executor (Tom)
•
Add Darwin startup scripts (David Wheeler)
•
Allow libpq to compile with Borland C++ compiler (Lester Godwin, Karl Waclawek)
•
Use our own version of getopt_long() if needed (Peter)
•
Convert administration scripts to C (Peter)
•
Bison >= 1.85 is now required to build the PostgreSQL grammar, if building from CVS
•
Merge documentation into one book (Peter)
•
Add Windows compatibility functions (Bruce)
•
Allow client interfaces to compile under MinGW (Bruce)
•
New ereport() function for error reporting (Tom)
•
Support Intel compiler on Linux (Peter)
•
Improve Linux startup scripts (Slawomir Sudnik, Darko Prenosil)
•
Add support for AMD Opteron and Itanium (Jeffrey W. Baker, Bruce)
•
Remove --enable-recode option from configure This was no longer needed now that we have CREATE CONVERSION.
•
Generate a compile error if spinlock code is not found (Bruce) Platforms without spinlock code will now fail to compile, rather than silently using semaphores. This failure can be disabled with a new configure option.
E.246.3.15. Contrib Changes •
Change dbmirror license to BSD
•
Improve earthdistance (Bruno Wolff III)
•
Portability improvements to pgcrypto (Marko Kreen)
•
Prevent crash in xml (John Gray, Michael Richards)
•
Update oracle
•
Update mysql
•
Update cube (Bruno Wolff III)
•
Update earthdistance to use cube (Bruno Wolff III)
•
Update btree_gist (Oleg)
•
New tsearch2 full-text search module (Oleg, Teodor)
•
Add hash-based crosstab function to tablefuncs (Joe)
•
Add serial column to order connectby() siblings in tablefuncs (Nabil Sayegh,Joe)
•
Add named persistent connections to dblink (Shridhar Daithanka)
•
New pg_autovacuum allows automatic VACUUM (Matthew T. O’Connor)
•
Make pgbench honor environment variables PGHOST, PGPORT, PGUSER (Tatsuo)
Remove array module because features now included by default (Joe)
E.247. Release 7.3.21 Release date: 2008-01-07 This release contains a variety of fixes from 7.3.20, including fixes for significant security issues. This is expected to be the last PostgreSQL release in the 7.3.X series. Users are encouraged to update to a newer release branch soon.
E.247.1. Migration to Version 7.3.21 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
E.247.2. Changes •
Prevent functions in indexes from executing with the privileges of the user running VACUUM, ANALYZE, etc (Tom)
Functions used in index expressions and partial-index predicates are evaluated whenever a new table entry is made. It has long been understood that this poses a risk of trojan-horse code execution if one modifies a table owned by an untrustworthy user. (Note that triggers, defaults, check constraints, etc. pose the same type of risk.) But functions in indexes pose extra danger because they will be executed by routine maintenance operations such as VACUUM FULL, which are commonly performed automatically under a superuser account. For example, a nefarious user can execute code with superuser privileges by setting up a trojan-horse index definition and waiting for the next routine vacuum. The fix arranges for standard maintenance operations (including VACUUM, ANALYZE, REINDEX, and CLUSTER) to execute as the table owner rather than the calling user, using the same privilege-switching mechanism already used for SECURITY DEFINER functions. To prevent bypassing this security measure, execution of SET SESSION AUTHORIZATION and SET ROLE is now forbidden within a SECURITY DEFINER context. (CVE-2007-6600) •
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe) The fix that appeared for this in 7.3.20 was incomplete, as it plugged the hole for only some dblink functions. (CVE-2007-6601, CVE-2007-3278)
•
Fix potential crash in translate() when using a multibyte database encoding (Tom)
2580
Appendix E. Release Notes •
Make contrib/tablefunc’s crosstab() handle NULL rowid as a category in its own right, rather than crashing (Joe)
•
Require a specific version of Autoconf to be used when re-generating the configure script (Peter) This affects developers and packagers only. The change was made to prevent accidental use of untested combinations of Autoconf and PostgreSQL versions. You can remove the version check if you really want to use a different Autoconf version, but it’s your responsibility whether the result works or not.
E.248. Release 7.3.20 Release date: 2007-09-17 This release contains fixes from 7.3.19.
E.248.1. Migration to Version 7.3.20 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
E.248.2. Changes •
Prevent index corruption when a transaction inserts rows and then aborts close to the end of a concurrent VACUUM on the same table (Tom)
•
Make CREATE DOMAIN ... DEFAULT NULL work properly (Tom)
•
Fix crash when log_min_error_statement logging runs out of memory (Tom)
•
Require non-superusers who use /contrib/dblink to use only password authentication, as a security measure (Joe)
E.249. Release 7.3.19 Release date: 2007-04-23 This release contains fixes from 7.3.18, including a security fix.
E.249.1. Migration to Version 7.3.19 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
2581
Appendix E. Release Notes
E.249.2. Changes •
Support explicit placement of the temporary-table schema within search_path, and disable searching it for functions and operators (Tom) This is needed to allow a security-definer function to set a truly secure value of search_path. Without it, an unprivileged SQL user can use temporary objects to execute code with the privileges of the security-definer function (CVE-2007-2138). See CREATE FUNCTION for more information.
•
Fix potential-data-corruption bug in how VACUUM FULL handles UPDATE chains (Tom, Pavan Deolasee)
E.250. Release 7.3.18 Release date: 2007-02-05 This release contains a variety of fixes from 7.3.17, including a security fix.
E.250.1. Migration to Version 7.3.18 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
E.250.2. Changes •
Remove security vulnerability that allowed connected users to read backend memory (Tom) The vulnerability involves changing the data type of a table column used in a SQL function (CVE2007-0555). This error can easily be exploited to cause a backend crash, and in principle might be used to read database content that the user should not be able to access.
•
Fix rare bug wherein btree index page splits could fail due to choosing an infeasible split point (Heikki Linnakangas)
•
Tighten security of multi-byte character processing for UTF8 sequences over three bytes long (Tom)
E.251. Release 7.3.17 Release date: 2007-01-08 This release contains a variety of fixes from 7.3.16.
E.251.1. Migration to Version 7.3.17 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
2582
Appendix E. Release Notes
E.251.2. Changes • to_number() and to_char(numeric) are now STABLE, not IMMUTABLE, for new initdb installs
(Tom) This is because lc_numeric can potentially change the output of these functions. •
Improve index usage of regular expressions that use parentheses (Tom) This improves psql \d performance also.
E.252. Release 7.3.16 Release date: 2006-10-16 This release contains a variety of fixes from 7.3.15.
E.252.1. Migration to Version 7.3.16 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
E.252.2. Changes •
Fix corner cases in pattern matching for psql’s \d commands
•
Fix index-corrupting bugs in /contrib/ltree (Teodor)
•
Back-port 7.4 spinlock code to improve performance and support 64-bit architectures better
•
Fix SSL-related memory leak in libpq
•
Fix backslash escaping in /contrib/dbmirror
•
Adjust regression tests for recent changes in US DST laws
E.253. Release 7.3.15 Release date: 2006-05-23 This release contains a variety of fixes from 7.3.14, including patches for extremely serious security issues.
E.253.1. Migration to Version 7.3.15 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255. Full security against the SQL-injection attacks described in CVE-2006-2313 and CVE-2006-2314 might require changes in application code. If you have applications that embed untrustworthy strings
2583
Appendix E. Release Notes into SQL commands, you should examine them as soon as possible to ensure that they are using recommended escaping techniques. In most cases, applications should be using subroutines provided by libraries or drivers (such as libpq’s PQescapeStringConn()) to perform string escaping, rather than relying on ad hoc code to do it.
E.253.2. Changes •
Change the server to reject invalidly-encoded multibyte characters in all cases (Tatsuo, Tom) While PostgreSQL has been moving in this direction for some time, the checks are now applied uniformly to all encodings and all textual input, and are now always errors not merely warnings. This change defends against SQL-injection attacks of the type described in CVE-2006-2313.
•
Reject unsafe uses of \’ in string literals As a server-side defense against SQL-injection attacks of the type described in CVE-2006-2314, the server now only accepts ” and not \’ as a representation of ASCII single quote in SQL string literals. By default, \’ is rejected only when client_encoding is set to a client-only encoding (SJIS, BIG5, GBK, GB18030, or UHC), which is the scenario in which SQL injection is possible. A new configuration parameter backslash_quote is available to adjust this behavior when needed. Note that full security against CVE-2006-2314 might require client-side changes; the purpose of backslash_quote is in part to make it obvious that insecure clients are insecure.
•
Modify libpq’s string-escaping routines to be aware of encoding considerations This fixes libpq-using applications for the security issues described in CVE-2006-2313 and CVE2006-2314. Applications that use multiple PostgreSQL connections concurrently should migrate to PQescapeStringConn() and PQescapeByteaConn() to ensure that escaping is done correctly for the settings in use in each database connection. Applications that do string escaping “by hand” should be modified to rely on library routines instead.
•
Fix some incorrect encoding conversion functions win1251_to_iso, alt_to_iso, euc_tw_to_big5, euc_tw_to_mic, mic_to_euc_tw were
all broken to varying extents. •
Clean up stray remaining uses of \’ in strings (Bruce, Jan)
•
Fix server to use custom DH SSL parameters correctly (Michael Fuhr)
•
Fix various minor memory leaks
E.254. Release 7.3.14 Release date: 2006-02-14 This release contains a variety of fixes from 7.3.13.
E.254.1. Migration to Version 7.3.14 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.13, see Section E.255.
2584
Appendix E. Release Notes
E.254.2. Changes •
Fix potential crash in SET SESSION AUTHORIZATION (CVE-2006-0553) An unprivileged user could crash the server process, resulting in momentary denial of service to other users, if the server has been compiled with Asserts enabled (which is not the default). Thanks to Akio Ishida for reporting this problem.
•
Fix bug with row visibility logic in self-inserted rows (Tom) Under rare circumstances a row inserted by the current command could be seen as already valid, when it should not be. Repairs bug created in 7.3.11 release.
•
Fix race condition that could lead to “file already exists” errors during pg_clog file creation (Tom)
•
Fix to allow restoring dumps that have cross-schema references to custom operators (Tom)
•
Portability fix for testing presence of finite and isinf during configure (Tom)
E.255. Release 7.3.13 Release date: 2006-01-09 This release contains a variety of fixes from 7.3.12.
E.255.1. Migration to Version 7.3.13 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.10, see Section E.258. Also, you might need to REINDEX indexes on textual columns after updating, if you are affected by the locale or plperl issues described below.
E.255.2. Changes •
Fix character string comparison for locales that consider different character combinations as equal, such as Hungarian (Tom) This might require REINDEX to fix existing indexes on textual columns.
•
Set locale environment variables during postmaster startup to ensure that plperl won’t change the locale later This fixes a problem that occurred if the postmaster was started with environment variables specifying a different locale than what initdb had been told. Under these conditions, any use of plperl was likely to lead to corrupt indexes. You might need REINDEX to fix existing indexes on textual columns if this has happened to you.
•
Fix longstanding bug in strpos() and regular expression handling in certain rarely used Asian multibyte character sets (Tatsuo)
•
Fix bug in /contrib/pgcrypto gen_salt, which caused it not to use all available salt space for MD5 and XDES algorithms (Marko Kreen, Solar Designer) Salts for Blowfish and standard DES are unaffected.
2585
Appendix E. Release Notes •
Fix /contrib/dblink to throw an error, rather than crashing, when the number of columns specified is different from what’s actually returned by the query (Joe)
E.256. Release 7.3.12 Release date: 2005-12-12 This release contains a variety of fixes from 7.3.11.
E.256.1. Migration to Version 7.3.12 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.10, see Section E.258.
E.256.2. Changes •
Fix race condition in transaction log management There was a narrow window in which an I/O operation could be initiated for the wrong page, leading to an Assert failure or data corruption.
• /contrib/ltree •
fixes (Teodor)
Fix longstanding planning error for outer joins This bug sometimes caused a bogus error “RIGHT JOIN is only supported with merge-joinable join conditions”.
•
Prevent core dump in pg_autovacuum when a table has been dropped
E.257. Release 7.3.11 Release date: 2005-10-04 This release contains a variety of fixes from 7.3.10.
E.257.1. Migration to Version 7.3.11 A dump/restore is not required for those running 7.3.X. However, if you are upgrading from a version earlier than 7.3.10, see Section E.258.
E.257.2. Changes •
Fix error that allowed VACUUM to remove ctid chains too soon, and add more checking in code that follows ctid links This fixes a long-standing problem that could cause crashes in very rare circumstances.
2586
Appendix E. Release Notes •
Fix CHAR() to properly pad spaces to the specified length when using a multiple-byte character set (Yoshiyuki Asaba) In prior releases, the padding of CHAR() was incorrect because it only padded to the specified number of bytes without considering how many characters were stored.
•
Fix missing rows in queries like UPDATE a=... WHERE a... with GiST index on column a
•
Improve checking for partially-written WAL pages
•
Improve robustness of signal handling when SSL is enabled
•
Various memory leakage fixes
•
Various portability improvements
•
Fix PL/pgSQL to handle var := var correctly when the variable is of pass-by-reference type
E.258. Release 7.3.10 Release date: 2005-05-09 This release contains a variety of fixes from 7.3.9, including several security-related issues.
E.258.1. Migration to Version 7.3.10 A dump/restore is not required for those running 7.3.X. However, it is one possible way of handling a significant security problem that has been found in the initial contents of 7.3.X system catalogs. A dump/initdb/reload sequence using 7.3.10’s initdb will automatically correct this problem. The security problem is that the built-in character set encoding conversion functions can be invoked from SQL commands by unprivileged users, but the functions were not designed for such use and are not secure against malicious choices of arguments. The fix involves changing the declared parameter list of these functions so that they can no longer be invoked from SQL commands. (This does not affect their normal use by the encoding conversion machinery.) It is strongly recommended that all installations repair this error, either by initdb or by following the manual repair procedure given below. The error at least allows unprivileged database users to crash their server process, and might allow unprivileged users to gain the privileges of a database superuser. If you wish not to do an initdb, perform the following procedure instead. As the database superuser, do: BEGIN; UPDATE pg_proc SET proargtypes[3] = ’internal’::regtype WHERE pronamespace = 11 AND pronargs = 5 AND proargtypes[2] = ’cstring’::regtype; -- The command should report having updated 90 rows; -- if not, rollback and investigate instead of committing! COMMIT;
The above procedure must be carried out in each database of an installation, including template1, and ideally including template0 as well. If you do not fix the template databases then any subsequently created databases will contain the same error. template1 can be fixed in the same way as any other database, but fixing template0 requires additional steps. First, from any database issue:
2587
Appendix E. Release Notes UPDATE pg_database SET datallowconn = true WHERE datname = ’template0’;
Next connect to template0 and perform the above repair procedure. Finally, do: -- re-freeze template0: VACUUM FREEZE; -- and protect it against future alterations: UPDATE pg_database SET datallowconn = false WHERE datname = ’template0’;
E.258.2. Changes •
Change encoding function signature to prevent misuse
•
Repair ancient race condition that allowed a transaction to be seen as committed for some purposes (eg SELECT FOR UPDATE) slightly sooner than for other purposes This is an extremely serious bug since it could lead to apparent data inconsistencies being briefly visible to applications.
•
Repair race condition between relation extension and VACUUM This could theoretically have caused loss of a page’s worth of freshly-inserted data, although the scenario seems of very low probability. There are no known cases of it having caused more than an Assert failure.
•
Fix comparisons of TIME WITH TIME ZONE values The comparison code was wrong in the case where the --enable-integer-datetimes configuration switch had been used. NOTE: if you have an index on a TIME WITH TIME ZONE column, it will need to be REINDEXed after installing this update, because the fix corrects the sort order of column values.
•
Fix EXTRACT(EPOCH) for TIME WITH TIME ZONE values
•
Fix mis-display of negative fractional seconds in INTERVAL values This error only occurred when the --enable-integer-datetimes configuration switch had been used.
•
Additional buffer overrun checks in plpgsql (Neil)
•
Fix pg_dump to dump trigger names containing % correctly (Neil)
•
Prevent to_char(interval) from dumping core for month-related formats
•
Fix contrib/pgcrypto for newer OpenSSL builds (Marko Kreen)
•
Still more 64-bit fixes for contrib/intagg
•
Prevent incorrect optimization of functions returning RECORD
E.259. Release 7.3.9 Release date: 2005-01-31 This release contains a variety of fixes from 7.3.8, including several security-related issues.
2588
Appendix E. Release Notes
E.259.1. Migration to Version 7.3.9 A dump/restore is not required for those running 7.3.X.
E.259.2. Changes •
Disallow LOAD to non-superusers On platforms that will automatically execute initialization functions of a shared library (this includes at least Windows and ELF-based Unixen), LOAD can be used to make the server execute arbitrary code. Thanks to NGS Software for reporting this.
•
Check that creator of an aggregate function has the right to execute the specified transition functions This oversight made it possible to bypass denial of EXECUTE permission on a function.
•
Fix security and 64-bit issues in contrib/intagg
•
Add needed STRICT marking to some contrib functions (Kris Jurka)
•
Avoid buffer overrun when plpgsql cursor declaration has too many parameters (Neil)
•
Fix planning error for FULL and RIGHT outer joins The result of the join was mistakenly supposed to be sorted the same as the left input. This could not only deliver mis-sorted output to the user, but in case of nested merge joins could give outright wrong answers.
•
Fix plperl for quote marks in tuple fields
•
Fix display of negative intervals in SQL and GERMAN datestyles
E.260. Release 7.3.8 Release date: 2004-10-22 This release contains a variety of fixes from 7.3.7.
E.260.1. Migration to Version 7.3.8 A dump/restore is not required for those running 7.3.X.
E.260.2. Changes •
Repair possible failure to update hint bits on disk Under rare circumstances this oversight could lead to “could not access transaction status” failures, which qualifies it as a potential-data-loss bug.
•
Ensure that hashed outer join does not miss tuples Very large left joins using a hash join plan could fail to output unmatched left-side rows given just the right data distribution.
2589
Appendix E. Release Notes •
Disallow running pg_ctl as root This is to guard against any possible security issues.
•
Avoid using temp files in /tmp in make_oidjoins_check This has been reported as a security issue, though it’s hardly worthy of concern since there is no reason for non-developers to use this script anyway.
E.261. Release 7.3.7 Release date: 2004-08-16 This release contains one critical fix over 7.3.6, and some minor items.
E.261.1. Migration to Version 7.3.7 A dump/restore is not required for those running 7.3.X.
E.261.2. Changes •
Prevent possible loss of committed transactions during crash Due to insufficient interlocking between transaction commit and checkpointing, it was possible for transactions committed just before the most recent checkpoint to be lost, in whole or in part, following a database crash and restart. This is a serious bug that has existed since PostgreSQL 7.1.
•
Remove asymmetrical word processing in tsearch (Teodor)
•
Properly schema-qualify function names when pg_dump’ing a CAST
E.262. Release 7.3.6 Release date: 2004-03-02 This release contains a variety of fixes from 7.3.5.
E.262.1. Migration to Version 7.3.6 A dump/restore is not required for those running 7.3.*.
E.262.2. Changes •
Revert erroneous changes in rule permissions checking A patch applied in 7.3.3 to fix a corner case in rule permissions checks turns out to have disabled rule-related permissions checks in many not-so-corner cases. This would for example allow users
2590
Appendix E. Release Notes to insert into views they weren’t supposed to have permission to insert into. We have therefore reverted the 7.3.3 patch. The original bug will be fixed in 8.0. •
Repair incorrect order of operations in GetNewTransactionId() This bug could result in failure under out-of-disk-space conditions, including inability to restart even after disk space is freed.
•
Ensure configure selects -fno-strict-aliasing even when an external value for CFLAGS is supplied On some platforms, building with -fstrict-aliasing causes bugs.
•
Make pg_restore handle 64-bit off_t correctly This bug prevented proper restoration from archive files exceeding 4 GB.
•
Make contrib/dblink not assume that local and remote type OIDs match (Joe)
Fix UPDATE when child table’s column numbering differs from parent
•
Increase default value of max_fsm_relations
•
Fix problem when fetching backwards in a cursor for a single-row query
•
Make backward fetch work properly with cursor on SELECT DISTINCT query
•
Fix problems with loading pg_dump files containing contrib/lo usage
•
Fix problem with all-numeric user names
•
Fix possible memory leak and core dump during disconnect in libpgtcl
•
Make plpython’s spi_execute command handle nulls properly (Andrew Bosma)
•
Adjust plpython error reporting so that its regression test passes again
•
Work with bison 1.875
•
Handle mixed-case names properly in plpgsql’s %type (Neil)
•
Fix core dump in pltcl when executing a query rewritten by a rule
•
Repair array subscript overruns (per report from Yichen Xie)
•
Reduce MAX_TIME_PRECISION from 13 to 10 in floating-point case
•
Correctly case-fold variable names in per-database and per-user settings
•
Fix coredump in plpgsql’s RETURN NEXT when SELECT into record returns no rows
•
Fix outdated use of pg_type.typprtlen in python client interface
•
Correctly handle fractional seconds in timestamps in JDBC driver
•
Improve performance of getImportedKeys() in JDBC
•
Make shared-library symlinks work standardly on HPUX (Giles)
•
Repair inconsistent rounding behavior for timestamp, time, interval
2595
Appendix E. Release Notes •
SSL negotiation fixes (Nathan Mueller)
•
Make libpq’s ~/.pgpass feature work when connecting with PQconnectDB
•
Update my2pg, ora2pg
•
Translation updates
•
Add casts between types lo and oid in contrib/lo
•
fastpath code now checks for privilege to call function
E.267. Release 7.3.1 Release date: 2002-12-18 This release contains a variety of fixes for version 7.3.
E.267.1. Migration to Version 7.3.1 A dump/restore is not required for those running version 7.3. However, it should be noted that the main PostgreSQL interface library, libpq, has a new major version number for this release, which might require recompilation of client code in certain cases.
E.267.2. Changes •
Fix a core dump of COPY TO when client/server encodings don’t match (Tom)
•
Allow pg_dump to work with pre-7.2 servers (Philip)
•
contrib/adddepend fixes (Tom)
•
Fix problem with deletion of per-user/per-database config settings (Tom)
•
contrib/vacuumlo fix (Tom)
•
Allow ’password’ encryption even when pg_shadow contains MD5 passwords (Bruce)
•
contrib/dbmirror fix (Steven Singer)
•
Optimizer fixes (Tom)
•
contrib/tsearch fixes (Teodor Sigaev, Magnus)
•
Allow locale names to be mixed case (Nicolai Tufar)
•
Increment libpq library’s major version number (Bruce)
•
pg_hba.conf error reporting fixes (Bruce, Neil)
•
Add SCO Openserver 5.0.4 as a supported platform (Bruce)
•
Prevent EXPLAIN from crashing server (Tom)
•
SSL fixes (Nathan Mueller)
•
Prevent composite column creation via ALTER TABLE (Tom)
2596
Appendix E. Release Notes
E.268. Release 7.3 Release date: 2002-11-27
E.268.1. Overview Major changes in this release: Schemas Schemas allow users to create objects in separate namespaces, so two people or applications can have tables with the same name. There is also a public schema for shared tables. Table/index creation can be restricted by removing privileges on the public schema. Drop Column PostgreSQL now supports the ALTER TABLE ... DROP COLUMN functionality. Table Functions Functions returning multiple rows and/or multiple columns are now much easier to use than before. You can call such a “table function” in the SELECT FROM clause, treating its output like a table. Also, PL/pgSQL functions can now return sets. Prepared Queries PostgreSQL now supports prepared queries, for improved performance. Dependency Tracking PostgreSQL now records object dependencies, which allows improvements in many areas. DROP statements now take either CASCADE or RESTRICT to control whether dependent objects are also dropped. Privileges Functions and procedural languages now have privileges, and functions can be defined to run with the privileges of their creator. Internationalization Both multibyte and locale support are now always enabled. Logging A variety of logging options have been enhanced. Interfaces A large number of interfaces have been moved to http://gborg.postgresql.org where they can be developed and released independently. Functions/Identifiers By default, functions can now take up to 32 parameters, and identifiers can be up to 63 bytes long. Also, OPAQUE is now deprecated: there are specific “pseudo-datatypes” to represent each of the former meanings of OPAQUE in function argument and result types.
E.268.2. Migration to Version 7.3 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. If your application examines the system catalogs, additional changes
2597
Appendix E. Release Notes will be required due to the introduction of schemas in 7.3; for more information, see: http://developer.postgresql.org/~momjian/upgrade_tips_7.3. Observe the following incompatibilities: •
Pre-6.3 clients are no longer supported.
• pg_hba.conf
now has a column for the user name and additional features. Existing files need to
be adjusted. •
Several postgresql.conf logging parameters have been renamed.
• LIMIT #,#
has been disabled; use LIMIT # OFFSET #.
statements with column lists must specify a value for each specified column. For example, INSERT INTO tab (col1, col2) VALUES (’val1’) is now invalid. It’s still allowed to supply fewer columns than expected if the INSERT does not have a column list.
• INSERT
• serial
columns are no longer automatically UNIQUE; thus, an index will not automatically be
created. •
A SET command inside an aborted transaction is now rolled back. no longer considers missing trailing columns to be null. All columns need to be specified. (However, one can achieve a similar effect by specifying a column list in the COPY command.)
• COPY
•
The data type timestamp is now equivalent to timestamp without time zone, instead of timestamp with time zone.
•
Pre-7.3 databases loaded into 7.3 will not have the new object dependencies for serial columns, unique constraints, and foreign keys. See the directory contrib/adddepend/ for a detailed description and a script that will add such dependencies.
•
An empty string (”) is no longer allowed as the input into an integer field. Formerly, it was silently interpreted as 0.
E.268.3. Changes E.268.3.1. Server Operation •
Add pg_locks view to show locks (Neil)
•
Security fixes for password negotiation memory allocation (Neil)
•
Remove support for version 0 FE/BE protocol (PostgreSQL 6.2 and earlier) (Tom)
•
Reserve the last few backend slots for superusers, add parameter superuser_reserved_connections to control this (Nigel J. Andrews)
E.268.3.2. Performance •
Improve startup by calling localtime() only once (Tom)
•
Cache system catalog information in flat files for faster startup (Tom)
•
Improve caching of index information (Tom)
•
Optimizer improvements (Tom, Fernando Nasser)
2598
Appendix E. Release Notes •
Catalog caches now store failed lookups (Tom)
•
Hash function improvements (Neil)
•
Improve performance of query tokenization and network handling (Peter)
•
Speed improvement for large object restore (Mario Weilguni)
•
Mark expired index entries on first lookup, saving later heap fetches (Tom)
Improve options supported by /contrib/vacuumlo (Mario Weilguni)
•
Improvements to /contrib/intarray (Oleg, Teodor Sigaev, Andrey Oktyabrski)
•
Add /contrib/reindexdb utility (Shaun Thomas)
•
Add indexing to /contrib/isbn_issn (Dan Weston)
•
Add /contrib/dbmirror (Steven Singer)
•
Improve /contrib/pgbench (Neil)
•
Add /contrib/tablefunc table function examples (Joe)
•
Add /contrib/ltree data type for tree structures (Teodor Sigaev, Oleg Bartunov)
•
Move /contrib/pg_controldata, pg_resetxlog into main tree (Bruce)
•
Fixes to /contrib/cube (Bruno Wolff)
2607
Appendix E. Release Notes •
Improve /contrib/fulltextindex (Christopher)
E.269. Release 7.2.8 Release date: 2005-05-09 This release contains a variety of fixes from 7.2.7, including one security-related issue.
E.269.1. Migration to Version 7.2.8 A dump/restore is not required for those running 7.2.X.
E.269.2. Changes •
Repair ancient race condition that allowed a transaction to be seen as committed for some purposes (eg SELECT FOR UPDATE) slightly sooner than for other purposes This is an extremely serious bug since it could lead to apparent data inconsistencies being briefly visible to applications.
•
Repair race condition between relation extension and VACUUM This could theoretically have caused loss of a page’s worth of freshly-inserted data, although the scenario seems of very low probability. There are no known cases of it having caused more than an Assert failure.
•
Fix EXTRACT(EPOCH) for TIME WITH TIME ZONE values
•
Additional buffer overrun checks in plpgsql (Neil)
•
Fix pg_dump to dump index names and trigger names containing % correctly (Neil)
•
Prevent to_char(interval) from dumping core for month-related formats
•
Fix contrib/pgcrypto for newer OpenSSL builds (Marko Kreen)
E.270. Release 7.2.7 Release date: 2005-01-31 This release contains a variety of fixes from 7.2.6, including several security-related issues.
E.270.1. Migration to Version 7.2.7 A dump/restore is not required for those running 7.2.X.
2608
Appendix E. Release Notes
E.270.2. Changes •
Disallow LOAD to non-superusers On platforms that will automatically execute initialization functions of a shared library (this includes at least Windows and ELF-based Unixen), LOAD can be used to make the server execute arbitrary code. Thanks to NGS Software for reporting this.
•
Add needed STRICT marking to some contrib functions (Kris Jurka)
•
Avoid buffer overrun when plpgsql cursor declaration has too many parameters (Neil)
•
Fix planning error for FULL and RIGHT outer joins The result of the join was mistakenly supposed to be sorted the same as the left input. This could not only deliver mis-sorted output to the user, but in case of nested merge joins could give outright wrong answers.
•
Fix display of negative intervals in SQL and GERMAN datestyles
E.271. Release 7.2.6 Release date: 2004-10-22 This release contains a variety of fixes from 7.2.5.
E.271.1. Migration to Version 7.2.6 A dump/restore is not required for those running 7.2.X.
E.271.2. Changes •
Repair possible failure to update hint bits on disk Under rare circumstances this oversight could lead to “could not access transaction status” failures, which qualifies it as a potential-data-loss bug.
•
Ensure that hashed outer join does not miss tuples Very large left joins using a hash join plan could fail to output unmatched left-side rows given just the right data distribution.
•
Disallow running pg_ctl as root This is to guard against any possible security issues.
•
Avoid using temp files in /tmp in make_oidjoins_check This has been reported as a security issue, though it’s hardly worthy of concern since there is no reason for non-developers to use this script anyway.
•
Update to newer versions of Bison
2609
Appendix E. Release Notes
E.272. Release 7.2.5 Release date: 2004-08-16 This release contains a variety of fixes from 7.2.4.
E.272.1. Migration to Version 7.2.5 A dump/restore is not required for those running 7.2.X.
E.272.2. Changes •
Prevent possible loss of committed transactions during crash Due to insufficient interlocking between transaction commit and checkpointing, it was possible for transactions committed just before the most recent checkpoint to be lost, in whole or in part, following a database crash and restart. This is a serious bug that has existed since PostgreSQL 7.1.
•
Fix corner case for btree search in parallel with first root page split
•
Fix buffer overrun in to_ascii (Guido Notari)
•
Fix core dump in deadlock detection on machines where char is unsigned
•
Fix failure to respond to pg_ctl stop -m fast after Async_NotifyHandler runs
•
Repair memory leaks in pg_dump
•
Avoid conflict with system definition of isblank() function or macro
E.273. Release 7.2.4 Release date: 2003-01-30 This release contains a variety of fixes for version 7.2.3, including fixes to prevent possible data loss.
E.273.1. Migration to Version 7.2.4 A dump/restore is not required for those running version 7.2.*.
E.273.2. Changes •
Fix some additional cases of VACUUM "No one parent tuple was found" error
•
Prevent VACUUM from being called inside a function (Bruce)
•
Ensure pg_clog updates are sync’d to disk before marking checkpoint complete
•
Avoid integer overflow during large hash joins
•
Make GROUP commands work when pg_group.grolist is large enough to be toasted
•
Fix errors in datetime tables; some timezone names weren’t being recognized
2610
Appendix E. Release Notes •
Fix integer overflows in circle_poly(), path_encode(), path_add() (Neil)
•
Repair long-standing logic errors in lseg_eq(), lseg_ne(), lseg_center()
E.274. Release 7.2.3 Release date: 2002-10-01 This release contains a variety of fixes for version 7.2.2, including fixes to prevent possible data loss.
E.274.1. Migration to Version 7.2.3 A dump/restore is not required for those running version 7.2.*.
E.274.2. Changes •
Prevent possible compressed transaction log loss (Tom)
•
Prevent non-superuser from increasing most recent vacuum info (Tom)
•
Handle pre-1970 date values in newer versions of glibc (Tom)
•
Fix possible hang during server shutdown
•
Prevent spinlock hangs on SMP PPC machines (Tomoyuki Niijima)
•
Fix pg_dump to properly dump FULL JOIN USING (Tom)
E.275. Release 7.2.2 Release date: 2002-08-23 This release contains a variety of fixes for version 7.2.1.
E.275.1. Migration to Version 7.2.2 A dump/restore is not required for those running version 7.2.*.
E.275.2. Changes •
Allow EXECUTE of "CREATE TABLE AS ... SELECT" in PL/pgSQL (Tom)
•
Fix for compressed transaction log id wraparound (Tom)
•
Fix PQescapeBytea/PQunescapeBytea so that they handle bytes > 0x7f (Tatsuo)
•
Fix for psql and pg_dump crashing when invoked with non-existent long options (Tatsuo)
•
Fix crash when invoking geometric operators (Tom)
2611
Appendix E. Release Notes •
Allow OPEN cursor(args) (Tom)
•
Fix for rtree_gist index build (Teodor)
•
Fix for dumping user-defined aggregates (Tom)
•
contrib/intarray fixes (Oleg)
•
Fix for complex UNION/EXCEPT/INTERSECT queries using parens (Tom)
•
Fix to pg_convert (Tatsuo)
•
Fix for crash with long DATA strings (Thomas, Neil)
•
Fix for repeat(), lpad(), rpad() and long strings (Neil)
E.276. Release 7.2.1 Release date: 2002-03-21 This release contains a variety of fixes for version 7.2.
E.276.1. Migration to Version 7.2.1 A dump/restore is not required for those running version 7.2.
E.276.2. Changes •
Ensure that sequence counters do not go backwards after a crash (Tom)
Additional time zones and time zone fixes (Thomas)
•
Allow psql \connect to handle mixed case database and user names (Tom)
•
Return proper OID on command completion even with ON INSERT rules (Tom)
•
Allow COPY FROM to use 8-bit DELIMITERS (Tatsuo)
•
Fix bug in extract/date_part for milliseconds/microseconds (Tatsuo)
•
Improve handling of multiple UNIONs with different lengths (Tom)
•
contrib/btree_gist improvements (Teodor Sigaev)
•
contrib/tsearch dictionary improvements, see README.tsearch for an additional installation step (Thomas T. Thai, Teodor Sigaev)
•
Fix for array subscripts handling (Tom)
•
Allow EXECUTE of "CREATE TABLE AS ... SELECT" in PL/pgSQL (Tom)
2612
Appendix E. Release Notes
E.277. Release 7.2 Release date: 2002-02-04
E.277.1. Overview This release improves PostgreSQL for use in high-volume applications. Major changes in this release: VACUUM Vacuuming no longer locks tables, thus allowing normal user access during the vacuum. A new VACUUM FULL command does old-style vacuum by locking the table and shrinking the on-disk copy of the table. Transactions There is no longer a problem with installations that exceed four billion transactions. OIDs OIDs are now optional. Users can now create tables without OIDs for cases where OID usage is excessive. Optimizer The system now computes histogram column statistics during ANALYZE, allowing much better optimizer choices. Security A new MD5 encryption option allows more secure storage and transfer of passwords. A new Unix-domain socket authentication option is available on Linux and BSD systems. Statistics Administrators can use the new table access statistics module to get fine-grained information about table and index usage. Internationalization Program and library messages can now be displayed in several languages.
E.277.2. Migration to Version 7.2 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release. Observe the following incompatibilities: •
The semantics of the VACUUM command have changed in this release. You might wish to update your maintenance procedures accordingly.
•
In this release, comparisons using = NULL will always return false (or NULL, more precisely). Previous releases automatically transformed this syntax to IS NULL. The old behavior can be reenabled using a postgresql.conf parameter.
•
The pg_hba.conf and pg_ident.conf configuration is now only reloaded after receiving a SIGHUP signal, not with each connection.
2613
Appendix E. Release Notes •
The function octet_length() now returns the uncompressed data length.
•
The date/time value ’current’ is no longer available. You will need to rewrite your applications.
•
The timestamp(), time(), and interval() functions are no longer available. Instead of timestamp(), use timestamp ’string’ or CAST.
The SELECT ... LIMIT #,# syntax will be removed in the next release. You should change your queries to use separate LIMIT and OFFSET clauses, e.g. LIMIT 10 OFFSET 20.
E.277.3. Changes E.277.3.1. Server Operation •
Create temporary files in a separate directory (Bruce)
•
Delete orphaned temporary files on postmaster startup (Bruce)
•
Added unique indexes to some system tables (Tom)
•
System table operator reorganization (Oleg Bartunov, Teodor Sigaev, Tom)
•
Renamed pg_log to pg_clog (Tom)
•
Enable SIGTERM, SIGQUIT to kill backends (Jan)
•
Removed compile-time limit on number of backends (Tom)
•
Better cleanup for semaphore resource failure (Tatsuo, Tom)
•
Allow safe transaction ID wraparound (Tom)
•
Removed OIDs from some system tables (Tom)
•
Removed "triggered data change violation" error check (Tom)
•
SPI portal creation of prepared/saved plans (Jan)
•
Allow SPI column functions to work for system columns (Tom)
•
Long value compression improvement (Tom)
•
Statistics collector for table, index access (Jan)
•
Truncate extra-long sequence names to a reasonable value (Tom)
•
Measure transaction times in milliseconds (Thomas)
•
Fix TID sequential scans (Hiroshi)
•
Superuser ID now fixed at 1 (Peter E)
•
New pg_ctl "reload" option (Tom)
E.277.3.2. Performance •
Optimizer improvements (Tom)
•
New histogram column statistics for optimizer (Tom)
•
Reuse write-ahead log files rather than discarding them (Tom)
•
Cache improvements (Tom)
2614
Appendix E. Release Notes •
IS NULL, IS NOT NULL optimizer improvement (Tom)
•
Improve lock manager to reduce lock contention (Tom)
•
Keep relcache entries for index access support functions (Tom)
•
Allow better selectivity with NaN and infinities in NUMERIC (Tom)
•
R-tree performance improvements (Kenneth Been)
•
B-tree splits more efficient (Tom)
E.277.3.3. Privileges •
Change UPDATE, DELETE privileges to be distinct (Peter E)
•
New REFERENCES, TRIGGER privileges (Peter E)
•
Allow GRANT/REVOKE to/from more than one user at a time (Peter E)
•
New has_table_privilege() function (Joe Conway)
•
Allow non-superuser to vacuum database (Tom)
•
New SET SESSION AUTHORIZATION command (Peter E)
•
Fix bug in privilege modifications on newly created tables (Tom)
•
Disallow access to pg_statistic for non-superuser, add user-accessible views (Tom)
E.277.3.4. Client Authentication •
Fork postmaster before doing authentication to prevent hangs (Peter E)
•
Add ident authentication over Unix domain sockets on Linux, *BSD (Helge Bahmann, Oliver Elphick, Teodor Sigaev, Bruce)
•
Add a password authentication method that uses MD5 encryption (Bruce)
•
Allow encryption of stored passwords using MD5 (Bruce)
•
PAM authentication (Dominic J. Eidson)
•
Load pg_hba.conf and pg_ident.conf only on startup and SIGHUP (Bruce)
E.277.3.5. Server Configuration •
Interpretation of some time zone abbreviations as Australian rather than North American now settable at run time (Bruce)
•
New parameter to set default transaction isolation level (Peter E)
•
New parameter to enable conversion of "expr = NULL" into "expr IS NULL", off by default (Peter E)
•
New parameter to control memory usage by VACUUM (Tom)
•
New parameter to set client authentication timeout (Tom)
•
New parameter to set maximum number of open files (Tom)
2615
Appendix E. Release Notes
E.277.3.6. Queries •
Statements added by INSERT rules now execute after the INSERT (Jan)
•
Prevent unadorned relation names in target list (Bruce)
•
NULLs now sort after all normal values in ORDER BY (Tom)
•
New IS UNKNOWN, IS NOT UNKNOWN Boolean tests (Tom)
•
New SHARE UPDATE EXCLUSIVE lock mode (Tom)
•
New EXPLAIN ANALYZE command that shows run times and row counts (Martijn van Oosterhout)
•
Fix problem with LIMIT and subqueries (Tom)
•
Fix for LIMIT, DISTINCT ON pushed into subqueries (Tom)
•
Fix nested EXCEPT/INTERSECT (Tom)
E.277.3.7. Schema Manipulation •
Fix SERIAL in temporary tables (Bruce)
•
Allow temporary sequences (Bruce)
•
Sequences now use int8 internally (Tom)
•
New SERIAL8 creates int8 columns with sequences, default still SERIAL4 (Tom)
•
Make OIDs optional using WITHOUT OIDS (Tom)
•
Add %TYPE syntax to CREATE TYPE (Ian Lance Taylor)
•
Add ALTER TABLE / DROP CONSTRAINT for CHECK constraints (Christopher Kings-Lynne)
•
New CREATE OR REPLACE FUNCTION to alter existing function (preserving the function OID) (Gavin Sherry)
Make ALTER TABLE / RENAME COLUMN update column names of indexes (Brent Verner)
•
Fix for ALTER TABLE / ADD CONSTRAINT ... CHECK with inherited tables (Stephan Szabo)
•
ALTER TABLE RENAME update foreign-key trigger arguments correctly (Brent Verner)
•
DROP AGGREGATE and COMMENT ON AGGREGATE now accept an aggtype (Tom)
•
Add automatic return type data casting for SQL functions (Tom)
•
Allow GiST indexes to handle NULLs and multikey indexes (Oleg Bartunov, Teodor Sigaev, Tom)
•
Enable partial indexes (Martijn van Oosterhout)
E.277.3.8. Utility Commands •
Add RESET ALL, SHOW ALL (Marko Kreen)
•
CREATE/ALTER USER/GROUP now allow options in any order (Vince)
2616
Appendix E. Release Notes •
Add LOCK A, B, C functionality (Neil Padgett)
•
New ENCRYPTED/UNENCRYPTED option to CREATE/ALTER USER (Bruce)
•
New light-weight VACUUM does not lock table; old semantics are available as VACUUM FULL (Tom)
•
Disable COPY TO/FROM on views (Bruce)
•
COPY DELIMITERS string must be exactly one character (Tom)
•
VACUUM warning about index tuples fewer than heap now only appears when appropriate (Martijn van Oosterhout)
•
Fix privilege checks for CREATE INDEX (Tom)
•
Disallow inappropriate use of CREATE/DROP INDEX/TRIGGER/VIEW (Tom)
E.277.3.9. Data Types and Functions •
SUM(), AVG(), COUNT() now uses int8 internally for speed (Tom)
•
Add convert(), convert2() (Tatsuo)
•
New function bit_length() (Peter E)
•
Make the "n" in CHAR(n)/VARCHAR(n) represents letters, not bytes (Tatsuo)
•
CHAR(), VARCHAR() now reject strings that are too long (Peter E)
•
BIT VARYING now rejects bit strings that are too long (Peter E)
•
BIT now rejects bit strings that do not match declared size (Peter E)
•
INET, CIDR text conversion functions (Alex Pilosov)
•
INET, CIDR operators 1) (Tom) Fix for "f1 datetime DEFAULT ’now’" (Tom) Fix problems with CURRENT_DATE used in DEFAULT (Tom) Allow comment-only lines, and ;;; lines too. (Tom) Improve recovery after failed disk writes, disk full (Hiroshi) Fix cases where table is mentioned in FROM but not joined (Tom) Allow HAVING clause without aggregate functions (Tom) Fix for "--" comment and no trailing newline, as seen in perl interface Improve pg_dump failure error reports (Bruce) Allow sorts and hashes to exceed 2GB file sizes (Tom) Fix for pg_dump dumping of inherited rules (Tom) Fix for NULL handling comparisons (Tom) Fix inconsistent state caused by failed CREATE/DROP commands (Hiroshi) Fix for dbname with dash Prevent DROP INDEX from interfering with other backends (Tom) Fix file descriptor leak in verify_password() Fix for "Unable to identify an operator =$" problem Fix ODBC so no segfault if CommLog and Debug enabled (Dirk Niggemann) Fix for recursive exit call (Massimo) Fix for extra-long timezones (Jeroen van Vianen) Make pg_dump preserve primary key information (Peter E) Prevent databases with single quotes (Peter E) Prevent DROP DATABASE inside transaction (Peter E) ecpg memory leak fixes (Stephen Birch) Fix for SELECT null::text, SELECT int4fac(null) and SELECT 2 + (null) (Tom) Y2K timestamp fix (Massimo) Fix for VACUUM ’HEAP_MOVED_IN was not expected’ errors (Tom) Fix for views with tables/columns containing spaces (Tom) Prevent privileges on indexes (Peter E) Fix for spinlock stuck problem when error is generated (Hiroshi) Fix ipcclean on Linux Fix handling of NULL constraint conditions (Tom) Fix memory leak in odbc driver (Nick Gorham) Fix for privilege check on UNION tables (Tom) Fix to allow SELECT ’a’ LIKE ’a’ (Tom) Fix for SELECT 1 + NULL (Tom) Fixes to CHAR Fix log() on numeric type (Tom) Deprecate ’:’ and ’;’ operators Allow vacuum of temporary tables Disallow inherited columns with the same name as new columns Recover or force failure when disk space is exhausted (Hiroshi) Fix INSERT INTO ... SELECT with AS columns matching result columns Fix INSERT ... SELECT ... GROUP BY groups by target columns not source columns (Tom) Fix CREATE TABLE test (a char(5) DEFAULT text ”, b int4) with INSERT (Tom)
2632
Appendix E. Release Notes Fix UNION with LIMIT Fix CREATE TABLE x AS SELECT 1 UNION SELECT 2 Fix CREATE TABLE test(col char(2) DEFAULT user) Fix mismatched types in CREATE TABLE ... DEFAULT Fix SELECT * FROM pg_class where oid in (0,-1) Fix SELECT COUNT(’asdf’) FROM pg_class WHERE oid=12 Prevent user who can create databases can modifying pg_database table (Peter E) Fix btree to give a useful elog when key > 1/2 (page - overhead) (Tom) Fix INSERT of 0.0 into DECIMAL(4,4) field (Tom) Enhancements -----------New CLI interface include file sqlcli.h, based on SQL3/SQL98 Remove all limits on query length, row length limit still exists (Tom) Update jdbc protocol to 2.0 (Jens Glaser ) Add TRUNCATE command to quickly truncate relation (Mike Mascari) Fix to give super user and createdb user proper update catalog rights (Peter E) Allow ecpg bool variables to have NULL values (Christof) Issue ecpg error if NULL value for variable with no NULL indicator (Christof) Allow ^C to cancel COPY command (Massimo) Add SET FSYNC and SHOW PG_OPTIONS commands(Massimo) Function name overloading for dynamically-loaded C functions (Frankpitt) Add CmdTuples() to libpq++(Vince) New CREATE CONSTRAINT TRIGGER and SET CONSTRAINTS commands(Jan) Allow CREATE FUNCTION/WITH clause to be used for all language types configure --enable-debug adds -g (Peter E) configure --disable-debug removes -g (Peter E) Allow more complex default expressions (Tom) First real FOREIGN KEY constraint trigger functionality (Jan) Add FOREIGN KEY ... MATCH FULL ... ON DELETE CASCADE (Jan) Add FOREIGN KEY ... MATCH referential actions (Don Baccus) Allow WHERE restriction on ctid (physical heap location) (Hiroshi) Move pginterface from contrib to interface directory, rename to pgeasy (Bruce) Change pgeasy connectdb() parameter ordering (Bruce) Require SELECT DISTINCT target list to have all ORDER BY columns (Tom) Add Oracle’s COMMENT ON command (Mike Mascari ) libpq’s PQsetNoticeProcessor function now returns previous hook(Peter E) Prevent PQsetNoticeProcessor from being set to NULL (Peter E) Make USING in COPY optional (Bruce) Allow subselects in the target list (Tom) Allow subselects on the left side of comparison operators (Tom) New parallel regression test (Jan) Change backend-side COPY to write files with permissions 644 not 666 (Tom) Force permissions on PGDATA directory to be secure, even if it exists (Tom) Added psql LASTOID variable to return last inserted oid (Peter E) Allow concurrent vacuum and remove pg_vlock vacuum lock file (Tom) Add privilege check for vacuum (Peter E) New libpq functions to allow asynchronous connections: PQconnectStart(), PQconnectPoll(), PQresetStart(), PQresetPoll(), PQsetenvStart(), PQsetenvPoll(), PQsetenvAbort (Ewan Mellor) New libpq PQsetenv() function (Ewan Mellor) create/alter user extension (Peter E) New postmaster.pid and postmaster.opts under $PGDATA (Tatsuo) New scripts for create/drop user/db (Peter E) Major psql overhaul (Peter E) Add const to libpq interface (Peter E) New libpq function PQoidValue (Peter E)
2633
Appendix E. Release Notes Show specific non-aggregate causing problem with GROUP BY (Tom) Make changes to pg_shadow recreate pg_pwd file (Peter E) Add aggregate(DISTINCT ...) (Tom) Allow flag to control COPY input/output of NULLs (Peter E) Make postgres user have a password by default (Peter E) Add CREATE/ALTER/DROP GROUP (Peter E) All administration scripts now support --long options (Peter E, Karel) Vacuumdb script now supports --all option (Peter E) ecpg new portable FETCH syntax Add ecpg EXEC SQL IFDEF, EXEC SQL IFNDEF, EXEC SQL ELSE, EXEC SQL ELIF and EXEC SQL ENDIF directives Add pg_ctl script to control backend start-up (Tatsuo) Add postmaster.opts.default file to store start-up flags (Tatsuo) Allow --with-mb=SQL_ASCII Increase maximum number of index keys to 16 (Bruce) Increase maximum number of function arguments to 16 (Bruce) Allow configuration of maximum number of index keys and arguments (Bruce) Allow unprivileged users to change their passwords (Peter E) Password authentication enabled; required for new users (Peter E) Disallow dropping a user who owns a database (Peter E) Change initdb option --with-mb to --enable-multibyte Add option for initdb to prompts for superuser password (Peter E) Allow complex type casts like col::numeric(9,2) and col::int2::float8 (Tom) Updated user interfaces on initdb, initlocation, pg_dump, ipcclean (Peter E) New pg_char_to_encoding() and pg_encoding_to_char() functions (Tatsuo) libpq non-blocking mode (Alfred Perlstein) Improve conversion of types in casts that don’t specify a length New plperl internal programming language (Mark Hollomon) Allow COPY IN to read file that do not end with a newline (Tom) Indicate when long identifiers are truncated (Tom) Allow aggregates to use type equivalency (Peter E) Add Oracle’s to_char(), to_date(), to_datetime(), to_timestamp(), to_number() conversion functions (Karel Zak ) Add SELECT DISTINCT ON (expr [, expr ...]) targetlist ... (Tom) Check to be sure ORDER BY is compatible with the DISTINCT operation (Tom) Add NUMERIC and int8 types to ODBC Improve EXPLAIN results for Append, Group, Agg, Unique (Tom) Add ALTER TABLE ... ADD FOREIGN KEY (Stephan Szabo) Allow SELECT .. FOR UPDATE in PL/pgSQL (Hiroshi) Enable backward sequential scan even after reaching EOF (Hiroshi) Add btree indexing of boolean values, >= and lowbound AND x < highbound (Tom) Use DNF instead of CNF where appropriate (Tom, Taral) Further cleanup for OR-of-AND WHERE-clauses (Tom) Make use of index in OR clauses (x = 1 AND y = 2) OR (x = 2 AND y = 4) (Tom) Smarter optimizer computations for random index page access (Tom) New SET variable to control optimizer costs (Tom) Optimizer queries based on LIMIT, OFFSET, and EXISTS qualifications (Tom) Reduce optimizer internal housekeeping of join paths for speedup (Tom) Major subquery speedup (Tom) Fewer fsync writes when fsync is not disabled (Tom) Improved LIKE optimizer estimates (Tom) Prevent fsync in SELECT-only queries (Vadim) Make index creation use psort code, because it is now faster (Tom) Allow creation of sort temp tables > 1 Gig Source Tree Changes ------------------Fix for linux PPC compile New generic expression-tree-walker subroutine (Tom) Change form() to varargform() to prevent portability problems Improved range checking for large integers on Alphas Clean up #include in /include directory (Bruce) Add scripts for checking includes (Bruce) Remove un-needed #include’s from *.c files (Bruce) Change #include’s to use and "" as appropriate (Bruce)
2636
Appendix E. Release Notes Enable Windows compilation of libpq Alpha spinlock fix from Uncle George Overhaul of optimizer data structures (Tom) Fix to cygipc library (Yutaka Tanida) Allow pgsql to work on newer Cygwin snapshots (Dan) New catalog version number (Tom) Add Linux ARM Rename heap_replace to heap_update Update for QNX (Dr. Andreas Kardos) New platform-specific regression handling (Tom) Rename oid8 -> oidvector and int28 -> int2vector (Bruce) Included all yacc and lex files into the distribution (Peter E.) Remove lextest, no longer needed (Peter E) Fix for libpq and psql on Windows (Magnus) Internally change datetime and timespan into timestamp and interval (Thomas) Fix for plpgsql on BSD/OS Add SQL_ASCII test case to the regression test (Tatsuo) configure --with-mb now deprecated (Tatsuo) NT fixes NetBSD fixes (Johnny C. Lam ) Fixes for Alpha compiles New multibyte encodings
E.286. Release 6.5.3 Release date: 1999-10-13 This is basically a cleanup release for 6.5.2. We have added a new PgAccess that was missing in 6.5.2, and installed an NT-specific fix.
E.286.1. Migration to Version 6.5.3 A dump/restore is not required for those running 6.5.*.
E.286.2. Changes Updated version of pgaccess 0.98 NT-specific patch Fix dumping rules on inherited tables
E.287. Release 6.5.2 Release date: 1999-09-15
2637
Appendix E. Release Notes This is basically a cleanup release for 6.5.1. We have fixed a variety of problems reported by 6.5.1 users.
E.287.1. Migration to Version 6.5.2 A dump/restore is not required for those running 6.5.*.
E.287.2. Changes subselect+CASE fixes(Tom) Add SHLIB_LINK setting for solaris_i386 and solaris_sparc ports(Daren Sefcik) Fixes for CASE in WHERE join clauses(Tom) Fix BTScan abort(Tom) Repair the check for redundant UNIQUE and PRIMARY KEY indexes(Thomas) Improve it so that it checks for multicolumn constraints(Thomas) Fix for Windows making problem with MB enabled(Hiroki Kataoka) Allow BSD yacc and bison to compile pl code(Bruce) Fix SET NAMES working int8 fixes(Thomas) Fix vacuum’s memory consumption(Hiroshi,Tatsuo) Reduce the total memory consumption of vacuum(Tom) Fix for timestamp(datetime) Rule deparsing bugfixes(Tom) Fix quoting problems in mkMakefile.tcldefs.sh.in and mkMakefile.tkdefs.sh.in(Tom) This is to re-use space on index pages freed by vacuum(Vadim) document -x for pg_dump(Bruce) Fix for unary operators in rule deparser(Tom) Comment out FileUnlink of excess segments during mdtruncate()(Tom) IRIX linking fix from Yu Cao >[email protected]< Repair logic error in LIKE: should not return LIKE_ABORT when reach end of pattern before end of text(Tom) Repair incorrect cleanup of heap memory allocation during transaction abort(Tom) Updated version of pgaccess 0.98
E.288. Release 6.5.1 Release date: 1999-07-15 This is basically a cleanup release for 6.5. We have fixed a variety of problems reported by 6.5 users.
E.288.1. Migration to Version 6.5.1 A dump/restore is not required for those running 6.5.
2638
Appendix E. Release Notes
E.288.2. Changes Add NT README file Portability fixes for linux_ppc, IRIX, linux_alpha, OpenBSD, alpha Remove QUERY_LIMIT, use SELECT...LIMIT Fix for EXPLAIN on inheritance(Tom) Patch to allow vacuum on multisegment tables(Hiroshi) R-Tree optimizer selectivity fix(Tom) ACL file descriptor leak fix(Atsushi Ogawa) New expression subtree code(Tom) Avoid disk writes for read-only transactions(Vadim) Fix for removal of temp tables if last transaction was aborted(Bruce) Fix to prevent too large row from being created(Bruce) plpgsql fixes Allow port numbers 32k - 64k(Bruce) Add ^ precedence(Bruce) Rename sort files called pg_temp to pg_sorttemp(Bruce) Fix for microseconds in time values(Tom) Tutorial source cleanup New linux_m68k port Fix for sorting of NULL’s in some cases(Tom) Shared library dependencies fixed (Tom) Fixed glitches affecting GROUP BY in subselects(Tom) Fix some compiler warnings (Tomoaki Nishiyama) Add Win1250 (Czech) support (Pavel Behal)
E.289. Release 6.5 Release date: 1999-06-09 This release marks a major step in the development team’s mastery of the source code we inherited from Berkeley. You will see we are now easily adding major features, thanks to the increasing size and experience of our world-wide development team. Here is a brief summary of the more notable changes: Multiversion concurrency control(MVCC) This removes our old table-level locking, and replaces it with a locking system that is superior to most commercial database systems. In a traditional system, each row that is modified is locked until committed, preventing reads by other users. MVCC uses the natural multiversion nature of PostgreSQL to allow readers to continue reading consistent data during writer activity. Writers continue to use the compact pg_log transaction system. This is all performed without having to allocate a lock for every row like traditional database systems. So, basically, we no longer are restricted by simple table-level locking; we have something better than row-level locking. Hot backups from pg_dump pg_dump takes advantage of the new MVCC features to give a consistent database dump/backup while the database stays online and available for queries. Numeric data type We now have a true numeric data type, with user-specified precision.
2639
Appendix E. Release Notes Temporary tables Temporary tables are guaranteed to have unique names within a database session, and are destroyed on session exit. New SQL features We now have CASE, INTERSECT, and EXCEPT statement support. We have new LIMIT/OFFSET, SET TRANSACTION ISOLATION LEVEL, SELECT ... FOR UPDATE, and an improved LOCK TABLE command. Speedups We continue to speed up PostgreSQL, thanks to the variety of talents within our team. We have sped up memory allocation, optimization, table joins, and row transfer routines. Ports We continue to expand our port list, this time including Windows NT/ix86 and NetBSD/arm32. Interfaces Most interfaces have new versions, and existing functionality has been improved. Documentation New and updated material is present throughout the documentation. New FAQs have been contributed for SGI and AIX platforms. The Tutorial has introductory information on SQL from Stefan Simkovics. For the User’s Guide, there are reference pages covering the postmaster and more utility programs, and a new appendix contains details on date/time behavior. The Administrator’s Guide has a new chapter on troubleshooting from Tom Lane. And the Programmer’s Guide has a description of query processing, also from Stefan, and details on obtaining the PostgreSQL source tree via anonymous CVS and CVSup.
E.289.1. Migration to Version 6.5 A dump/restore using pg_dump is required for those wishing to migrate data from any previous release of PostgreSQL. pg_upgrade can not be used to upgrade to this release because the on-disk structure of the tables has changed compared to previous releases. The new Multiversion Concurrency Control (MVCC) features can give somewhat different behaviors in multiuser environments. Read and understand the following section to ensure that your existing applications will give you the behavior you need.
E.289.1.1. Multiversion Concurrency Control Because readers in 6.5 don’t lock data, regardless of transaction isolation level, data read by one transaction can be overwritten by another. In other words, if a row is returned by SELECT it doesn’t mean that this row really exists at the time it is returned (i.e. sometime after the statement or transaction began) nor that the row is protected from being deleted or updated by concurrent transactions before the current transaction does a commit or rollback. To ensure the actual existence of a row and protect it against concurrent updates one must use SELECT FOR UPDATE or an appropriate LOCK TABLE statement. This should be taken into account when porting applications from previous releases of PostgreSQL and other environments. Keep the above in mind if you are using contrib/refint.* triggers for referential integrity. Additional techniques are required now. One way is to use LOCK parent_table IN SHARE ROW
2640
Appendix E. Release Notes EXCLUSIVE MODE command if a transaction is going to update/delete a primary key and use LOCK parent_table IN SHARE MODE command if a transaction is going to update/insert a foreign key. Note: Note that if you run a transaction in SERIALIZABLE mode then you must execute the LOCK commands above before execution of any DML statement (SELECT/INSERT/DELETE/UPDATE/FETCH/COPY_TO) in the transaction.
These inconveniences will disappear in the future when the ability to read dirty (uncommitted) data (regardless of isolation level) and true referential integrity will be implemented.
E.289.2. Changes Bug Fixes --------Fix textfloat8 and textfloat4 conversion functions(Thomas) Fix for creating tables with mixed-case constraints(Billy) Change exp()/pow() behavior to generate error on underflow/overflow(Jan) Fix bug in pg_dump -z Memory overrun cleanups(Tatsuo) Fix for lo_import crash(Tatsuo) Adjust handling of data type names to suppress double quotes(Thomas) Use type coercion for matching columns and DEFAULT(Thomas) Fix deadlock so it only checks once after one second of sleep(Bruce) Fixes for aggregates and PL/pgsql(Hiroshi) Fix for subquery crash(Vadim) Fix for libpq function PQfnumber and case-insensitive names(Bahman Rafatjoo) Fix for large object write-in-middle, no extra block, memory consumption(Tatsuo) Fix for pg_dump -d or -D and quote special characters in INSERT Repair serious problems with dynahash(Tom) Fix INET/CIDR portability problems Fix problem with selectivity error in ALTER TABLE ADD COLUMN(Bruce) Fix executor so mergejoin of different column types works(Tom) Fix for Alpha OR selectivity bug Fix OR index selectivity problem(Bruce) Fix so \d shows proper length for char()/varchar()(Ryan) Fix tutorial code(Clark) Improve destroyuser checking(Oliver) Fix for Kerberos(Rodney McDuff) Fix for dropping database while dirty buffers(Bruce) Fix so sequence nextval() can be case-sensitive(Bruce) Fix !!= operator Drop buffers before destroying database files(Bruce) Fix case where executor evaluates functions twice(Tatsuo) Allow sequence nextval actions to be case-sensitive(Bruce) Fix optimizer indexing not working for negative numbers(Bruce) Fix for memory leak in executor with fjIsNull Fix for aggregate memory leaks(Erik Riedel) Allow user name containing a dash to grant privileges Cleanup of NULL in inet types Clean up system table bugs(Tom) Fix problems of PAGER and \? command(Masaaki Sakaida)
2641
Appendix E. Release Notes Reduce default multisegment file size limit to 1GB(Peter) Fix for dumping of CREATE OPERATOR(Tom) Fix for backward scanning of cursors(Hiroshi Inoue) Fix for COPY FROM STDIN when using \i(Tom) Fix for subselect is compared inside an expression(Jan) Fix handling of error reporting while returning rows(Tom) Fix problems with reference to array types(Tom,Jan) Prevent UPDATE SET oid(Jan) Fix pg_dump so -t option can handle case-sensitive tablenames Fixes for GROUP BY in special cases(Tom, Jan) Fix for memory leak in failed queries(Tom) DEFAULT now supports mixed-case identifiers(Tom) Fix for multisegment uses of DROP/RENAME table, indexes(Ole Gjerde) Disable use of pg_dump with both -o and -d options(Bruce) Allow pg_dump to properly dump group privileges(Bruce) Fix GROUP BY in INSERT INTO table SELECT * FROM table2(Jan) Fix for computations in views(Jan) Fix for aggregates on array indexes(Tom) Fix for DEFAULT handles single quotes in value requiring too many quotes Fix security problem with non-super users importing/exporting large objects(Tom) Rollback of transaction that creates table cleaned up properly(Tom) Fix to allow long table and column names to generate proper serial names(Tom) Enhancements -----------Add "vacuumdb" utility Speed up libpq by allocating memory better(Tom) EXPLAIN all indexes used(Tom) Implement CASE, COALESCE, NULLIF expression(Thomas) New pg_dump table output format(Constantin) Add string min()/max() functions(Thomas) Extend new type coercion techniques to aggregates(Thomas) New moddatetime contrib(Terry) Update to pgaccess 0.96(Constantin) Add routines for single-byte "char" type(Thomas) Improved substr() function(Thomas) Improved multibyte handling(Tatsuo) Multiversion concurrency control/MVCC(Vadim) New Serialized mode(Vadim) Fix for tables over 2gigs(Peter) New SET TRANSACTION ISOLATION LEVEL(Vadim) New LOCK TABLE IN ... MODE(Vadim) Update ODBC driver(Byron) New NUMERIC data type(Jan) New SELECT FOR UPDATE(Vadim) Handle "NaN" and "Infinity" for input values(Jan) Improved date/year handling(Thomas) Improved handling of backend connections(Magnus) New options ELOG_TIMESTAMPS and USE_SYSLOG options for log files(Massimo) New TCL_ARRAYS option(Massimo) New INTERSECT and EXCEPT(Stefan) New pg_index.indisprimary for primary key tracking(D’Arcy) New pg_dump option to allow dropping of tables before creation(Brook) Speedup of row output routines(Tom) New READ COMMITTED isolation level(Vadim) New TEMP tables/indexes(Bruce) Prevent sorting if result is already sorted(Jan)
2642
Appendix E. Release Notes New memory allocation optimization(Jan) Allow psql to do \p\g(Bruce) Allow multiple rule actions(Jan) Added LIMIT/OFFSET functionality(Jan) Improve optimizer when joining a large number of tables(Bruce) New intro to SQL from S. Simkovics’ Master’s Thesis (Stefan, Thomas) New intro to backend processing from S. Simkovics’ Master’s Thesis (Stefan) Improved int8 support(Ryan Bradetich, Thomas, Tom) New routines to convert between int8 and text/varchar types(Thomas) New bushy plans, where meta-tables are joined(Bruce) Enable right-hand queries by default(Bruce) Allow reliable maximum number of backends to be set at configure time (--with-maxbackends and postmaster switch (-N backends))(Tom) GEQO default now 10 tables because of optimizer speedups(Tom) Allow NULL=Var for MS-SQL portability(Michael, Bruce) Modify contrib check_primary_key() so either "automatic" or "dependent"(Anand) Allow psql \d on a view show query(Ryan) Speedup for LIKE(Bruce) Ecpg fixes/features, see src/interfaces/ecpg/ChangeLog file(Michael) JDBC fixes/features, see src/interfaces/jdbc/CHANGELOG(Peter) Make % operator have precedence like /(Bruce) Add new postgres -O option to allow system table structure changes(Bruce) Update contrib/pginterface/findoidjoins script(Tom) Major speedup in vacuum of deleted rows with indexes(Vadim) Allow non-SQL functions to run different versions based on arguments(Tom) Add -E option that shows actual queries sent by \dt and friends(Masaaki Sakaida) Add version number in start-up banners for psql(Masaaki Sakaida) New contrib/vacuumlo removes large objects not referenced(Peter) New initialization for table sizes so non-vacuumed tables perform better(Tom) Improve error messages when a connection is rejected(Tom) Support for arrays of char() and varchar() fields(Massimo) Overhaul of hash code to increase reliability and performance(Tom) Update to PyGreSQL 2.4(D’Arcy) Changed debug options so -d4 and -d5 produce different node displays(Jan) New pg_options: pretty_plan, pretty_parse, pretty_rewritten(Jan) Better optimization statistics for system table access(Tom) Better handling of non-default block sizes(Massimo) Improve GEQO optimizer memory consumption(Tom) UNION now supports ORDER BY of columns not in target list(Jan) Major libpq++ improvements(Vince Vielhaber) pg_dump now uses -z(ACL’s) as default(Bruce) backend cache, memory speedups(Tom) have pg_dump do everything in one snapshot transaction(Vadim) fix for large object memory leakage, fix for pg_dumping(Tom) INET type now respects netmask for comparisons Make VACUUM ANALYZE only use a readlock(Vadim) Allow VIEWs on UNIONS(Jan) pg_dump now can generate consistent snapshots on active databases(Vadim) Source Tree Changes ------------------Improve port matching(Tom) Portability fixes for SunOS Add Windows NT backend port and enable dynamic loading(Magnus and Daniel Horak) New port to Cobalt Qube(Mips) running Linux(Tatsuo) Port to NetBSD/m68k(Mr. Mutsuki Nakajima) Port to NetBSD/sun3(Mr. Mutsuki Nakajima)
2643
Appendix E. Release Notes Port to NetBSD/macppc(Toshimi Aoki) Fix for tcl/tk configuration(Vince) Removed CURRENT key word for rule queries(Jan) NT dynamic loading now works(Daniel Horak) Add ARM32 support(Andrew McMurry) Better support for HP-UX 11 and UnixWare Improve file handling to be more uniform, prevent file descriptor leak(Tom) New install commands for plpgsql(Jan)
E.290. Release 6.4.2 Release date: 1998-12-20 The 6.4.1 release was improperly packaged. This also has one additional bug fix.
E.290.1. Migration to Version 6.4.2 A dump/restore is not required for those running 6.4.*.
E.290.2. Changes Fix for datetime constant problem on some platforms(Thomas)
E.291. Release 6.4.1 Release date: 1998-12-18 This is basically a cleanup release for 6.4. We have fixed a variety of problems reported by 6.4 users.
E.291.1. Migration to Version 6.4.1 A dump/restore is not required for those running 6.4.
E.291.2. Changes Add pg_dump -N flag to force double quotes around identifiers. This is the default(Thomas) Fix for NOT in where clause causing crash(Bruce) EXPLAIN VERBOSE coredump fix(Vadim) Fix shared-library problems on Linux Fix test for table existence to allow mixed-case and whitespace in
2644
Appendix E. Release Notes the table name(Thomas) Fix a couple of pg_dump bugs Configure matches template/.similar entries better(Tom) Change builtin function names from SPI_* to spi_* OR WHERE clause fix(Vadim) Fixes for mixed-case table names(Billy) contrib/linux/postgres.init.csh/sh fix(Thomas) libpq memory overrun fix SunOS fixes(Tom) Change exp() behavior to generate error on underflow(Thomas) pg_dump fixes for memory leak, inheritance constraints, layout change update pgaccess to 0.93 Fix prototype for 64-bit platforms Multibyte fixes(Tatsuo) New ecpg man page Fix memory overruns(Tatsuo) Fix for lo_import() crash(Bruce) Better search for install program(Tom) Timezone fixes(Tom) HP-UX fixes(Tom) Use implicit type coercion for matching DEFAULT values(Thomas) Add routines to help with single-byte (internal) character type(Thomas) Compilation of libpq for Windows fixes(Magnus) Upgrade to PyGreSQL 2.2(D’Arcy)
E.292. Release 6.4 Release date: 1998-10-30 There are many new features and improvements in this release. Thanks to our developers and maintainers, nearly every aspect of the system has received some attention since the previous release. Here is a brief, incomplete summary: •
Views and rules are now functional thanks to extensive new code in the rewrite rules system from Jan Wieck. He also wrote a chapter on it for the Programmer’s Guide.
•
Jan also contributed a second procedural language, PL/pgSQL, to go with the original PL/pgTCL procedural language he contributed last release.
•
We have optional multiple-byte character set support from Tatsuo Ishii to complement our existing locale support.
•
Client/server communications has been cleaned up, with better support for asynchronous messages and interrupts thanks to Tom Lane.
•
The parser will now perform automatic type coercion to match arguments to available operators and functions, and to match columns and expressions with target columns. This uses a generic mechanism which supports the type extensibility features of PostgreSQL. There is a new chapter in the User’s Guide which covers this topic.
•
Three new data types have been added. Two types, inet and cidr, support various forms of IP network, subnet, and machine addressing. There is now an 8-byte integer type available on some
2645
Appendix E. Release Notes platforms. See the chapter on data types in the User’s Guide for details. A fourth type, serial, is now supported by the parser as an amalgam of the int4 type, a sequence, and a unique index. •
Several more SQL92-compatible syntax features have been added, including INSERT DEFAULT VALUES
•
The automatic configuration and installation system has received some attention, and should be more robust for more platforms than it has ever been.
E.292.1. Migration to Version 6.4 A dump/restore using pg_dump or pg_dumpall is required for those wishing to migrate data from any previous release of PostgreSQL.
E.292.2. Changes Bug Fixes --------Fix for a tiny memory leak in PQsetdb/PQfinish(Bryan) Remove char2-16 data types, use char/varchar(Darren) Pqfn not handles a NOTICE message(Anders) Reduced busywaiting overhead for spinlocks with many backends (dg) Stuck spinlock detection (dg) Fix up "ISO-style" timespan decoding and encoding(Thomas) Fix problem with table drop after rollback of transaction(Vadim) Change error message and remove non-functional update message(Vadim) Fix for COPY array checking Fix for SELECT 1 UNION SELECT NULL Fix for buffer leaks in large object calls(Pascal) Change owner from oid to int4 type(Bruce) Fix a bug in the oracle compatibility functions btrim() ltrim() and rtrim() Fix for shared invalidation cache overflow(Massimo) Prevent file descriptor leaks in failed COPY’s(Bruce) Fix memory leak in libpgtcl’s pg_select(Constantin) Fix problems with username/passwords over 8 characters(Tom) Fix problems with handling of asynchronous NOTIFY in backend(Tom) Fix of many bad system table entries(Tom) Enhancements -----------Upgrade ecpg and ecpglib,see src/interfaces/ecpc/ChangeLog(Michael) Show the index used in an EXPLAIN(Zeugswetter) EXPLAIN invokes rule system and shows plan(s) for rewritten queries(Jan) Multibyte awareness of many data types and functions, via configure(Tatsuo) New configure --with-mb option(Tatsuo) New initdb --pgencoding option(Tatsuo) New createdb -E multibyte option(Tatsuo) Select version(); now returns PostgreSQL version(Jeroen) libpq now allows asynchronous clients(Tom) Allow cancel from client of backend query(Tom) psql now cancels query with Control-C(Tom) libpq users need not issue dummy queries to get NOTIFY messages(Tom) NOTIFY now sends sender’s PID, so you can tell whether it was your own(Tom)
2646
Appendix E. Release Notes PGresult struct now includes associated error message, if any(Tom) Define "tz_hour" and "tz_minute" arguments to date_part()(Thomas) Add routines to convert between varchar and bpchar(Thomas) Add routines to allow sizing of varchar and bpchar into target columns(Thomas) Add bit flags to support timezonehour and minute in data retrieval(Thomas) Allow more variations on valid floating point numbers (e.g. ".1", "1e6")(Thomas) Fixes for unary minus parsing with leading spaces(Thomas) Implement TIMEZONE_HOUR, TIMEZONE_MINUTE per SQL92 specs(Thomas) Check for and properly ignore FOREIGN KEY column constraints(Thomas) Define USER as synonym for CURRENT_USER per SQL92 specs(Thomas) Enable HAVING clause but no fixes elsewhere yet. Make "char" type a synonym for "char(1)" (actually implemented as bpchar)(Thomas) Save string type if specified for DEFAULT clause handling(Thomas) Coerce operations involving different data types(Thomas) Allow some index use for columns of different types(Thomas) Add capabilities for automatic type conversion(Thomas) Cleanups for large objects, so file is truncated on open(Peter) Readline cleanups(Tom) Allow psql \f \ to make spaces as delimiter(Bruce) Pass pg_attribute.atttypmod to the frontend for column field lengths(Tom,Bruce) Msql compatibility library in /contrib(Aldrin) Remove the requirement that ORDER/GROUP BY clause identifiers be included in the target list(David) Convert columns to match columns in UNION clauses(Thomas) Remove fork()/exec() and only do fork()(Bruce) Jdbc cleanups(Peter) Show backend status on ps command line(only works on some platforms)(Bruce) Pg_hba.conf now has a sameuser option in the database field Make lo_unlink take oid param, not int4 New DISABLE_COMPLEX_MACRO for compilers that cannot handle our macros(Bruce) Libpgtcl now handles NOTIFY as a Tcl event, need not send dummy queries(Tom) libpgtcl cleanups(Tom) Add -error option to libpgtcl’s pg_result command(Tom) New locale patch, see docs/README/locale(Oleg) Fix for pg_dump so CONSTRAINT and CHECK syntax is correct(ccb) New contrib/lo code for large object orphan removal(Peter) New psql command "SET CLIENT_ENCODING TO ’encoding’" for multibytes feature, see /doc/README.mb(Tatsuo) contrib/noupdate code to revoke update permission on a column libpq can now be compiled on Windows(Magnus) Add PQsetdbLogin() in libpq New 8-byte integer type, checked by configure for OS support(Thomas) Better support for quoted table/column names(Thomas) Surround table and column names with double-quotes in pg_dump(Thomas) PQreset() now works with passwords(Tom) Handle case of GROUP BY target list column number out of range(David) Allow UNION in subselects Add auto-size to screen to \d? commands(Bruce) Use UNION to show all \d? results in one query(Bruce) Add \d? field search feature(Bruce) Pg_dump issues fewer \connect requests(Tom) Make pg_dump -z flag work better, document it in manual page(Tom) Add HAVING clause with full support for subselects and unions(Stephan) Full text indexing routines in contrib/fulltextindex(Maarten) Transaction ids now stored in shared memory(Vadim) New PGCLIENTENCODING when issuing COPY command(Tatsuo) Support for SQL92 syntax "SET NAMES"(Tatsuo)
2647
Appendix E. Release Notes Support for LATIN2-5(Tatsuo) Add UNICODE regression test case(Tatsuo) Lock manager cleanup, new locking modes for LLL(Vadim) Allow index use with OR clauses(Bruce) Allows "SELECT NULL ORDER BY 1;" Explain VERBOSE prints the plan, and now pretty-prints the plan to the postmaster log file(Bruce) Add indexes display to \d command(Bruce) Allow GROUP BY on functions(David) New pg_class.relkind for large objects(Bruce) New way to send libpq NOTICE messages to a different location(Tom) New \w write command to psql(Bruce) New /contrib/findoidjoins scans oid columns to find join relationships(Bruce) Allow binary-compatible indexes to be considered when checking for valid Indexes for restriction clauses containing a constant(Thomas) New ISBN/ISSN code in /contrib/isbn_issn Allow NOT LIKE, IN, NOT IN, BETWEEN, and NOT BETWEEN constraint(Thomas) New rewrite system fixes many problems with rules and views(Jan) * Rules on relations work * Event qualifications on insert/update/delete work * New OLD variable to reference CURRENT, CURRENT will be remove in future * Update rules can reference NEW and OLD in rule qualifications/actions * Insert/update/delete rules on views work * Multiple rule actions are now supported, surrounded by parentheses * Regular users can create views/rules on tables they have RULE permits * Rules and views inherit the privileges of the creator * No rules at the column level * No UPDATE NEW/OLD rules * New pg_tables, pg_indexes, pg_rules and pg_views system views * Only a single action on SELECT rules * Total rewrite overhaul, perhaps for 6.5 * handle subselects * handle aggregates on views * handle insert into select from view works System indexes are now multikey(Bruce) Oidint2, oidint4, and oidname types are removed(Bruce) Use system cache for more system table lookups(Bruce) New backend programming language PL/pgSQL in backend/pl(Jan) New SERIAL data type, auto-creates sequence/index(Thomas) Enable assert checking without a recompile(Massimo) User lock enhancements(Massimo) New setval() command to set sequence value(Massimo) Auto-remove unix socket file on start-up if no postmaster running(Massimo) Conditional trace package(Massimo) New UNLISTEN command(Massimo) psql and libpq now compile under Windows using win32.mak(Magnus) Lo_read no longer stores trailing NULL(Bruce) Identifiers are now truncated to 31 characters internally(Bruce) Createuser options now available on the command line Code for 64-bit integer supported added, configure tested, int8 type(Thomas) Prevent file descriptor leaf from failed COPY(Bruce) New pg_upgrade command(Bruce) Updated /contrib directories(Massimo) New CREATE TABLE DEFAULT VALUES statement available(Thomas) New INSERT INTO TABLE DEFAULT VALUES statement available(Thomas) New DECLARE and FETCH feature(Thomas) libpq’s internal structures now not exported(Tom)
2648
Appendix E. Release Notes Allow up to 8 key indexes(Bruce) Remove ARCHIVE key word, that is no longer used(Thomas) pg_dump -n flag to suppress quotes around identifiers disable system columns for views(Jan) new INET and CIDR types for network addresses(TomH, Paul) no more double quotes in psql output pg_dump now dumps views(Terry) new SET QUERY_LIMIT(Tatsuo,Jan) Source Tree Changes ------------------/contrib cleanup(Jun) Inline some small functions called for every row(Bruce) Alpha/linux fixes HP-UX cleanups(Tom) Multibyte regression tests(Soonmyung.) Remove --disabled options from configure Define PGDOC to use POSTGRESDIR by default Make regression optional Remove extra braces code to pgindent(Bruce) Add bsdi shared library support(Bruce) New --without-CXX support configure option(Brook) New FAQ_CVS Update backend flowchart in tools/backend(Bruce) Change atttypmod from int16 to int32(Bruce, Tom) Getrusage() fix for platforms that do not have it(Tom) Add PQconnectdb, PGUSER, PGPASSWORD to libpq man page NS32K platform fixes(Phil Nelson, John Buller) SCO 7/UnixWare 2.x fixes(Billy,others) Sparc/Solaris 2.5 fixes(Ryan) Pgbuiltin.3 is obsolete, move to doc files(Thomas) Even more documentation(Thomas) Nextstep support(Jacek) Aix support(David) pginterface manual page(Bruce) shared libraries all have version numbers merged all OS-specific shared library defines into one file smarter TCL/TK configuration checking(Billy) smarter perl configuration(Brook) configure uses supplied install-sh if no install script found(Tom) new Makefile.shlib for shared library configuration(Tom)
E.293. Release 6.3.2 Release date: 1998-04-07 This is a bug-fix release for 6.3.x. Refer to the release notes for version 6.3 for a more complete summary of new features. Summary: •
Repairs automatic configuration support for some platforms, including Linux, from breakage inadvertently introduced in version 6.3.1.
2649
Appendix E. Release Notes •
Correctly handles function calls on the left side of BETWEEN and LIKE clauses.
A dump/restore is NOT required for those running 6.3 or 6.3.1. A make distclean, make, and make install is all that is required. This last step should be performed while the postmaster is not running. You should re-link any custom applications that use PostgreSQL libraries. For upgrades from pre-6.3 installations, refer to the installation and migration instructions for version 6.3.
E.293.1. Changes Configure detection improvements for tcl/tk(Brook Milligan, Alvin) Manual page improvements(Bruce) BETWEEN and LIKE fix(Thomas) fix for psql \connect used by pg_dump(Oliver Elphick) New odbc driver pgaccess, version 0.86 qsort removed, now uses libc version, cleanups(Jeroen) fix for buffer over-runs detected(Maurice Gittens) fix for buffer overrun in libpgtcl(Randy Kunkee) fix for UNION with DISTINCT or ORDER BY(Bruce) gettimeofday configure check(Doug Winterburn) Fix "indexes not used" bug(Vadim) docs additions(Thomas) Fix for backend memory leak(Bruce) libreadline cleanup(Erwan MAS) Remove DISTDIR(Bruce) Makefile dependency cleanup(Jeroen van Vianen) ASSERT fixes(Bruce)
Repair byte ordering for mixed-endian clients and servers.
•
Minor updates to allowed SQL syntax.
•
Improvements to the configuration autodetection for installation.
A dump/restore is NOT required for those running 6.3. A make distclean, make, and make install is all that is required. This last step should be performed while the postmaster is not running. You should re-link any custom applications that use PostgreSQL libraries. For upgrades from pre-6.3 installations, refer to the installation and migration instructions for version 6.3.
2650
Appendix E. Release Notes
E.294.1. Changes ecpg cleanup/fixes, now version 1.1(Michael Meskes) pg_user cleanup(Bruce) large object fix for pg_dump and tclsh (alvin) LIKE fix for multiple adjacent underscores fix for redefining builtin functions(Thomas) ultrix4 cleanup upgrade to pg_access 0.83 updated CLUSTER manual page multibyte character set support, see doc/README.mb(Tatsuo) configure --with-pgport fix pg_ident fix big-endian fix for backend communications(Kataoka) SUBSTR() and substring() fix(Jan) several jdbc fixes(Peter) libpgtcl improvements, see libptcl/README(Randy Kunkee) Fix for "Datasize = 0" error(Vadim) Prevent \do from wrapping(Bruce) Remove duplicate Russian character set entries Sunos4 cleanup Allow optional TABLE key word in LOCK and SELECT INTO(Thomas) CREATE SEQUENCE options to allow a negative integer(Thomas) Add "PASSWORD" as an allowed column identifier(Thomas) Add checks for UNION target fields(Bruce) Fix Alpha port(Dwayne Bailey) Fix for text arrays containing quotes(Doug Gibson) Solaris compile fix(Albert Chin-A-Young) Better identify tcl and tk libs and includes(Bruce)
E.295. Release 6.3 Release date: 1998-03-01 There are many new features and improvements in this release. Here is a brief, incomplete summary: •
Many new SQL features, including full SQL92 subselect capability (everything is here but targetlist subselects).
•
Support for client-side environment variables to specify time zone and date style.
•
Socket interface for client/server connection. This is the default now so you might need to start postmaster with the -i flag.
•
Better password authorization mechanisms. Default table privileges have changed.
•
Old-style time travel has been removed. Performance has been improved.
Note: Bruce Momjian wrote the following notes to introduce the new release.
2651
Appendix E. Release Notes There are some general 6.3 issues that I want to mention. These are only the big items that cannot be described in one sentence. A review of the detailed changes list is still needed. First, we now have subselects. Now that we have them, I would like to mention that without subselects, SQL is a very limited language. Subselects are a major feature, and you should review your code for places where subselects provide a better solution for your queries. I think you will find that there are more uses for subselects than you might think. Vadim has put us on the big SQL map with subselects, and fully functional ones too. The only thing you cannot do with subselects is to use them in the target list. Second, 6.3 uses Unix domain sockets rather than TCP/IP by default. To enable connections from other machines, you have to use the new postmaster -i option, and of course edit pg_hba.conf. Also, for this reason, the format of pg_hba.conf has changed. Third, char() fields will now allow faster access than varchar() or text. Specifically, the text and varchar() have a penalty for access to any columns after the first column of this type. char() used to also have this access penalty, but it no longer does. This might suggest that you redesign some of your tables, especially if you have short character columns that you have defined as varchar() or text. This and other changes make 6.3 even faster than earlier releases. We now have passwords definable independent of any Unix file. There are new SQL USER commands. See the Administrator’s Guide for more information. There is a new table, pg_shadow, which is used to store user information and user passwords, and it by default only SELECT-able by the postgres super-user. pg_user is now a view of pg_shadow, and is SELECT-able by PUBLIC. You should keep using pg_user in your application without changes. User-created tables now no longer have SELECT privilege to PUBLIC by default. This was done because the ANSI standard requires it. You can of course GRANT any privileges you want after the table is created. System tables continue to be SELECT-able by PUBLIC. We also have real deadlock detection code. No more sixty-second timeouts. And the new locking code implements a FIFO better, so there should be less resource starvation during heavy use. Many complaints have been made about inadequate documentation in previous releases. Thomas has put much effort into many new manuals for this release. Check out the doc/ directory. For performance reasons, time travel is gone, but can be implemented using triggers (see pgsql/contrib/spi/README). Please check out the new \d command for types, operators, etc. Also, views have their own privileges now, not based on the underlying tables, so privileges on them have to be set separately. Check /pgsql/interfaces for some new ways to talk to PostgreSQL. This is the first release that really required an explanation for existing users. In many ways, this was necessary because the new release removes many limitations, and the work-arounds people were using are no longer needed.
E.295.1. Migration to Version 6.3 A dump/restore using pg_dump or pg_dumpall is required for those wishing to migrate data from any previous release of PostgreSQL.
for tcl library crash(Jan) for array handling, from Gerhard Hintermayer acl error, and remove duplicate pqtrace(Bruce) psql \e for empty file(Bruce) for textcat on varchar() fields(Bruce) for DBT Sendproc (Zeugswetter Andres) vacuum analyze syntax problem(Bruce) for international identifiers(Tatsuo) aggregates on inherited tables(Bruce) substr() for out-of-bounds data for select 1=1 or 2=2, select 1=1 and 2=2, and select sum(2+2)(Bruce) notty output to show status result. -q option still turns it off(Bruce) for count(*), aggs with views and multiple tables and sum(3)(Bruce) cluster(Bruce) for PQtrace start/stop several times(Bruce) a variety of locking problems like newer lock waiters getting lock before older waiters, and having readlock people not share locks if a writer is waiting for a lock, and waiting writers not getting priority over waiting readers(Bruce) Fix crashes in psql when executing queries from external files(James) Fix problem with multiple order by columns, with the first one having NULL values(Jeroen) Use correct hash table support functions for float8 and int4(Thomas) Re-enable JOIN= option in CREATE OPERATOR statement (Thomas) Change precedence for boolean operators to match expected behavior(Thomas) Generate elog(ERROR) on over-large integer(Bruce) Allow multiple-argument functions in constraint clauses(Thomas) Check boolean input literals for ’true’,’false’,’yes’,’no’,’1’,’0’ and throw elog(ERROR) if unrecognized(Thomas) Major large objects fix Fix for GROUP BY showing duplicates(Vadim) Fix for index scans in MergeJoin(Vadim) Enhancements -----------Subselects with EXISTS, IN, ALL, ANY key words (Vadim, Bruce, Thomas) New User Manual(Thomas, others) Speedup by inlining some frequently-called functions Real deadlock detection, no more timeouts(Bruce) Add SQL92 "constants" CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP, CURRENT_USER(Thomas) Modify constraint syntax to be SQL92-compliant(Thomas) Implement SQL92 PRIMARY KEY and UNIQUE clauses using indexes(Thomas) Recognize SQL92 syntax for FOREIGN KEY. Throw elog notice(Thomas) Allow NOT NULL UNIQUE constraint clause (each allowed separately before)(Thomas) Allow PostgreSQL-style casting ("::") of non-constants(Thomas) Add support for SQL3 TRUE and FALSE boolean constants(Thomas) Support SQL92 syntax for IS TRUE/IS FALSE/IS NOT TRUE/IS NOT FALSE(Thomas) Allow shorter strings for boolean literals (e.g. "t", "tr", "tru")(Thomas) Allow SQL92 delimited identifiers(Thomas) Implement SQL92 binary and hexadecimal string decoding (b’10’ and x’1F’)(Thomas) Support SQL92 syntax for type coercion of literal strings (e.g. "DATETIME ’now’")(Thomas) Add conversions for int2, int4, and OID types to and from text(Thomas) Use shared lock when building indexes(Vadim) Free memory allocated for a user query inside transaction block after this query is done, was turned off in
![Urban Airship - PostgreSQL wiki [PDF]](https://m.moam.info/img/260x300/urban-airship-postgresql-wiki-pdf_647bc06a098a9e977a8b45ee.jpg)
![Urban Airship - PostgreSQL wiki [PDF]](https://m.moam.info/img/260x300/urban-airship-postgresql-wiki-pdf_6480fcf5098a9e03188b45dc.jpg)
![PostgreSQL Internals (1) - HPE.com [PDF]](https://m.moam.info/img/260x300/postgresql-internals-1-hpecom-pdf_64837cc7098a9e6b088b4616.jpg)
![[PDF] PostgreSQL Server Programming - Second ... - Google Sites](https://m.moam.info/img/260x300/pdf-postgresql-server-programming-second-google-si_6477b42f097c4786708c0130.jpg)
![[PDF] PostgreSQL Server Programming - Second ... - Google Sites](https://m.moam.info/img/260x300/pdf-postgresql-server-programming-second-google-si_6477474e097c474e708b865e.jpg)