Postmodern is a Common Lisp library for interacting with PostgreSQL. It uses the frontend/backend protocol to communicate directly with PostgreSQL over a TCP/IP socket. Features include:
- Efficient communication with the database server without need for foreign libraries.
- Support for UTF-8 on Unicode-aware Lisp implementations.
- A lispy syntax for expressing SQL queries.
- Convenient support for prepared statements and stored procedures.
- Defclass-like definition of tables and associated accessor classes.
For more details and downloads see http://common-lisp.net/project/postmodern/
RE: Postgres bulk upload
Hi Ravikanth,
Our company just creates development tools and has not done any work to enhance the internal operation of PostgreSQL. You might contact Command Prompt. You should also post your question to one of the PostgreSQL mailing lists.
Postgres bulk upload
Hi
We are currently in the midst of developing the next generation version of our flagship product, Strongmail Message Studio. One of the big changes we made in this version is switching the embedded database from MySQL to Postgres. But one of the biggest issues we have been facing since the switch to Postgres is in regarding to the database import issue. Our product lets users to import huge files (could be in the order of 10M – 20M rows).
But of the current behavior (limitation) of Postgres, we are seeing a huge performance regression. Let me just give you more details. In order to import such huge files, we observed that we should use the database native import (Postgres COPY operation : http://www.postgresql.org/docs/8.1/interactive/sql-copy.html). But unfortunately, Postgres does not seem to have the IGNORE option, and hence the COPU operation stops at the first error. In most of the typical scenarios, the data files will invariably have some errors, and hence we are being forced to use JDBC batch updates. But, we noticed that JDBC batching is extremely slow (10-20 times slower) compared to COPY operation.
Here is the URL from MySQL - http://dev.mysql.com/doc/refman/5.0/en/load-data.html
As you can see MySQL “load data infile” has IGNORE option. If you specify IGNORE, input rows that duplicate an existing row on a unique key value are skipped.
This is a critical functionality for our company and we are willing to work with any professional services to enhance Postgres if necessary. I see that your company has a lot of experience and expertise in enhancing Postgres internals. So, we would like to use your services to enhance Postgres COPY operation to provide IGNORE option. Would you be able to give us some details regarding whether this is feasible and if so how much the cost might be etc.
Look forward to hearing from you.
Thanks,
Ravikanth Kasamsetty
Director, Software Development
(650) 421-4200 x2036 Direct
(650) 421-4201 Fax
Unlock the Business Potential of Digital Messaging
www.strongmail.com