Using MySQL BLOB Data via ODBC in SSIS, SQL Server & Business Objects
Whilst trying to build a centralised cross-platform alerting system I spotted a peculiar issue when trying to move the output of a SHOW FULL PROCESSLIST command on MySQL via ODBC. It seems that the output of the SHOW FULL PROCESSLIST command returns both integers and binary (BLOB) data types even though to they eye (that is, in the MySQL Query Browser) most of the columns appear to be short text fields.
Despite the fact that the data looks like text whenever I tried to return the data into an application, I tried SSIS, SQL Server Linked Servers and Business Objects, each time the data would come back unusable or an error would be returned. Business Objects gave me the key by declaring “This is a BLOB.” as you can see in the following screenshots…
SQL Server Integration Services
SQL Server Linked Server
Business Objects Desktop Intelligence
As with my recent post about loading data into MySQL with SSIS the saviour turns out to be an ODBC configuration setting, this time in the Metadata tab of the MySQL ODBC driver. All you have to do is check the “Always handle binary function results as character data” and instantly your problems will be solved…
Out of a crazy fit of completeness I also took screenshots of the final results and it’d be a shame to waste them so here they are…
SQL Server Integration Services
SQL Server Linked Server
Business Objects Desktop Intelligence
Categories: Business Objects, DBA, Microsoft SQL Server, MySQL, Open Source, SSIS Tags: binary, blob, business objects, DBA, ETL, Integration Services, mys, MySQL, ODBC, SQL Server, SSIS
Loading Data into MySQL with SSIS via ODBC (sql_mode = ANSI)
I was performing a test earlier, a proof-of-concept to show that we could use Integration Services to move data from one platform (PostgreSQL) to another (MySQL) without SQL Server itself being involved at all.
Unfortunately I hit a slight snag on the first simple attempt, I could read the data without a hitch but despite multiple attempts at tweaking SSIS settings and reconfiguring connections I could not manage to get data into the MySQL target table. Amongst other messages in the Output window in BIDS I could see the following error…
[MySQL][ODBC 5.1 Driver][mysqld-5.0.45-log]You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘”tblDestination” (“col1″, “col2″, “col3″, “col4″, “col5″, “c’ at line 1
After some Googling I determined that the issue was down to an incompatibility between the SQL being generated by SSIS and the default MySQL engine, specifically that MySQL was not setup to accept ANSI compliant SQL. Most of the solutions out there seemed to require that the entire server was configured in the ANSI compliant mode which was a problem for me as it’s a shared server and I can’t go reconfiguring it at will. Thankfully MySQL provides the ability to set the mode on a per connection level with the following command…
SET SESSION sql_mode= 'ansi';
The trick is – how to get SSIS to run this before inserting the data without messing with the package itself? The easiest way is to have the ODBC driver send the command at the point SSIS opens the connection, this can be configured in the settings of the ODBC source…
This is a relatively common feature of ODBC drivers and allows you to setup initial configuration options for this exact reason, now I know that I will not impact any other users of the server and my SSIS package works just fine.
Categories: Microsoft SQL Server, MySQL, Open Source, PostgreSQL, SSIS Tags: .NET, ADO, BIDS, ETL, Integration Services, MySQL, ODBC, Postgres, PostgreSQL, SSIS
Quick Tip – MySQL Equivalent of ISNUMERIC()
In one of the source databases for my data warehouse I have a table of contact telephone numbers, stored as VARCHAR() values since where a number is not available the record might say “unknown”, or “N/A”. So to extract a list of all valid telephone telephone numbers I need to perform a logical test and one of the simplest checks I can perform is to verify that the number is actually a number.
This would be pretty straight-forward in Excel with the ISNUMBER() function, or in T-SQL with ISNUMERIC(), but neither work in MySQL so after a little searching around I came across this solution…
SELECT DISTINCT contact_number
FROM customers
WHERE contact_number REGEXP (‘[0-9]‘)
Effectively we’re processing a regular expression on the contents of the ‘contact_number’ field, it may seem like using a sledgehammer to crack a nut and I’ve no idea how performance would differ from a more simple approach but it worked and I guess that’s the point.
Categories: MySQL, Open Source Tags: ISNUMBER, ISNUMERIC, MySQL, RegExp, Regular Expressions
MySQL 32-bit ODBC Invalid Attribute String 64-bit Windows 7
Having just migrated my development PC to Windows 7 I’m slowly encountering perculiar issues as I setup all of the software and connections I used to have in Windows XP. This morning I was trying to write a Business Objects report against a MySQL database and because Desktop Intelligence is a 32-bit application if I want it to talk to MySQL I have to use the 32-bit driver.
I obtained the latest driver (5.1.7) from MySQL’s standard ODBC Connector page, installed it and added a System DSN without a hitch, the odd part came when Business Objects was returning only one row from a query that should return a couple of hundred. Having run the same query on XP (I’m parallel running now) I suspected that the problem must be with the ODBC configuration so I attempted to delete the DSN only to receive the “Invalid attribute string” error…
A little Googling later led me to MySQL Bug #56233, in the discussion Fred Zappert frames the simplest solution suggested which is to uninstall version 5.1.7 and install version 5.1.6 instead. Oddly MySQL don’t make it especially clear how to get hold of previous minor versions of the ODBC drivers but you can get it from here: mysql-connector-odbc-5.1.6-win32.msi
Alternatively you could always visit bisql.net’s Tools, Utilities and ODBC Drivers page where I have a link for 5.1.6 which I’ll keep in place until the next Windows 7 compatible driver is released.
Categories: Business Objects, MySQL, Windows Tags: 32-bit, 64-bit, bug, business objects, Database, DeskI, Desktop Intelligence, error, MySQL, ODBC, Win7, Windows 7
BusObj + MySQL – SQLBindParameter State 07001
I recently encountered a strange problem in Business Objects whilst trying to create a derived table against a MySQL database and whilst the query was a complex one it executed without fault in the MySQL Query Browser so I knew that wasn’t the probem. The error message I encountered was “SQLBindParameter not used for all parameters”…
I tried the same query as a Freehand SQL query in DeskI and received the same error, after banging my head against the problem for about 15 minutes I decided to take my favourite solution – I went for lunch! On return the solution seemed blindingly obvious, at the very top of my query I’d included a comment but in the comment I’d included a question mark – as soon as I removed the “?” the query ran without a hitch. I’m not quite sure of the reason behind this but I suspect that one of the Business Objects, ODBC or MySQL layers treats the “?” as a reserved character to indicate a parameterised query – if anyone has the answer i’d be interested to know.
Categories: Business Objects, MySQL Tags: business objects, DeskI, sql, SQLBindParameter, Universe
Creating Databases and Users in MySQL
Historically I’ve been a Microsoft SQL Server guy but I’ve been doing quite a bit of query & analysis work on MySQL lately, though I’ve never performed any serious MySQL DBA. Well, the other day I was asked by a friend how to create a user and allow read/write access to a newly created database and this had to be done in SQL (i.e. no GUI tools or PHPMyAdmin).
Since I was starting from scratch I thought I’d put together a little script to create the database, add some data, create a user, prove that the user had write access and then tidy up after myself (always good to do!).
CREATE DATABASE ash_db;
USE ash_db;
CREATE TABLE ash_tbl (id INT NOT NULL,name CHAR(50) NOT NULL);
INSERT INTO ash_tbl (id,name) VALUES(1,'Ash');
SELECT * FROM ash_tbl;
CREATE USER 'ash_user' IDENTIFIED BY 'ash_pass';
GRANT ALL PRIVILEGES ON ash_db.* TO ash_user;
-- LOGIN AS ash_user THEN EXECUTE THIS...
INSERT INTO ash_tbl (id,name) VALUES(2,'Burton');
-- THEN LOG OUT AND COME BACK HERE
SELECT * FROM ash_tbl;
DROP TABLE ash_tbl;
DROP USER ash_user;
USE information_schema;
DROP DATABASE ash_db;
CREATE USER 'ash_user' IDENTIFIED BY 'ash_pass';
GRANT ALL PRIVILEGES ON ash_db.* TO ash_user;
-- LOGIN AS ash_user THEN EXECUTE THIS...
INSERT INTO ash_tbl (id,name) VALUES(2,'Burton');
-- THEN LOG OUT AND COME BACK HERE
SELECT * FROM ash_tbl;
DROP TABLE ash_tbl;
DROP USER ash_user;
USE information_schema;
DROP DATABASE ash_db;
