We thought that it was a table size issue, may be the driver could not handle huge data. However, the problem was with the “unique record identifier” while linking the table, as JM reported here. I did not have this problem any longer when I did not select any fields for the unique record identifier while linking the table.

The data came from this report came from a complex query using SQL server and Access that had this format:

Some options I investigated and tried (be sure to read this page, you might find many more):

• Google’s Chart API (http://code.google.com/apis/chart/types.html): you can embed these in Google spreadsheets, create HTML pages using Excel via VBA, or embed them in Excel sheets. Alas, none would work the way I would like them to work: In-cell graphics
• Use Sparklines for Excel add-in: this add-in will create great in-cell charts (bar, bullet graphs, sparklines, etc), but copying them down is difficult and resource intensive, and any change you make in the column size will alter the shape of that chart. In addition, I had more than 400 rows to populate–these would be too many objects in a spreadsheet for Excel to handle. My machine froze when I tried to copy it for 10 rows.
• Use R’s implementation by Jason Dieterle (search on this page for Jason Dieterle (email), January 28, 2008. Works very good. I modified the code to print max and mins only, but the function generates a graphic, which needed to be embeded in LaTex file. It did not work nicely. Charts were too big to fit in a cell of a table.
• Create bar charts in Excel using REPT function. I tried different font sizes and styles, but it didn’t look pretty, for the range of data varied. (Hint: make the alignment of text 90, use pipe signs, create columns and graphs for each data value, remove gridlines, keep the columns very close. It almost worked.)
• Use spark package for LaTex, doesn’t work in pdflatex, and you have to play a lot with the settings
• Use sparklines package for LaTex. Problem was that data needed to be normalized (or scaled) from 0 to 1, and needed extra parameters for min and max points. Solution: create a normalize function in Excel, and write a big formula to produce the exact needed string for the sparklines to work i.e.:
  \begin{sparkline}{5}
\sparkdot 1 1 blue
\sparkdot 0.2 0 red
\spark 0.2 0 0.4 0.0625 0.6 0.5625 0.8 0.75 1 1 /
\end{sparkline}

Here’s the normalize function:

Public Function Normalize(cell2Normalize As Range, WholeRng As Range)
       Normalize = (cell2Normalize.Value - WorksheetFunction.Min(WholeRng)) / (WorksheetFunction.Max(WholeRng) - WorksheetFunction.Min(WholeRng))
End Function

Here’s the big formula to convert the range for data values from 2005 to 2009 (in the range E2:I2) to the sparkline LaTex environment.

="\begin{sparkline}{5} " & "\sparkdot " & CHOOSE(MATCH(MAX(E2:I2),E2:I2,0),0.2,0.4,0.6,0.8,1) & " " & 1 & " blue " & "\sparkdot " & CHOOSE(MATCH(MIN(E2:I2),E2:I2,0),0.2,0.4,0.6,0.8,1) & " " & 0 & " red " & " \spark 0.2 " & Normalize(E2,E2:I2) & " 0.4 " & Normalize(F2,E2:I2) & " 0.6 " & Normalize(G2,E2:I2) & " 0.8 " & Normalize(H2,E2:I2) & " 1 " & Normalize(I2,E2:I2) & " / \end{sparkline}"

This last option worked beautifully. I dragged the formula down. I selected the data, and clicked on “Convert Table to LaTex” button (using this add-in). Copied the LaTex code to clipboard and pasted it in my LaTex editor. Manually merged the rows for colleges (using \multirow), and generated a beautiful looking pdf.

I was very happy. Printed it in color. Got a request back very soon that there should be total rows.

I forgot about the sparklines and created a report in Access with plain old numbers in less than 15 mins. Gave it back.

I tried to repeat this – Sparkline in Cognos -in Access with no luck.

(I later tried it one more time: Got the data in Excel using External data> Access, created a pivottable with rows and all, did some formatting, copied and pasted values and formats, inserted sparkline code, converted it to LaTex, copied and pasted in LaTex editor, and here’s the beautiful looking sample pdf of 16)

I wish there were simple reporting solutions that included awesome data visualization tools. (BTW, Excel 2010 will have sparklines: link) For this report, I did try Sweave, R, and LaTex, but because of the time constraints I could not investigate it further.

Please comment if you know any other way which meet (or don’t, Tableau is certainly one) these conditions: inexpensive (read free), efficient, and repeatable.

“The search key was not found in any record”

After some research, I found that either it was a bug caused by Microsoft Jet 4.0 Service Pack or it was the size of the database (should be not greater than 2GB). It turned out to be the later one. I fixed this problem by compacting and resizing the database. In Access 2007, you click on the Office Button (Top Left Corner), then Manage > Compact and Repair Database.

1. Create tables in Access (for small tables)
2. Paste Field names: either in a table or in the import specs form

When I am importing data of the web, usually they have field names and data types given that page, so it makes sense to import that table in Excel (using Import from Web option), and then copy the field names and directly paste them the import specs form in Access. However, I kept running in this error today, which MS acknowledged and provided a solution, caused due to a security patch.

I used a workaround to paste those field names: copy and paste the data in a text file, and the again copy and paste it in Excel.

Solution Update: Colleen provided one more MS KB number to fix this problem. So, there are two KB articles which acknowledge this problem:

KB967699: http://support.microsoft.com/kb/967699

KB958437: http://support.microsoft.com/kb/958437

You selected more records than can be copied onto the Clipboard at one time. Divide the records into two or more groups, and then copy and paste one group at a time. The maximum number of records you can paste at one time is approximately 65,000.

Access 2007 Export to Excel Error

I was unable to understand the reason, because I thought Excel 2007 did not limit number of rows to 65,536. It turns out however that it is the clipboard limitation.

Solution: If you check off the “export data with formatting and layout” option, Access exports successfully.

You can export successfully using Transferspreadsheet (VBA) option too.

Here is the KB from MS.

Solution Update:Thanks to Brett, it looks like that only if you export table/query using right click > Export and have the checkbox cleared, then Access will export succesfully. That is what I always do, and it works for me. I have never tried External data option for exporting.

This can very well be designed using forms, but this should get one started. Use Alt + F11 to see the code, then execute procedure ConvertTbl2Arff to convert a table.

Some highlights of this procedure
- Takes care of spaces in Attribute name and data values
- Finds unique values of nominal variables
- Assigns equivalent ARFF datatype
- Replaces missing values with question marks