Send As SMS
The Industrial Science Blog: Complexity Science, Simulation, and Business
Thursday, September 08, 2005
 
data data everywhere, and not a byte to eat
In every project we do, there is the "data" phase. In order to do our analysis, build/test/run the model, show results, before any of that is possible, we need data.

Many of you know what this REALLY means. Data is never innocent nor clean. It always comes with some story, or hidden skeletons from many parts of the organization. We're always relieved when we hear that data is available; actually getting it and understanding what it really means is another story altogether.

There have been incredible advances in recent years that have made "getting the data" much easier. Thanks to zip files and the "power" of XL, we can query, sort, pivot table, and provide all kinds of data. If we go beyond the magical 65,536 lines in XL, we can split up the data in different tabs/worksheets, or go with Access or MySQL.

We also have these wonderful "thumb drives" or USB drives. And we can now burn CDs. Simply amazing! No more "sneaker-net"-ting the 2MB disks around, or trying to line up the infra-red ports on laptops. Thanks to all the $ weve spent on ERPs, we have LOTS of data. And thanks to ftp sites, servers, shared directories, USB ports, VPNs, it's easier than ever to get data from here to there.

Volume is nice, and thanks to our ever-increasing HD space, we can store more and more stuff. So what? Have we gotten better at understanding what our organizations? How they work? The needles may not have increased, but the haystacks have certainly gotten larger. In fact, we often find that we have to limit the data gathering effort so we can get on with the rest of the work. It sounds crazy sometimes, yes, there is more data available, but we have to STOP gathering the data.

Visualization helps, but even here, the trick is to LIMIT what you try to show. Technology is nice, but it no longer is the constraint to what we're trying to do with data. It's often our ability to grasp, understand, and communicate. And here, sometimes, less is more.
- posted by hcpark @ 10:25 AM  
Comments: Post a Comment

<< Home