2009 July

Web Sharing Questions
07-31-2009 by dan
Why develop anything for the web if it is locked into your proprietary framework? I’m not talking about your intranet, I’m talking about your publicly available (or maybe even subscription based) internet presence. In this day and age, anything you create that is not nimble enough for “mashups” will actually give YOU trouble as well as your customers. For example? What if you wanted to do summary statistics and unit tests of your database right now? Would your developers have to start coding like mad? I know the ones on my database would (mostly because that would be me).

Why would a nimble architecture help here? Because you could consume the data in some ready made statistical software (R anyone?) and do some quick analysis without new custom queries or development work. Same thing with unit tests — pull out specific sets of data with your agile framework, and test them with whatever software you like.

Here are my questions about developing this sort of framework:
- How to get your database browse-ready in a RESTful way? What is the software involved? Do you need special Apache plugins?
- How to get applications you do choose to develop to be able to consume your data properly — Do they consume a feed? Browse the RESTful setup? Both?
- If you build an app, how do you make it ultra-portable? Youtube videos, google gadgets, site searches, etc are all easily plugged into a website, and if you provide the proper XML attributes, you can even reskin them so your clients can easily keep their own look and feel with your technology. What is the technology involved here? Is it simply a matter of bundling attributes in HTML/XML and source javascript, with access to your web API? Or is there more?
- Scalability! Do any of the above questions impose a ridiculous amount of overhead on your database or webserver?
These are questions I don’t yet have answers to. I’ll write here as I find answers.
Tags: Business, Computing, Service | Comments (0)

Interesting Articles for July 31st

by dan

	Shared Unemployment, Not Bubble Unwind, Starting to Dominate Foreclosure Activity
	Shared TARP Index is Down $148 Billion
	Shared "All I do is work here"
	Shared Advice on writing research articles
	Shared Telling Big Earnings Lies is Easy
	New blog post: Web Services for Data http://dandube.com/blog/?p=293 #fb [#]
	Shared Stevey’s Home Page – The Five Essential Phone-Screen Questions
	Shared Daily Show — So you think you can Douche
	Shared I now pronounce you monetized: a YouTube video case study

Comments (0)

Web Services for Data
07-30-2009 by dan
The problem:

I have a large data warehouse stored in an Oracle database. The existing framework for “consuming” this information is very old and static.

The question:

Can I take this rich dataset and construct methods for extracting the data that will be extremely flexible, and run parallel to the existing static framework (so as to no break what is already there)? What I am looking for is something like a web service, which accepts a basic set of parameters and returns some kind of data “object”. It should be generalized so that I can ask for one data element, or a long list of data elements.

Why do I want to do this? To put it simply, it is too difficult to extract information from this database in the current form. All custom queries must be constructed off-line, and require a large effort to get into the existing framework. What I’d like is to supply a set of data “building blocks” that can be “mashed” together to create reports, summaries, unit tests, and new datafeeds that I hadn’t explicitly defined at the outset. Even better would be the ability to also pull blocks from outside sources — like salesforce.com or google finance.

So, I’m now researching the best way to provide the data “building blocks”. I can see some of these individual blocks becoming quite large — someone might ask for a time series of daily transactions over the past 10 years — and so a “pull” architecture probably makes the most sense. If I am pulling a big block of data out of the system, what format should I transmit it in? Currently, most of the data is viewed in the tried-and-true table format of a database or spreadsheet. Keeping it as CSV, then, would be logical, but I want this system to be flexible — it might provide a CSV by default, but it should also provide XML or JSON on request. (Can I transfer in a compressed format, and decompress on the other end?)

It should be obvious where I am going with all of this — if I can pull building blocks in whatever configuration I want, then I can insert them into whatever system I want — including the old static system. Obviously, I’d like to completely supplant the old system, but I will have to build toward that.

The other advantage to this approach will be more transparency about what is actually in the data catalog. A side effect of a well designed pull framework will be a somewhat self-documenting catalog of what is actually available and how it was constructed. This should also allow me to design natural unit tests for verifying the integrity of the data (and the framework).

Is a web service the correct solution here? It should be universally available to any device, permission-able, and as fast as possible. The output from the framework should be completely separate from any service that consumes the data. Do web services fit this mold?
Tags: Cloud, Computing, Formatting, Service | Comments (5)

Interesting Articles for July 30th

by dan

	Shared Toyota iQ Font: When Driving a Car Becomes Writing
	Shared A Comment on Seasonal Adjustments
	New blog post: Agile Database Integration http://dandube.com/blog/?p=278 #fb [#]
	Shared Domain-Driven Design
	Shared InfoQ: DDD: putting the model to work
	Shared InfoQ: Eric Evans on DDD: Strategic Design

Comments (0)

Agile Database Integration
07-29-2009 by dan
I attended the BTV PHP User Group meeting last Thursday where I listened to a talk about using a better data model for website development. I couldn’t agree more with the content.

I hate being locked into a corner by the database scheme, and even more by the code that is using the database scheme. The slideshow addresses the latter problem, not so much the former (although it does touch on it briefly). My realm of influence is definitely closer to the data than the code, but I am painfully aware of the interaction between them, and the restrictions each place on each other. So, while I am not a PHP developer, this topic is very interesting to me, and I am considering pursuing some of the books that were mentioned about developing database-smart systems.

What this presentation did not get into, however, is what to do if something in your schema DOES have to change. What if you can’t “decorate” your classes, because some fundamental change has occurred in your data (or your understanding of your data)? I suppose you can just grep for every occurrence of the old schema details, and replace with the new details. I want an application that manages the structure of the service ALONG with the structure of the database itself, and is completely dynamic from that perspective. Designing with this layered approach is definitely a step in this direction, but it doesn’t quite get me the whole way there.
Tags: Cloud, Computing, Data Collection, Presentation | Comments (0)

Interesting Articles for July 28th

07-28-2009 by dan

	Shared Guest Post: Is the Problem the Models or the Modelers?
	Shared Incredible Shrinking Boomer Economy
	New blog post: Interesting Articles for July 28th http://dandube.com/blog/?p=286 #fb [#]
	Shared Aquaponics arrives in Buffalo
	this gives me wanderlust: http://bit.ly/TjK9Z [#]
	water in the last one reminded me of this: http://bit.ly/74OhL [#]
	witnessed firemen come to put out someone’s camp fire at leddy beach in #btv last night [#]
	RT @billsimmon Wow. The Bluebird Tavern really is that good. Super yum. #btv — couldn’t agree more! [#]
	Shared Four short links: 28 July 2009
	Shared Makers and Managers: What You Are, and How It Can Help Your Career
	New blog post: Customer Service Problems http://dandube.com/blog/?p=284 #fb [#]
	Shared More Ticks, More Misery – Room for Debate Blog – NYTimes.com
	Shared YouTube – aquaponics systems
	Shared Matthew Yglesias » The Powers That Be
	Shared YouTube – William Shatner Reads Sarah Palin Farewell Speech

Comments (0)

Customer Service Problems
by dan
Yesterday, I was trying to buy a plane ticket on jetblue.com using my browser of choice, Firefox 3.5. I ran into a security problem just as I was about to buy the ticket — Firefox stopped the transaction and told me that “despite being connected via an encrypted channel, the information you are about to transmit will be sent unencrypted and will be easily seen by onlookers”. I am paraphrasing Firefox, but that was the essence of the message. So I stopped. Hell no! I’m not submitting my credit card information if my browser is about to send something unencrypted.

I called customer service and had to fight through 3 representatives to get to a supervisor. The supervisor was very reluctant to help me, but I eventually talked my way to an IT person. The IT person wasn’t the right guy, either, but he at least suggested an alternative (and told me that jetblue only supports IE — really? Your site is public! You need to support ALL browsers!). Here’s a summary that I sent to jetblue’s complaint form about what happened:

Website Security Problem

Hello. I have booked flights on jetblue.com several times using the Firefox web browser. Today, I tried to book a flight and got a security message — a very concerning one.

I am using Firefox 3.5.

I chose a flight, chose my seats, and entered all my information. I got to the payment screen, and put in my Mastercard number and related info. I clicked Submit.

My browser stopped the transfer somewhere between your site and the site it was trying to contact, though, because some information was about to be sent on an UN-encrypted channel, despite the fact that I was connected to your site securely.

I talked with customer service, and eventually even to a very helpful IT person at jetblue.com, who advised me to fill out this form. No one was able to say anything about this problem, and no one had any information about WHAT was being transmitted from jetblue.com in an UN-encrypted fashion. I was told to try Internet Explorer to fix my problem.

I used Internet Explorer, and the problem did indeed disappear. HOWEVER, the larger issue is this — what is your site transmitting that is not encypted? Did Internet Explorer not check for this, and Firefox caught it? Or is it simply a bug with viewing the site with Firefox?

I think this is a pretty urgent matter, because if you are transmitting credit card numbers on un-encrypted channels, you could be providing phishers with personal data.

I really think that the application was making a mistake here. Something that shouldn’t have been sent unencrypted was being sent unencrypted (I fully concede that it may NOT have been my credit card information, but that’s not a risk I’m willing to take!).

The challenge is that customer service was NOT the right set of people to hear about this problem, I should have talked to someone on the website team. Nowhere on the jetblue site is there any “technical problem” hotline, though, so the customer service people all told me to ignore it, since “our site is very secure”. The site may use very secure protocols, but you can still have a bug in the system that allows communication via an unsecured connection, which renders your “very secure site” very un-secure. I hope that someone is on the other end of the complaint form, and will forward my message along to the web team — could be bad news for jetblue customers otherwise!
Tags: Business, Security, Service | Comments (2)

Interesting Articles for July 28th

by dan

	Shared An Incoherent Truth
	Shared Merle Hazard Video: Bailout
	Shared More Bank Failures in the Last Five Weeks Than in All of Last Year
	Shared Watch The Divergence Between Earnings and Revenues
	Shared Practical Object Oriented Models In Sql
	New blog post: WordPress Add-ons http://dandube.com/blog/?p=252 #fb [#]
	Shared The Big Picture – How Semantic Technologies Introduce a New Paradigm for Interaction
	Shared New Media Office Hours (East)
	New blog post: For Technorati http://dandube.com/blog/?p=279 #fb [#]
	Shared How Much Is the World Pooping? – Twttrpoop – Gizmodo
	Shared Option ARMs: Good News, Bad News
	RT @gagepac Sense & Sensibility & Sea Monsters!! http://tinyurl.com/m8gwde Pride & Prejudice & Zombies!! http://tinyurl.com/dx637n [#]
	had fun at the #btv new media office hours [#]

Comments (0)

For Technorati
07-27-2009 by dan
Here is the technorati confirm code: ra5tikn8gm
Comments (0)
WordPress Add-ons
by dan
This post is to document the stuff that I have done to WordPress to make it work the way it does on my site. More for my reference than anything, but you might find it useful.
- Google Reader widget — this is the sidebar list I have under “Stuff I just read”
- Twitter Tools — this is the sidebar list I have under “Stuff I’m doing”. It also tweets everytime I post a new blog entry, and has the option to do a tweet digest post on a regular basis.
- Delicious Linkroll — this is actually just a text widget I added to the sidebar (under “Stuff I’ve Bookmarked”), with the following code as the text:
  .delicious-posts li { list-style-type:circle; margin-left: 40px; } .delicious-posts ul { margin-left: 0; padding-left: 2.5em; } <script src="http://feeds.delicious.com/v2/js/dandube ?title=&count=5&sort=date" type="text/javascript"> </script>
- Google Analytics for WordPress — you have to have some kind of tracking software, and in my opinion, this is the best. Make sure you disable tracking the admin, so you don’t end up just watching yourself look at your own website!
- Add to Any — I hate when I see an article out on the web and can’t just add it to my Delicious account with the click of a button. This solves the problem, but for ANY service.
- Tweetmeme – This one is somewhat redundant with Add to Any, but I like seeing the tweet counts.
- Gravatar Signup — if you have a gravatar and are posting to my site, you should be able to use it!
- Lifestream — this creates a digest post on a regular basis of all the services you wish to post updates from. I use it for Google Reader, Twitter and Delicious primarily, but it literally does EVERYTHING. I like having an archive of all this stuff in one place (ie, my blog).
- Lifestream CSS hack — The lifestream items were sometimes overflowing the primary content DIV, so I added this style to my CSS file:div.lifestream_label { width: 450px; }
- Hack to Twittertools to use Facebook “selective status” — In the file called YOURBLOG/wp-content/plugins/twitter-tools.php, find the line that says this: $this->tweet_format = $this->tweet_prefix and add #fb to the end of the string, like this: $this->tweet_format = $this->tweet_prefix.': %s %s #fb'; Now, Facebook selective status will pick up your new blog post tweets as your status, even though it is usually ignoring tweets.
Tags: Formatting, Meta, Wordpress | Comments (0)