Alvion Technologies

Managing Alvion’s Customer’s Big Data
Alvion Technologies, Inc. provides outsourced data management services to some of the largest marketing data list owners in the world – names like Axiom, Equifax, Experian, and Dunn & Bradstreet.  Their customers’ datasets range up to 190 million records, and 200 attributes.  Combined, Alvion manages about a terabyte of data for customers as an outsourcing service.  Individual customers submit anywhere from 5,000 specialized records of information, to 120 gigabytes of data.  Alvion then runs customer-specific data transformations and uploads it to their production servers for access by the end user – the customers of the original data owners.  Data update cycles vary from daily to annually.

Of this terabyte of data, roughly 550 gigabytes is held in Sybase IQ and can be queried via web-based storefronts Alvion hosts for the data owners. 
“We use IQ in production because its biggest strength as far as we are concerned are those really, really fast counts.” Bojan Belovic, Database Administrator.

End users of this data typically run queries and counts to determine optimal selection criteria for specific direct marketing programs.  This means that a typical user session includes numerous queries, run with slight variations, to find the strongest candidates for final data selection.  When users query over and over, faster is better.  Much better.

Alvion’s World-Class Data Warehouse Application Service
Alvion sits squarely in the crossroads of e-commerce and business intelligence, leveraging their exclusive technologies, extensive very large data warehousing (VLDW) expertise, and marketing know-how to provide customers with fast, reliable marketing data. 

Several years ago, Alvion was making news with high-capacity, high-speed and low-cost marketing list services.  But that combination was tricky to maintain.  To control costs, they started out with Windows NT-based production servers. Soon, they were in need of a quick save to make this infrastructure work at the high performance levels required for their self-service, online business model.

That’s where Sybase IQ originally came into the picture.  In fact, that early Sybase IQ pilot installation was up and running in three days, with benefits immediately apparent.  On a random set of previously submitted queries, Sybase IQ reduced large-list response times by a factor of six.  One notable query, previously taking 53 minutes, now completed in 3 seconds. Customers were happy.  Life was good.

“The response time is absolutely the number one reason we are using Sybase IQ,  We are talking about an order of magnitude difference.” – Bojan Belovic

Upping the Ante
Alvion boasts 12,000 registered users.  With over 2,000 ad hoc queries daily, Sybase IQ average response times were still under 20 seconds. 

For its growth in data volume, Alvion was suitably prepared due to Sybase IQ’s unique data compression.  With Sybase IQ, disk space savings add value, especially considering the cost of the large, high speed drives required for high-use VLDWs.  Plus, the extra drives typically used to tune-up database performance were unneeded.

Today, Alvion has 50,000 registered users.  Daily query loads have risen by 275% to 7,500, with one third of those very complex, demanding even more system resources.  As big data grew bigger, Alvion’s commitment to market leading response times forced an infrastructure build out.

The Challenge: Alvion’s Success
Growth in Alvion’s business means four things:

• More data hosted from new customers
• Growth in existing customers’ record counts
• Increasing user sophistication and query complexity, and overall data mining goals
• Four-fold end user growth

In the past, to keep pace with growth and maintain performance, Alvion would add Windows “reader” servers running Sybase IQ – a process Sybase calls “Multiplexing”.  Still, with the required data model, this left one soft spot.  Despite handling the big data and all the input/output, the “writer” server was subject to Windows’ two-gigabyte user space ceiling – inherent in its 32-bit design – which would eventually become a bottleneck.  Alvion needed to continue providing fast service and accommodate tremendous growth. 

Deploying the Sybase and Sun Reference Architecture
Although the NT platform had performed well, the data load run-time now forced their hand.  Nightly updates, necessarily confined to the single writer server, originally completed easily overnight.  Now, that critical data load stretched beyond 12 hours, motivating a hardware reexamination.

Consultation with Sybase field engineers and product experts turned up a recent joint effort by Sybase and Sun Microsystems that addressed hardware and software architecture for data warehouse applications handling raw datasets up to 100 terabytes.

“We looked at a couple of other vendors but Sybase definitely popped up as the best.” – Bojan Belovic

It’s the Infrastructure!
Alvion selected Sun hardware for maximum performance, while allowing headroom for growth with their present footprint.  The implementation includes a four-CPU Sun Fire™ V880 with eight gigabytes of RAM, a two-CPU Sun Fire™ V480 with four gigabytes, and two four-CPU Sun Fire™ V440s, each with eight gigabytes.

To accommodate current loads, Alvion now runs half of production on Windows and half on Sun, all happily accessing the same Sun Fiber Channel-based SAN.  The new Sun machines easily handle their increased customer query loads.  But most importantly, that Windows-imposed writer server bottleneck is gone.  That function now runs on the big V880, leveraging its eight gigabytes of RAM, finishing in half the time.

Bojan Belovic, Alvion DBA, sums up the basis for their Sun move: “If our datasets were smaller, with growth merely in queries loads, the Windows environment might have been appropriate.  We’ve scaled Sybase IQ simply by adding servers.  However, we needed to manage the larger data sets as well as increased query complexity.  That required a hardware platform change.  Fortunately for us, Sybase IQ is well suited for the Solaris environment so the software platform remained unchanged.”

What’s Ahead
Until now, Alvion divided its data warehouse tasks between queries and order fulfillment.  With the new Sun hardware in place, Alvion is looking to consolidate all operations there.  To do this, they will add the other half of their customers’ terabyte of data to the Sun-Sybase IQ servers.  Amazingly, Sybase IQ provides such effective compression that the new data will not impact the system’s performance or query speed.  However, this change will dramatically improve Alvion’s competitive advantage by shortening data fulfillment times, while maintaining the trademark query speeds so appreciated by its customers.

Belovic believes that Sybase IQ is up to that task, too.

“Our fast query times are what defines us in this industry.  We are building in more complexity with Sybase IQ as the backbone, while maintaining lightning-fast response times.  That’s going to take us a long way.”