Why ETL Tools Might Hobble Your Real-Time Reporting & Analytics

Companies report large investments in their data warehousing (DW) or Business Intelligence (BI) systems with a large portion of the software budget spent on extract, transformation and load (ETL) tools that provide ease of populating such warehouses, and the ability to manipulate data to map to the new schemas and data structures.

Companies do need DW or BI for analytics on sales, forecasting, stock management and so much more, and they certainly wouldn’t want to run the additional analytics workloads on top of already taxed core systems, so keeping them isolated is a good idea and a solid architectural decision. Considering, however, the extended infrastructure and support costs involved with managing a new ETL layer, it’s not just the building of them that demands effort. There’s the ongoing scheduling of jobs, on-call support (when jobs fail), the challenge of an ever-shrinking batch window when such jobs are able to run without affecting other production workloads, and other such considerations which make the initial warehouse implementation expense look like penny candy.

So not only do such systems come with extravagant cost, but to make matters worse, the vast majority of jobs run overnight. That means, in most cases, the best possible scenario is that you’re looking at yesterday’s data (not today’s). All your expensive reports and analytics are at least a day later than what you require. What happened to the promised near-realtime information you were expecting?
Contrast the typical BI/DW architecture to the better option of building out your analytics and report processing using realtime routing and transformation with tools such as IBM MQ, DataPower and Integration Bus. Most of the application stacks that process this data in realtime have all the related keys in memory –customer number or ids, order numbers, account details, etc. – and are using them to create or update the core systems. Why duplicate all of that again in your BI/DW ETL layer? If you do, you’re dependent on ETLs going into the core systems to find what happened during that period and extracting all that data again to put it somewhere else.

Alongside this, most organizations are already running application messaging and notifications between applications. lf you have all the data keys in memory, use a DW object, method, function or macro to drop the data as an application message into your messaging layer. The message can then be routed to your DW or BI environment for Transformation and Loading there, no Extraction needed, and you can get rid of your ETL tools.

Simplify your environments and lower the cost of operation. If you have multiple DW or BI environments, use the Pub/Sub capabilities of IBM MQ to distribute the message. You may be exchanging a nominal increase in CPU for eliminating problems, headaches and costs to your DW or BI.

Rethinking your strategy in terms of EAI while removing the whole process and overhead of ETL may indeed bring your whole business analytics to the near-realtime reporting and analytics you expected. Consider that your strategic payoff. Best regards in your architecture endeavors!
Image by Mark Morgan.

When to Automate BPM: Top 5 Questions

As we wade even deeper into the digital economy, workload automation has become more of a necessity and less of an amenity. Customers and clients expect you to be able to produce results in real time, and a more robust BPM (business process management) automation tool can get you there. Gone are the days when BPM tools were simply helpful, now they are essential for every industry.
Banking: Your customers need to manage their accounts online and receive accurate information about their finances.
Utilities: Predicting and preventing blackouts will keep your customers satisfied and detecting meter fraud will boost your bottom line.
Sales: Monitor and adjust custom-designed analytics that help you target the right audience with the right message.
IT management: Automate data-intensive or computing-intensive processes to boost the efficiency and accuracy of your IT department.
Internal workflow: Bridge the gap between business and IT with workflow that moves more fluidly between employees and departments.
A more robust BPM tool will help you align business processes more effectively with your customer expectations. If your company is struggling to keep up with customer demands and adapt to changing technology, BPM could help your business grow.
When is it time to consider using a more robust BPM automation tool?

  1. Multiple User Access: Does your business process span departments and route to various user roles?  Are there multiple types of user authority? Are there multiple ways (fax, email, mobile app, etc.) to trigger the process?  Do you need to allow for multiple types of user access (view, update, admin)? How many users will require access?
  2. Integration: Is there a need for integration with other applications? Does the process need to access or update multiple data sources? How and where is the data captured and stored? How much data and in what form is data being passed from step to step in the process?
  3. Complexity: Does it take more than 2-3 steps to complete a process? Does the information have to be escalated to more than one approver? Are there many roles or actors involved? Are there multiple inputs that trigger the business process? Are there many exceptions? Are there many decision points?
  4. Monitoring and Analytics: Is there a need for visibility to track and monitor the progress of the process? Do you want to measure the process based on cost and time? Are there KPIs you want to view? Is there a need for insight to each of the process instances?
  5. Room for Growth: Is there a need to have the environment scalable? Based on the number of users and the amount of data and other applications interfacing with the process, is performance important? Is it important to have the environment portable to different platforms?

Companies that successfully implement BPM automation tools typically experience a number of benefits including increased productivity, improved metrics, insight into operations and reduced costs. Careful pre-planning for BPM automation will ensure a more successful and cost effective implementation.

The Business Value of SOA (or How to convince your line-of-business executive that integration is worth the cost)

In this digital economy, your company cannot afford to be hindered by inflexible IT. As we throw around phrases like “vendor lock-in” or “vendor consolidation,” the truth is, successful business thrives on harnessing the power of the next big thing. Sure, you might have one option today, but tomorrow you could have one hundred or one thousand. That’s why integration strategies are vital to both business and IT growth. One way to ensure your multiple programs, systems and applications integrate effectively is through SOA.

Service Oriented Architecture (SOA) Defined:
To an IT professional, SOA integrates all your systems, so that different programs and applications, all from different vendors and running on different machines, can communicate smoothly and effectively. This also means that your legacy infrastructures can coexist with your new cloud services.
To a business executive, SOA creates a more competitive business edge by improving the efficiency of collaboration between business processes and IT. SOA drives growth by boosting productivity, enhancing performance and eliminating frustrations with IT.

LOB executives command the budget; therefore, they wield ultimate decision-making power when it comes to purchasing the software and hardware to meet your IT needs. Engineers, developers and IT talent inhabit the nooks and crannies, so they must prove that these nuts and bolts translate into profits. Looking to convince your LOB executives to open the coffers a bit wider? Here are a few ways to add some business value lingo into your tech-heavy talk.
Bottom Line Value of SOA:

  • Integrates with current infrastructure: SOA means you can keep your mainframe and leverage existing legacy applications. IT developers can build additional functionality without having to spend thousands rebuilding the entire infrastructure.
  • Decrease development costs: SOA breaks down an application into small, independently functional pieces, which can be reused in multiple applications, thus bringing down the cost of development.
  • Better scalability – Since location is no longer an issue, the service can be on-premise, in the cloud or both. SOA can run on different servers. This increases your company’s ability to scale up to service more customers, or scale down if consumer habits change.
  • Reduce maintenance and support costs – In the past SOA could get costly, but now services like IBM’s Enterprise Service Bus can bring down those operating costs significantly. New capabilities can be delivered quickly and more efficiently.

So after you’ve spent your lunch hour elegantly wooing your LOB executive, consider partnering with TxMQ to execute your SOA needs. We can help at any point in the process, from assessment to deployment, and even maintenance and support. Request your free, no obligation discovery session and see exactly what it will take to boost productivity and profitability with a more secure and agile integration of your various applications.

IBM MQ For Click And Collect Retail

Few industries have gone through as much change as retailing. Every retailer in business today is dynamic. That’s the only way to survive – especially because the space continues to change rapidly with innovations such as multi-channel retailing, leaner and more responsive inventory management, and the new click-and-collect phenomena.
Click and collect is sometimes called “click and mortar” because a customer shops and buys online, but opts to collect the order in-store. It beats waiting (and paying) for the postman, it saves browsing time in the store, and shoppers can quickly research products and reviews.
Click and collect is scaling rapidly, but in order to offer the service, both the retailer and the customer must be able to access real-time stock and delivery schedules. Then, the item must be reserved for the customer at the instant of purchase.

…MQ Advanced is a mobile enabler, because of its quality-of-service message delivery, as well as the baked-in, lightweight MQTT protocol to support always-on push notifications that don’t hog battery or data.

That’s where IBM MQ for click and collect comes in, because MQ Advanced supports more reliable asynchronous queries to stock and information at all stores in the network. Thus stock is updated accurately and reliably in real-time via transaction coordination. And remember that many customers will be accessing the storefront via mobile, which creates its own set of problems. But MQ Advanced is a mobile enabler, because of its quality-of-service message delivery, as well as the baked-in, lightweight MQTT protocol to support always-on push notifications that don’t hog battery or data. (MQTT was originally designed for small, unreliable sensors for things like oil pipelines and machinery. It was adopted as the open Internet of Things standard and also powers Facebook Messenger.)
I should also point out the fact that retail is a great example of an industry sector that needs bulletproof, foolproof IT spread across geography. The same system must run in every store, it must run well, and it must be accessible by folks who aren’t always tech savvy. This has generally led to file-based solutions for retail IT, and the resultant need to batch process. But today’s systems, as noted above, must trickle-update based on real-time transactional data. That means stores must process data more quickly and move the data into the enterprise much more rapidly, and central system must update in real-time. The answer, again, is already within MQ Advanced.
Managed File Transfer is native inside MQ advanced, which means the point-of-sale files can move into the enterprise over an IBM MQ network with secure, reliable, traceable, guaranteed delivery.
According to IBM, a large US grocery retailer batch processed its data, which created a delayed analysis and made it difficult to detect theft. Using IBM MQ, IBM MQ Managed File Transfer and MQTT, and the company’s data warehouse now receives near-real-time transaction data from 2,400 different stores.
TxMQ is widely regarded as one of the premier MQ solutions shops in North America. We’ve been in business since 1979. Our customers typically start with our free, no-obligation discovery session. Want to improve your enterprise application performance? Want to know about how MQ is driving the digital economy, and how you can climb aboard? Let’s get the conversation started. Click Here For Our Free Discovery Session Offer.
(Image from DaveBleasdale)
[feature_headline type=”center” level=”h5″ looks_like=”h5″]Cool MQ-Related Content From TxMQ[/feature_headline]
[recent_posts type=”post” count=”6″ section=”mq” orientation=”vertical” fade=”true”]

DataPower Firmware V7.2 Upgrade Details (More Cloud!)

One of the coolest things about IBM DataPower is how it provides a single security and integration gateway solution using a simple, policy-driven interface. And it’s ultra-flexible, so it’s easily deployed to multiple cloud environments. And that’s really the trick, isn’t it? The need to deploy across today’s growing mix of public, private and hybrid cloud mixes. It’s not as easy as it seems.
The good news is that the newest DataPower Gateway Firmware V7.2 upgrade extends multiple-cloud support with enhanced security and integration capabilities for mobile, API and web workloads. Want to run DataPower to gate Amazon Web Services? Great, this firmware update is definitely for you.
Cloud environments like Amazon AWS and IBM SoftLayer are hosting environments that allow a business to deploy an optimal set of supported infrastructure resources. Using DataPower, businesses can then quickly build gateway solutions and rely upon the cloud provider to maintain the underlying infrastructure. The result is an overall lower cost to deploy and secure workloads (especially mobile workloads), plus additional flexibility for faster responses to market trends and business opportunities.
The DataPower Firmware V7.2 enables this type of process improvement through several key upgrades:

  • Use GatewayScript to easily integrate Systems of Record with Systems of Engagement
  • Deploy IBM DataPower Gateways on Amazon Elastic Compute Cloud (EC2) and IBM SoftLayer CCI
  • Enhanced hybrid cloud integration: Use the Secure Gateway service to securely connect between IBM Bluemix applications and on-premise services secured by DataPower Gateways
  • Enhanced message-level security for mobile, API and web workloads with JSON web encryption for message confidentiality, JSON Signature for message integrity, plus JSON Web Token and JSON Web Key
  • Protect mission-critical applications with enhanced TLS protocol using Elliptic Curve Cryptography (ECC)
  • Build automation, deployment and migration scripts more quickly using the new Representational State Transfer (REST)-based management API
  • Enjoy the increased high availability with enhanced load-balancer group healthchecks
  • Support for IMS Commit mode 0 for more flexible integration with systems of record.
  • IBM WebSphereeXtreme Scale 8.6+ support for distributed caching

TxMQ specializes in DataPower upgrades and services. Need some quick advice about deploying this firmware upgrade into production? Want to decouple from core IT and speed your time to market? Want to integrate your systems of record with your new systems of engagement? That’s what we do. Let’s start a conversation today: Click the chat icon below, or email our president [email protected].
(Image from Paul)

How IBM MQ v8 Powers Secure Cloud Integration

In this quickly growing digital economy, where we have an increasing demand on things like security, cloud and mobility, IBM MQ has been growing to meet those demands. To pick two of the three topics, MQ v8 can deliver secure cloud integration straight of the box.
It is important to know what type of cloud are we’re really talking about. Are you talking about moving all of your services into the cloud – even your virtual desktops? Or are you talking about a hybrid cloud where with a mix of cloud computing supplementing your own services? Or are you talking about a private cloud, where you’ll have segments of internal computing services totally isolated from general services. There are different considerations for each scenario.
Regardless of the type of cloud-computing services you’re using, you still need to integrate these services, and you really need to ensure that your integration has security, data integrity and the capability of sending messages once-only with assured delivery. Cloud can’t provide that. MQ can and does. And it does if out of the box with several recent enhancements to ensure secure integration.
With the digital economy, we’re all sharing all this data, including personal data, banking and health data. We need to keep this data secure when it’s being shared, and control who has access to it. Then of course there’s the large compliance piece we need to meet. How does MQ meet all these demands? The answer is authentication, and MQ’s solution is still the same as being asked for ID of proof at the post office when you go to pick up a package. MQ v8 has been enhanced to support full user authentication right out of the box. No more custom exits and plugins.
For distributed platforms, you have local OS authentication, or you can actually go to centralized data. For z/OS you’re still focused on local authentication.
And this next point is important: MQ for quite some time has supported certificate authentication of applications connected to MQ services. But this always meant that the public MQ key had to be shared with everyone. MQ now has been enhanced to support the use of multiple certificates for authentication, securing of connections and encryption using separate key pairs. MQ still support SSL and TLS, although there are strong recommendations for switching from SSL to TLS based on the POODLE vulnerability.
(Image from mrdorkesq)

4 Ways BPM Can Boost Healthcare Revenue

Healthcare – it’s the digital era’s slipperiest slope. On the one hand, technology can greatly enhance the way we document and share patient data. On the other hand, since digital health records are vulnerable to security breaches, there are complex regulations about how providers record this data.
What does this dichotomy add up to? A cluster of complicated conundrums, that’s for sure. Unused data swells up behind bottlenecks, patients don’t have enough access to information, hackers have too much access to information and confusing legislation gets churned out constantly to try and smooth out the process. On top of all that, there are old school providers who not only have trouble digitally documenting encounters, they have trouble even understanding why they’re supposed to do it this way in the first place.
That’s why I think it’s important that healthcare organizations consider using BPM (business process management) solutions. The only way to sort out the mess is to streamline your workflow, so that the decision-makers in your organization can analyze the entire landscape of your healthcare documentation process. Here are some ways BPM can help:

  • Take advantage of data analytics
  • Respond to tech and policy developments in the industry
  • Plan for and meet important state and federal deadlines
  • Manage software applications used in routine encounters
  • Coordinate information exchange, which improves quality of care
  • Get different providers and team members on the same page

So that’s a nutshell version of how BPM software can improve efficiency, streamline workflow and promote collaboration in healthcare. But what does that mean for your company? How does that translate into more revenue?
4 Ways BPM Can Boost Healthcare Revenue

  • Reduce documentation and billing errors – Less errors means more payments and better reimbursements. With a streamlined workflow you don’t have to stress out about losing income because your providers struggle with their EMRs.
  • Earn incentive payments – Payment reform is a reality. The more valuable care you provide, the more opportunities you have to receive incentive payments from the private and public sector.
  • Boost patient satisfaction – When your patients are happy, they spread the word. If your patients are unhappy, they spread it even louder. Satisfied patients improve your reputation as a provider focused on quality care.
  • Claims Processing — BPM can help you smoothly handle claims, compliance and reporting activities. You can issue invoices and receive payments in a more timely manner.

If you think BPM could help your healthcare organization unwind a few knots, you have a lot of options. However, if you find you’re short on time or personnel, consider contacting TxMQ. Our capability partnership means that we offer an initial analysis to determine your needs and then develop and deploy the right BPM solution. We also have access to highly-trained technical professionals that you can contract to maintain your solution or even train your employees on how to use it efficiently.
 

API Management Enables Dynamic Systems of Engagement

When I first heard the term API Management, my mind went back to the mid 1990’s when I was neck deep in object-oriented programming.   Ugh, I remembered combing through reams of documentation about application programming interfaces, object structures and the like. It wasn’t necessarily the fun part of developing applications. Back then, the context was connecting core application systems together, exchanging transactional and master data between applications such as CRM, ERP, and Supply Chain.

Nowadays many of the underlying technical details remain relevant, but the context has changed from API’s as a systems development concern to API management as a business model consideration and key topic in the board room. With innovation and speed to market as primary concerns among executive ranks, key enablers such as API Management solutions are getting due attention to help unlock the creative potential of next-gen application developers, both internally and externally, and to even develop new revenue streams.

From an external perspective, API management solutions can provide a welcoming experience for customers, business partners, and communities. Hackathons have exemplified the notion of turning loose a group of talented individuals to bring a fresh perspective on how core IT assets can be turned into something new and useful in ways that perhaps would have never been conceived internally. These new systems of engagement need to plug into core IT assets in a way that is securely controlled and measured, yet easy to access and understand in a self-service approach. Done well, API management provides the virtual welcome center for a self-guided tour.

From more of an internal approach, one of the trends that reinforces the need for API management is the growing tension between line-of-business and IT, which is typified by LOB-lead projects that attempt to minimize dependencies on the core IT organization. This has been described by industry analysts, such as Gartner, as the emergence of Bimodal IT where the business intends to drive more dynamic systems of engagement that give them some degree of freedom from the back-end systems of record that are heavy and slow to change. In this case, API management serves as the clean hand-off point that can help streamline the relationship between LOB-lead projects and core IT.

As with most any technology solution, realizing effective API management requires careful planning and perhaps some non-traditional IT skills. The rendering of IT services as API’s is not necessarily the hard part. Defining internal and external-facing processes to manage the lifecycle of API usage will be an important factor in the long term value proposition. What’s more, solid technical writing skills can be crucial to making or breaking the developers’ experience as they discover and embrace the IT assets that are made available. Nonetheless, a simplified pilot can be an effective entry point to get started on the journey.
TxMQ works with companies of all sizes and shapes to harness the potential of the digital economy. Get in touch today for information on how we can partner with you on your digital evolution.
(Image by opensource.com)
 
 

White Paper: SmartGridSB Solution Delivers Real-Time Visibility & Actionable Insight For Utilities Sector

Utilities companies have widely embraced smart meter technology, but often fail to implement networking tools and analytics to harvest actionable data from this new smart meter network.
Smart Grid Smart Business (SmartGridSB) delivers a real-time solution that empowers energy and utility companies to automate proactive, cost-saving decisions about infrastructure and operations. These automations are based on the integration of smart meter networks with analytic software solutions, which drive event correlation and decision management. The SmartGridSB solution provides visibility, insight and analytic awareness for smart grid operations. It streamlines information within the operational processes to identify patterns and analyze data that enable you to make the best response and then implement the correct action in less time.

[/et_pb_text][/et_pb_column][/et_pb_row][et_pb_row][et_pb_column type="4_4"][et_pb_text admin_label="Text - Want the %91white paper name%93?" _builder_version="3.0.98" background_layout="light"]

Want the full SmartGridSB Solutions white paper?

Fill in the form below to get it delivered right to your inbox:

[/et_pb_text][/et_pb_column][/et_pb_row][et_pb_row][et_pb_column type="1_4"][/et_pb_column][et_pb_column type="1_2"][et_pb_code admin_label="Code - HubSpot Form" _builder_version="3.0.98" background_color="rgba(86,86,91,0.08)" text_orientation="center" animation_style="fold" custom_padding="30px|20px|30px|20px" custom_css_main_element="border: solid 3px #199ad6;"]<center><!--[if lte IE 8]><!-- [et_pb_line_break_holder] --> <script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2-legacy.js"></script><!-- [et_pb_line_break_holder] --> <![endif]--><!-- [et_pb_line_break_holder] --> <script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2.js"></script><!-- [et_pb_line_break_holder] --><script><!-- [et_pb_line_break_holder] --> hbspt.forms.create({<!-- [et_pb_line_break_holder] --> portalId: '2682341',<!-- [et_pb_line_break_holder] --> formId: 'c950b59d-17a5-4865-a3d6-05ddc2492844'<!-- [et_pb_line_break_holder] --> });<!-- [et_pb_line_break_holder] --></script></center>[/et_pb_code][/et_pb_column][et_pb_column type="1_4"][/et_pb_column][/et_pb_row][/et_pb_section]