We will present our Office 365 use case scenarios, why we chose Cassandra + Spark, and walk through the architecture we chose for running DSE on Azure.
The presentation will feature demos on how you too can build similar applications.
Muvr is a real-time personal trainer system. It must be highly available, resilient and responsive, and so it relies on heavily on Spark, Mesos, Akka, Cassandra, and Kafka—the quintuple also known as the SMACK stack. In this talk, we are going to explore the architecture of the entire muvr system, exploring, in particular, the challenges of ingesting very large volume of data, applying trained models on the data to provide real-time advice to our users, and training & evaluating new models using the collected data. We will specifically emphasize on how we have used Cassandra for consuming lots of fast incoming biometric data from devices and sensors, and how to securely access the big data sets from Cassandra in Spark to compute the models.
We will finish by showing the mechanics of deploying such a distributed application. You will get a clear understanding of how Mesos, Marathon, in conjunction with Docker, is used to build an immutable infrastructure that allows us to provide reliable service to our users and a great environment for our engineers.
C* Summit 2013: Searching for a Needle in a Big Data Haystack by Jason Ruther...DataStax Academy
The presentation demonstrates how Solr may be used to create real-time analytics applications. In addition, Datastax Enterprise 3.0 will be showcased, which offers Solr version 4.0 with a number of improvements over the previous DSE release. A realtime financial application will run for the audience, and then a detailed look at how the application was built. An overview of Datastax Enterprise Solr features will be given, and how the many enhancements in DSE make it unique in the marketplace.
DataStax recently announced the general availability of DataStax Enterprise 4.7 (DSE 4.7), the leading database platform purpose-built for the performance and availability demands of web, mobile, and IOT applications. In this product launch webinar, Robin Schumacher, VP of Products, explores the wide range of enhancements in DSE 4.7 including enterprise class search, analytics, and in-memory.
This document provides an overview of Cassandra data modeling concepts. It discusses Cassandra data types like collections (sets, lists, maps) and how to model different types of tables, including static, dynamic, and time series tables. It also covers primary keys, clustering columns, query patterns, and other Cassandra features like lightweight transactions and user defined functions. The overall document is a guide to understanding Cassandra data modeling fundamentals.
Being able to rapidly iterate on, build, and test your code is key to being a productive developer. Without local automation, working with the numerous platforms and technologies in your stack can become very frustrating. In this webinar, Ben Bromhead CTO of Instaclustr will explore best practices to easily integrate Apache CassandraTM into your development workflow, so you spend more time writing good code and less time fighting your environment.
Webinar: DataStax Training - Everything you need to become a Cassandra RockstarDataStax
The document outlines a training program from DataStax on Apache Cassandra, including an introduction to various courses that cover topics such as core concepts, operations and performance tuning, building scalable Java applications, and data modeling. It provides details on the objectives, length, audience, prerequisites, and agenda for each course. The document also includes a schedule of public course dates and locations for attendees to sign up for training.
Cassandra Community Webinar: MySQL to Cassandra - What I Wish I'd KnownDataStax
A brief intro to how Barracuda Networks uses Cassandra and the ways in which they are replacing their MySQL infrastructure, with Cassandra. This presentation will include the lessons they've learned along the way during this migration.
Speaker: Michael Kjellman, Software Engineer at Barracuda Networks
Michael Kjellman is a Software Engineer, from San Francisco, working at Barracuda Networks. Michael works across multiple products, technologies, and languages. He primarily works on Barracuda's spam infrastructure and web filter classification data.
Cassandra CLuster Management by Japan Cassandra CommunityHiromitsu Komatsu
This document discusses best practices for managing Cassandra clusters based on Instaclustr's experience managing over 500 nodes and 3 million node-hours. It covers choosing the right Cassandra version, hardware configuration, cost estimation, load testing, data modeling practices, common issues like modeling errors and overload, and important monitoring techniques like logs, metrics, cfstats and histograms. Maintaining a well-designed cluster and proactively monitoring performance are keys to avoiding issues with Cassandra.
This document discusses real time analytics using Spark and Spark Streaming. It provides an introduction to Spark and highlights limitations of Hadoop for real-time analytics. It then describes Spark's advantages like in-memory processing and rich APIs. The document discusses Spark Streaming and the Spark Cassandra Connector. It also introduces DataStax Enterprise which integrates Spark, Cassandra and Solr to allow real-time analytics without separate clusters. Examples of streaming use cases and demos are provided.
The document discusses Apache Cassandra, a distributed database management system designed to handle large amounts of data across many commodity servers. It was developed at Facebook and modeled after Google's Bigtable. The summary discusses key concepts like its use of consistent hashing to distribute data, support for tunable consistency levels, and focus on scalability and availability over traditional SQL features. It also provides an overview of how Cassandra differs from relational databases by not supporting joins, having an optional schema, and using a prematerialized and transaction-less model.
The document discusses best practices for moving Cassandra pilots and proofs of concept (PoCs) to production. It recommends starting with defining queries, building out 5-8 pilots, and designing REST APIs first. When moving to production, considerations include infrastructure, testing, coding practices like using asynchronous execution, data modeling for queries and analytics, and application optimization techniques.
Data Pipelines with Spark & DataStax EnterpriseDataStax
This document discusses building data pipelines for both static and streaming data using Apache Spark and DataStax Enterprise (DSE). For static data, it recommends using optimized data storage formats, distributed and scalable technologies like Spark, interactive analysis tools like notebooks, and DSE for persistent storage. For streaming data, it recommends using scalable distributed technologies, Kafka to decouple producers and consumers, and DSE for real-time analytics and persistent storage across datacenters.
C* Summit 2013: Cassandra at eBay Scale by Feng Qu and Anurag JambhekarDataStax Academy
We have seen rapid adoption of C* at eBay in past two years. We have made tremendous efforts to integrate C* into existing database platforms, including Oracle, MySQL, Postgres, MongoDB, XMP etc.. We also scale C* to meet business requirement and encountered technical challenges you only see at eBay scale, 100TB data on hundreds of nodes. We will share our experience of deployment automation, managing, monitoring, reporting for both Apache Cassandra and DataStax enterprise.
Apache Spark is a fast, general-purpose, and easy-to-use cluster computing system for large-scale data processing. It provides APIs in Scala, Java, Python, and R. Spark is versatile and can run on YARN/HDFS, standalone, or Mesos. It leverages in-memory computing to be faster than Hadoop MapReduce. Resilient Distributed Datasets (RDDs) are Spark's abstraction for distributed data. RDDs support transformations like map and filter, which are lazily evaluated, and actions like count and collect, which trigger computation. Caching RDDs in memory improves performance of subsequent jobs on the same data.
Running Analytics at the Speed of Your BusinessRedis Labs
The speed at which you can extract insights from your data is increasingly a competitive edge for your business. Data and analytics have to be at lightning fast speeds to seriously impact your user acquisition.
Join this webinar featuring Forrester analyst Noel Yuhanna and Leena Joshi, VP Product Marketing at Redis Labs to learn how you can glean insights faster with new open source data processing frameworks like Spark and Redis.
In this webinar you will learn:
* Why analytics has to run at the real time speed of business
* How this can be achieved with next generation Big Data tools
* How data structures can optimize your hybrid transaction-analytics processing scenarios
This document discusses processing 50,000 transactions per second using Apache Spark and Apache Cassandra. It describes monitoring over 600 servers running Cassandra by developing a metric history system using Spark, Cassandra, and other tools. Key aspects covered include data modeling, writing data efficiently in batches, joining Spark and Cassandra tables for faster data extraction during rollups, and using Cassandra aggregates to further improve performance.
How jKool Analyzes Streaming Data in Real Time with DataStaxDataStax
In this webinar, Charles Rich, VP of Product Management at jKool will share their journey with DataStax; how jKool knew from the start that traditional relational databases wouldn’t work for the scalability and availability demands of time-series data, and why they turned to DataStax Enterprise for blazing performance and powerful enterprise search and analytics capabilities.
Managing Cassandra Databases with OpenStack TroveTesora
This document summarizes OpenStack Trove, an OpenStack service for provisioning and managing databases in OpenStack clouds. It discusses what OpenStack and Trove are, how Trove integrates with other OpenStack services, and Trove's capabilities like provisioning, backup/restore, replication, clustering, and resizing for both SQL and NoSQL databases like Cassandra, MongoDB, and PostgreSQL. It also introduces Tesora as a major contributor to Trove that provides an enterprise-grade Trove platform with additional support and customization options.
Cassandra nyc 2011 ilya maykov - ooyala - scaling video analytics with apac...ivmaykov
This document discusses scaling video analytics using Apache Cassandra. It provides an overview of Ooyala's video analytics platform and the challenges of scaling to support billions of log pings and terabytes of data daily. Cassandra is used to store over 10 terabytes of historical analytics data covering 4 years of growth. The key challenges addressed are scaling to handle enormous data volumes, providing fast processing and query speeds, supporting deep queries over many dimensions of data, ensuring accuracy, and allowing for rapid developer iteration. The document explains how Cassandra's data model and capabilities help meet these challenges through features like linear scalability, tunable consistency, and a rich data model.
In this talk Josep draws on his experience of building a data platform based on Cassandra and Spark to service the UK's foremost player in the connected homes market. Bringing streams of data online; productionising data science algorithms on spark; and delivering outputs via API's or Kafka messages.
Josep will explore the ups and the downs of bringing all this together and share what he's learned from 12 months of Cassandra and Spark development and operations.
Cassandra Summit 2014: A Train of Thoughts About Growing and Scalability — Bu...DataStax Academy
Presenter: Eiti Kimura, Senior Software Engineer at Movile
Apache Cassandra was adopted by Movile in 2009, and became a fundamental piece within the robust and scalable architecture to support more than 50 products, impacted by over 200MM users in Latin America. In this case we present the architecture of our ring, configuration details, detailed tuning, hardware used to be able to achieve our performance requirements (order of a few milliseconds), information storage strategies for network and disk space optimization, and best practices, in addition to showing the evolution of the architecture of simple systems to become scalable and distributed platforms. We introduced our cluster with a relatively low number of nodes (6) using commodity hardware to support critical high-performance applications. After this talk, you'll understand how Apache Cassandra was essential to evolve our systems and leverage the growth of our business. Movile is the leading mobile content company in Latin America. Movile’s products include mobile content, mobile TV, mobile learning, mobile games, mobile payment, mobile marketing and mobile commerce. Every month, it publishes content and services to more than 20 million mobile costumers. It has grown substantially over the last few years (with a more than 25-fold increase in its revenue over the last five years) both organically and through an aggressive M&A strategy, including five acquisitions in the last five years. Movile is positioning itself as a kind of Silicon Valley company based in Brazil. For the last two years, Movile has been named in the “Great Place to Work” list for technology companies in Brazil. The company shareholders include the founders of the company plus Naspers, a South-African media conglomerate.
Cassandra Summit 2014: Social Media Security Company Nexgate Relies on Cassan...DataStax Academy
Presenter: Harold Nguyen, Senior Data Scientist at Nexgate
In this talk, we focus on a use case by showing how Cassandra can detect spam and spammers on social media. We also show how we use Cassandra to train our 100+ social-media-security classifiers. The accuracy of any security product is directly tied to the breadth of the corpus of data upon which it is built. For Nexgate, this means that the success of our products is inextricably tied to our ability to save everything we've ever scanned, but in a way that is still readily accessible. In the days before NoSQL, this was hard. This talk is about how Datastax and Cassandra make it easy.
Cassandra Summit 2014: META — An Efficient Distributed Data Hub with Batch an...DataStax Academy
Stratio Crossdata is a distributed data platform that allows for both batch and streaming queries across multiple data stores. It uses Spark to enable operations not natively supported and provides connectors to integrate different data sources. The platform aims to simplify deployment, administration and querying for clients through its metadata management and support for features like full text search, joins and streaming queries.
This document proposes an email app that models conversations and topics as the core data structures. It outlines two data models - one where conversations are grouped by a hash of recipients and topics represent conversation threads, and another where conversations are grouped by a hash of recipients and emails are attached directly to conversations. The data models are designed for an app built using Scala, C*, AWS, and Spark.
Cassandra Summit 2014: Cassandra in Large Scale Enterprise Grade xPatterns De...DataStax Academy
Presenter: Claudiu Barbura, Senior Director of Engineering at Atigeo
xPatterns is a big data analytics platform-as-a-service that enables rapid development of enterprise-grade analytical applications. It provides tools, API sets and a management console for building an ELT pipeline with data monitoring and quality gates, a data warehouse for ad-hoc and scheduled querying, analysis, model building and experimentation, tools for exporting data to Cassandra and solrCloud clusters for real-time access through low-latency/high-throughput (automatically generated) apis as well as dashboard and visualization api/tools leveraging the available data and models. In this talk I'll share some of the hard lessons we've learned in the past three years while leveraging Cassandra (and Hector) in large-scale enterprise-grade deployments. We will focus on three specific areas, in which we identified consistent best practices & design patterns: data model optimization as a result of exporting data from HDFS/Hive/Shark into Cassandra through Spark/Hadoop MR jobs under Mesos with throttling, instrumentation and resilience features, automatically publishing geo-replicated, instrumented and monitored REST API's on top of the exported Cassandra data, and lessons learned from running Cassandra at scale from 0.6 to 2.0.6, including performance tuning, and tips and tricks. You will see live demos of our Publish to NoSql tools (Spark/Shark, Mesos, Hive, Cassandra ), a dashboard application built on top of generated data apis (D3.js, Cassandra) and xPatterns' monitoring and instrumentation consoles (Graphite, Ganglia, Nagios).
A good data model is key to getting the best performance from Apache Cassandra. The Log Structured Storage Engine and it's distributed architecture mean we cannot rely on a paradigm such as Normal Form to evaluate a model. Instead we need to design data models that support the read path of the application. In this talk Aaron Morton will walk through the key principles and patterns of a good CQL3 data model using simple examples.
Cassandra Summit 2014: Apache Cassandra at Telefonica CBSDataStax Academy
Presenter: Antonio Alcocer, Big Data Architect at Stratio
Telefonica is the incumbent telecommunications network operator in Spain and the fourth one in capitalisation in the world. Cyber security is one of our most successful businesses worldwide. We provide monitoring and protecting clients from attacks. We analyze millions of data from multiple sources including social media, DNS records, and underground internet, to generate alerts and security reports for our clients. This use case required a Big Data component capable of processing the data and extract its information in real-time; warnings and alerts are time-sensitive in order to deal efficiently with security attacks. Our original architecture was the typical one used for data fusion systems. It included several collectors, a processing layer based on legacy systems, and a data store. The initial setup included a MongoDB database and an ad-hoc application. This solution however proved to be unfit for the specific purpose of dispatching alerts. We proposed to use Cassandra and Spark instead. This approach did manage to fulfill our original specifications as intended. Our talk will explain the reasons why we migrated the architecture and how the adopted solution based on Spark and Cassandra solved our problem.
Presenter: Chris Lohfink, Engineer at Pythian
This session will cover a walk-through to provide an understanding of key metrics critical to operating a Cassandra cluster effectively. Without context to the metrics, we just have pretty graphs. With context, we have a powerful tool to determine problems before they happen and to debug production issues more quickly.
Like many startups, Coursera began its data storage journey with MySQL, a familiar and industry-proven database. As Coursera's user base grew from several thousand to many millions, we found that MySQL provided limited availability and restricted our ability to scale easily. New product initiatives and requirements provided a perfect opportunity to revisit our choice of core workhorse database.
After evaluating several NoSQL databases, including MongoDB, DynamoDB and HBase, we elected to transition to Cassandra . Cassandra's relative maturity, masterless architecture (for availability), tunable consistency, and stable low-latency performance made it a clear winner for our needs.
Learn more about what it takes to transition from SQL to Cassandra in this talk.
You've researched. You've discussed. You've had (multiple) meetings. You've installed. You've tested (hopefully). You've have decided. Now what (besides having attended a Cassandra Day)? What else are you going to need to put that Cassandra cluster into beta? Our evangelist team will give you the Cliff Notes to make that next step go as smooth as.... well... as smooth as it can be!
DataStax: How to Roll Cassandra into Production Without Losing your Health, M...DataStax Academy
This document provides guidance on how to successfully implement Apache Cassandra in a production environment without issues. It recommends starting with a small, well-defined project like monitoring website events or users, rather than trying to build a large, multi-year platform. The document outlines choosing a specific pain point to address, implementing a simple proof of concept using Cassandra for tasks like event tracking, and iterating from there. It cautions against copying relational data models into Cassandra and emphasizes understanding how Cassandra works differently from SQL databases. The goal is to start small and grow capability over time rather than taking on too much at once.
Battery Ventures: Simulating and Visualizing Large Scale Cassandra DeploymentsDataStax Academy
The SimianViz microservices simulator contains a model of Cassandra that allows large scale global deployments to be created and exercised by simulating failure modes and connecting the simulation to real monitoring tools to visualize the effects. The simulator is open source Go code at github.com/adrianco/spigo and is developing rapidly.
DataStax: Old Dogs, New Tricks. Teaching your Relational DBA to fetchDataStax Academy
Do you love some Cassandra, but that relational brain is still on? You aren't alone. Let's take that OLAP data model and get it OLTP. This will be an updated talk with some of the new features brought to you by Cassandra 3.0. Real techniques to translate application patterns into effective models. Common pitfalls that can slow you down and send you running back to RDBMS land. Don't do it! Finally, if you didn't get it right the first time, I'll show you how to fix that data model without any downtime. Turn a hot cup of fail into a tall glass of awesome!
The document summarizes new features in Cassandra versions 3.0 and 2.2, including improvements to user defined functions, aggregates, JSON support, hints, storage engine, and the introduction of materialized views. Version 2.2 allowed user defined functions and aggregates, as well as JSON support in CQL queries. Hints were stored in a system table requiring compaction for space reclamation. Version 3.0 moved hints to a file dropped on delivery, improved the storage engine design, and introduced materialized views for denormalizing data and improving query performance.
Cassandra Summit 2014: The Cassandra Experience at Orange — Season 2DataStax Academy
Presenter: Jean Armel Luce, Cassandra Administrator at Orange
At the Cassandra Summit Europe 2013, Jean Armel presented "The Cassandra Experience at Orange - Season 1", explaining the 1st steps of Cassandra at Orange (choice of Cassandra, migration without any interruption of service, improvements of the QoS after the migration). For the "Cassandra Experience at Orange - Season 2", Jean Armel is going to focus on 2 new features added to the PnS application during the last months: Graphs and analytics. A Cassandra table must have 1 and only 1 primary key, while some data have many logical identifiers. Designing data as a graph may help! As for analyitcs, Hadoop + Hive allow to do analytics on data stored in Cassandra. This presentation is going to highlight a few tips about the installation of Hadoop/Hive over C*, and about the isolation between mapreduce tasks and on line queries.
The Last Pickle: Distributed Tracing from Application to DatabaseDataStax Academy
Monitoring provides information on system performance, however tracing is necessary to understand individual request performance. Detailed query tracing has been provided by Cassandra since version 1.2 and is invaluable when diagnosing problems. Although knowing what queries to trace and why the application makes them still requires deep technical knowledge. By merging Application tracing via Zipkin and Cassandra query tracing we automate the process and make it easier to identify and resolve problems. In this talk Mick Semb Wever, Team Member at The Last Pickle, will introduce Cassandra query tracing and Zipkin. He will then propose an extension that allows clients to pass a trace identifier through to Cassandra, and a way to integrate Zipkin tracing into Cassandra. Driving all this is the desire to create one tracing view across the entire system.
This document summarizes new features in Cassandra 3.0, including user defined functions, improved garbage collection, hints management, materialized views, and a new storage engine. User defined functions allow running custom Java or JavaScript functions on Cassandra data. The G1 garbage collector replaces older collectors for better performance and predictability. Hints are now written to files instead of using Cassandra as a queue. Materialized views automatically create and maintain secondary indexes. The new storage engine reduces data duplication and wasted space.
Cassandra is a distributed database that provides high availability and scalability. It uses a ring topology to replicate and distribute data across multiple nodes. Cassandra sacrifices consistency in favor of availability and partition tolerance. Data is modeled using tables containing partitions and clustered rows accessed by partition and clustering keys. Writes are replicated across the ring and stored in memory and on disk for fault tolerance.
An introduction to core concepts in Apache Cassandra. We cover the evolution of database architecture as you try to scale a relational database to solve big data problems, and explain how Cassandra handles these problems efficiently.
Azure + DataStax Enterprise Powers Office 365 Per User StoreDataStax Academy
We will present our O365 use case scenarios, why we chose Cassandra + Spark, and walk through the architecture we chose for running DataStax Enterprise on azure.
3 Things to Learn About:
-How Kudu is able to fill the analytic gap between HDFS and Apache HBase
-The trade-offs between real-time transactional access and fast analytic performance
-How Kudu provides an option to achieve fast scans and random access from a single API
We will examine most of the features that this “Swiss knife” software provides. It is an in-memory fabric that fits between the database and the application layer. Apache Ignite is powered by the H2 engine. They have used it to create an in-memory distributed ACID, fully ANSI-99 complaint, Highly Available (HA) and scalable database. They have used a non-consensus (https://en.wikipedia.org/wiki/Rendezvous_hashing) clustering algorithm to be even more scalable compared to other NoSql solutions. This tool respects the relational data model that we have used for so many years and eliminates traditional problems like the “expensive joins” since it uses the RAM as the primary storage medium. We will see what this tool can do in action through hands-on examples.
Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...Pierre GRANDIN
What do you do when your usual setup or turnkey solution isn’t suited for your workload?
Most of the documentation and user feedback that you can find about OpenStack is written for the use-case of running a public facing cloud serving several external customers. When you want to host a single tenant with a single application the problem is completely different, you don't want publicly exposed APIs. You want to ensure optimal resource allocation to maximize your application performance. You want to leverage the fact that you own the infrastructure layer to optimize your instance placement strategy, and to get the best latency and to avoid creating SPOFs using affinity (or anti affinity rules).
This talk will focus on what we learned during a two years journey; from getting OpenStack up and running reliably, to investigating performance bottlenecks, to maximizing the performance of our private cloud.
The document provides information about upcoming presentations at the Brisbane Azure User Group from January 2019 to November 2019. It also includes announcements about new Azure services and features such as Azure Functions Premium Plan, Azure Search storage optimized tiers, data discovery and classification for Azure SQL Data Warehouse, Azure Front Door service reaching general availability, and Azure Backup for SQL Server in Azure VMs also reaching general availability. Additionally, it advertises events such as the Global Azure Bootcamp and Integration Down Under conference.
By David Smith. Presented at Microsoft Build (Seattle), May 7 2018.
Your data scientists have created predictive models using open-source tools, proprietary software, or some combination of both, and now you are interested in lifting and shifting those models to the cloud. In this talk, I'll describe how data scientists can transition their existing workflows — while using mostly the same tools and processes — to train and deploy machine learning models based on open source frameworks to Azure. I'll provide guidance on keeping connections to data sources up-to-date, evaluating and monitoring models, and deploying applications that make use of those models.
Customer Sharing: Trend Micro - Analytic Engine - A common Big Data computati...Amazon Web Services
In recent years, more and more enterprises notice about what values of Big Data can bring, and willing to devote more resources to Big Data field. Doing Hadoop for PoC and further for running in PROD. In common cases, enterprises need to get their servers first for running their Hadoop. By now, thanks for the popularity of Hadoop and its ecosystem. Enterprises have another choice for doing Hadoop, which is, doing it on Public Cloud platforms, such as Amazon, etc. Trend Micro also noticed this trends for Big Data on the cloud, and would like to leverage its elasticity to enable more chances to find more values from our Big Data with less of constraints. In this sharing, we would like to introduce our common Big Data computation platform - Analytic Engine (AE), which is a simple RESTful API service running on AWS for Trenders, with features, such as createCluster, deleteCluster and submitJob, etc. By now, Trenders can run their research jobs, and furthermore, build their own PoC/Staging/PROD levels of services based on AE, to get any computation resources they want, anytime and anyplace in Trend Micro, just by few RESTful API calls.
analytic engine - a common big data computation service on the awsScott Miao
This document summarizes Scott Miao's presentation on Analytic Engine (AE), a common big data computation service on AWS. AE provides a RESTful API for users to create AWS EMR clusters, submit jobs to clusters, and delete clusters. It handles job scheduling and delivery to clusters to optimize usage of AWS resources. Using AE and AWS services like EMR and S3 allows Trend Micro to scale their data and computation needs elastically with reduced operational overhead compared to managing infrastructure on their own.
The document provides tips for building a scalable and high-performance website, including using caching, load balancing, and monitoring. It discusses horizontal and vertical scalability, and recommends planning, testing, and version control. Specific techniques mentioned include static content caching, Memcached, and the YSlow performance tool.
Patterns and Pains of Migrating Legacy Applications to KubernetesJosef Adersberger
Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs, and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud native apps. But what to do if you’ve no shiny new cloud native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can!
We’re facing the challenge of migrating hundreds of JEE legacy applications of a German blue chip company onto a Kubernetes cluster within one year.
The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way.
Patterns and Pains of Migrating Legacy Applications to KubernetesQAware GmbH
Open Source Summit 2018, Vancouver (Canada): Talk by Josef Adersberger (@adersberger, CTO at QAware), Michael Frank (Software Architect at QAware) and Robert Bichler (IT Project Manager at Allianz Germany)
Abstract:
Running applications on Kubernetes can provide a lot of benefits: more dev speed, lower ops costs and a higher elasticity & resiliency in production. Kubernetes is the place to be for cloud-native apps. But what to do if you’ve no shiny new cloud-native apps but a whole bunch of JEE legacy systems? No chance to leverage the advantages of Kubernetes? Yes you can!
We’re facing the challenge of migrating hundreds of JEE legacy applications of a German blue chip company onto a Kubernetes cluster within one year.
The talk will be about the lessons we've learned - the best practices and pitfalls we've discovered along our way.
20150704 benchmark and user experience in sahara weitingWei Ting Chen
Sahara provides a way to deploy and manage Hadoop clusters within an OpenStack cloud. It addresses common customer needs like providing an elastic environment for data processing jobs, integrating Hadoop with the existing private cloud infrastructure, and reducing costs. Key challenges include speeding up cluster provisioning times, supporting complex data workflows, optimizing storage architectures, and improving performance when using remote object storage.
The document summarizes an upcoming talk on Azure Site Recovery and business continuity by Janaka Rangama of Empired Ltd. on August 9th at Index Consultants in Melbourne. It provides details on the speaker, topic, date, location and includes links to recent Microsoft Azure announcements on new government datacenter regions, Azure Stack ordering, and Azure Batch Rendering in public preview.
This document discusses SPN's journey to implement CI/CD on AWS. It begins with describing SPN's original process for delivering services which involved many manual steps. It then discusses DevOps goals of faster delivery, lower failure rates, and faster recovery compared to the original process. The document outlines using AWS services like CloudFormation, OpsWorks, and Auto Scaling to implement CI/CD and automate deploying a sample analytic engine service. Lessons learned include automating as much as possible, splitting CloudFormation templates, focusing on updates without impacting SLAs, and emphasizing monitoring and testing.
Docker is an open platform for developers and system administrators to build, ship and run distributed applications. Using Docker, companies in Jordan have been able to build powerful system architectures that allow speeding up delivery, easing deployment processes and at the same time cutting major hosting costs.
Osama Jaber shares his experience at ArabiaWeather in how they moved away from AWS to a highly-redundant, high-performance and low-cost solution using docker and other open-source technologies.
This document provides an overview and introduction to Windows Azure SQL Database. It discusses the security requirements and compliance certifications for the Azure platform. It also covers key features of SQL Database including service tiers, sizes and performance levels measured in Database Transaction Units (DTUs). The document reviews compatibility and limitations compared to on-premises SQL Server versions.
Similar to Azure + DataStax Enterprise (DSE) Powers Office365 Per User Store (20)
Forrester CXNYC 2017 - Delivering great real-time cx is a true craftDataStax Academy
Companies today are innovating with real-time data to deliver truly amazing customer experiences in the moment. Real-time data management for real-time customer experience is core to staying ahead of competition and driving revenue growth. Join Trays to learn how Comcast is differentiating itself from it's own historical reputation with Customer Experience strategies.
Introduction to DataStax Enterprise Graph DatabaseDataStax Academy
DataStax Enterprise (DSE) Graph is a built to manage, analyze, and search highly connected data. DSE Graph, built on NoSQL Apache Cassandra delivers continuous uptime along with predictable performance and scales for modern systems dealing with complex and constantly changing data.
Download DataStax Enterprise: Academy.DataStax.com/Download
Start free training for DataStax Enterprise Graph: Academy.DataStax.com/courses/ds332-datastax-enterprise-graph
Introduction to DataStax Enterprise Advanced Replication with Apache CassandraDataStax Academy
DataStax Enterprise Advanced Replication supports one-way distributed data replication from remote database clusters that might experience periods of network or internet downtime. Benefiting use cases that require a 'hub and spoke' architecture.
Learn more at http://www.datastax.com/2016/07/stay-100-connected-with-dse-advanced-replication
Advanced Replication docs – https://docs.datastax.com/en/latest-dse/datastax_enterprise/advRep/advRepTOC.html
This document discusses using Docker containers to run Cassandra clusters at Walmart. It proposes transforming existing Cassandra hardware into containers to better utilize unused compute. It also suggests building new Cassandra clusters in containers and migrating old clusters to double capacity on existing hardware and save costs. Benchmark results show Docker containers outperforming virtual machines on OpenStack and Azure in terms of reads, writes, throughput and latency for an in-house application.
The document discusses the evolution of Cassandra's data modeling capabilities over different versions of CQL. It covers features introduced in each version such as user defined types, functions, aggregates, materialized views, and storage attached secondary indexes (SASI). It provides examples of how to create user defined types, functions, materialized views, and SASI indexes in CQL. It also discusses when each feature should and should not be used.
Cisco has a large global IT infrastructure supporting many applications, databases, and employees. The document discusses Cisco's existing customer service and commerce systems (CSCC/SMS3) and some of the performance, scalability, and user experience issues. It then presents a proposed new architecture using modern technologies like Elasticsearch, Cassandra, and microservices to address these issues and improve agility, performance, scalability, uptime, and the user interface.
Data Modeling is the one of the first things to sink your teeth into when trying out a new database. That's why we are going to cover this foundational topic in enough detail for you to get dangerous. Data Modeling for relational databases is more than a touch different than the way it's approached with Cassandra. We will address the quintessential query-driven methodology through a couple of different use cases, including working with time series data for IoT. We will also demo a new tool to get you bootstrapped quickly with MovieLens sample data. This talk should give you the basics you need to get serious with Apache Cassandra.
Hear about how Coursera uses Cassandra as the core of its scalable online education platform. I'll discuss the strengths of Cassandra that we leverage, as well as some limitations that you might run into as well in practice.
In the second part of this talk, we'll dive into how best to effectively use the Datastax Java drivers. We'll dig into how the driver is architected, and use this understanding to develop best practices to follow. I'll also share a couple of interesting bug we've run into at Coursera.
This document promotes Datastax Academy and Certification resources for learning Cassandra including a three step process of learning Cassandra, getting certified, and profiting. It lists community evangelists like Luke Tillman, Patrick McFadin, Jon Haddad, and Duy Hai Doan who can provide help and resources.
Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & PythonDataStax Academy
This document summarizes three presentations from a Cassandra Meetup:
1. Jason Cacciatore discussed monitoring Cassandra health at scale across hundreds of clusters and thousands of nodes using the reactive stream processing system Mantis.
2. Minh Do explained how Cassandra uses the gossip protocol for tasks like discovering cluster topology and sharing load information. Gossip also has limitations and race conditions that can cause problems.
3. Chris Kalantzis presented Cassandra Tickler, an open source tool he created to help repair operations that get stuck by running lightweight consistency checks on an old Cassandra version or a node with space issues.
Cassandra @ Sony: The good, the bad, and the ugly part 1DataStax Academy
This talk covers scaling Cassandra to a fast growing user base. Alex and Isaias will cover new best practices and how to work with the strengths and weaknesses of Cassandra at large scale. They will discuss how to adapt to bottlenecks while providing a rich feature set to the playstation community.
Cassandra @ Sony: The good, the bad, and the ugly part 2DataStax Academy
The document discusses Cassandra's use by Sony Network Entertainment to handle the large amount of user and transaction data from the growing PlayStation Network. It describes how the relational database they previously used did not scale sufficiently, so they transitioned to using Cassandra in a denormalized and customized way. Some of the techniques discussed include caching user data locally on application servers, secondary indexing, and using a real-time indexer to enable personalized search by friends.
This document provides guidance on setting up server monitoring, application metrics, log aggregation, time synchronization, replication strategies, and garbage collection for a Cassandra cluster. Key recommendations include:
1. Use monitoring tools like Monit, Munin, Nagios, or OpsCenter to monitor processes, disk usage, and system performance. Aggregate all logs centrally with tools like Splunk, Logstash, or Greylog.
2. Install NTP to synchronize server times which are critical for consistency.
3. Use the NetworkTopologyStrategy replication strategy and avoid SimpleStrategy for production.
4. Avoid shared storage and focus on low latency and high throughput using multiple local disks.
5. Understand
Introduction to Data Modeling with Apache CassandraDataStax Academy
This document provides an introduction to data modeling with Apache Cassandra. It discusses how Cassandra data models are designed based on the queries an application will perform, unlike relational databases which are designed based on normalization rules. Key aspects covered include avoiding joins by denormalizing data, using a partition key to group related data on nodes, and controlling the clustering order of columns. The document provides examples of modeling time series and tag data in Cassandra.
The document discusses different data storage options for small, medium, and large datasets. It argues that relational databases do not scale well for large datasets due to limitations with replication, normalization, sharding, and high availability. The document then introduces Apache Cassandra as a fast, distributed, highly available, and linearly scalable database that addresses these limitations through its use of a hash ring architecture and tunable consistency levels. It describes Cassandra's key features including replication, compaction, and multi-datacenter support.
Enabling Search in your Cassandra Application with DataStax EnterpriseDataStax Academy
This document provides an overview of using Datastax Enterprise (DSE) Search to enable full-text search capabilities in Cassandra applications. It discusses how DSE Search integrates Solr/Lucene indexing with the Cassandra database to allow searching of application data without requiring a separate search cluster, external ETL processes, or custom application code for data management. The document also includes examples of different types of searches that can be performed, such as filtering, faceting, geospatial searches, and joins. It concludes with basic steps for getting started with DSE Search such as creating a Solr core and executing search queries using CQL.
The document discusses common bad habits that can occur when working with Apache Cassandra and provides recommendations to avoid them. Specifically, it addresses issues like sliding back into a relational mindset when the data model is different, improperly benchmarking Cassandra systems, having slow client performance, and neglecting important operations tasks. The presentation provides guidance on how to approach data modeling, querying, benchmarking, driver usage, and operations management in a Cassandra-oriented way.
This document provides an overview and examples of modeling data in Apache Cassandra. It begins with an introduction to thinking about data models and queries before modeling, and emphasizes that Cassandra requires modeling around queries due to its limitations on joins and indexes. The document then provides examples of modeling user, video, and other entity data for a video sharing application to support common queries. It also discusses techniques for handling queries that could become hotspots, such as bucketing or adding random values. The examples illustrate best practices for data duplication, materialized views, and time series data storage in Cassandra.
The document discusses best practices for using Apache Cassandra, including:
- Topology considerations like replication strategies and snitches
- Booting new datacenters and replacing nodes
- Security techniques like authentication, authorization, and SSL encryption
- Using prepared statements for efficiency
- Asynchronous execution for request pipelining
- Batch statements and their appropriate uses
- Improving performance through techniques like the new row cache
This is a two part talk in which we'll go over the architecture that enables Apache Cassandra’s linear scalability as well as how DataStax Drivers are able to take full advantage of it to provide developers with nicely designed and speedy clients extendable to the core.
Cracking AI Black Box - Strategies for Customer-centric Enterprise ExcellenceQuentin Reul
The democratization of Generative AI is ushering in a new era of innovation for enterprises. Discover how you can harness this powerful technology to deliver unparalleled customer value and securing a formidable competitive advantage in today's competitive market. In this session, you will learn how to:
- Identify high-impact customer needs with precision
- Harness the power of large language models to address specific customer needs effectively
- Implement AI responsibly to build trust and foster strong customer relationships
Whether you're at the early stages of your AI journey or looking to optimize existing initiatives, this session will provide you with actionable insights and strategies needed to leverage AI as a powerful catalyst for customer-driven enterprise success.
Finetuning GenAI For Hacking and DefendingPriyanka Aash
Generative AI, particularly through the lens of large language models (LLMs), represents a transformative leap in artificial intelligence. With advancements that have fundamentally altered our approach to AI, understanding and leveraging these technologies is crucial for innovators and practitioners alike. This comprehensive exploration delves into the intricacies of GenAI, from its foundational principles and historical evolution to its practical applications in security and beyond.
The History of Embeddings & Multimodal EmbeddingsZilliz
Frank Liu will walk through the history of embeddings and how we got to the cool embedding models used today. He'll end with a demo on how multimodal RAG is used.
Retrieval Augmented Generation Evaluation with RagasZilliz
Retrieval Augmented Generation (RAG) enhances chatbots by incorporating custom data in the prompt. Using large language models (LLMs) as judge has gained prominence in modern RAG systems. This talk will demo Ragas, an open-source automation tool for RAG evaluations. Christy will talk about and demo evaluating a RAG pipeline using Milvus and RAG metrics like context F1-score and answer correctness.
Choosing the Best Outlook OST to PST Converter: Key Features and Considerationswebbyacad software
When looking for a good software utility to convert Outlook OST files to PST format, it is important to find one that is easy to use and has useful features. WebbyAcad OST to PST Converter Tool is a great choice because it is simple to use for anyone, whether you are tech-savvy or not. It can smoothly change your files to PST while keeping all your data safe and secure. Plus, it can handle large amounts of data and convert multiple files at once, which can save you a lot of time. It even comes with 24*7 technical support assistance and a free trial, so you can try it out before making a decision. Whether you need to recover, move, or back up your data, Webbyacad OST to PST Converter is a reliable option that gives you all the support you need to manage your Outlook data effectively.
Keynote : AI & Future Of Offensive SecurityPriyanka Aash
In the presentation, the focus is on the transformative impact of artificial intelligence (AI) in cybersecurity, particularly in the context of malware generation and adversarial attacks. AI promises to revolutionize the field by enabling scalable solutions to historically challenging problems such as continuous threat simulation, autonomous attack path generation, and the creation of sophisticated attack payloads. The discussions underscore how AI-powered tools like AI-based penetration testing can outpace traditional methods, enhancing security posture by efficiently identifying and mitigating vulnerabilities across complex attack surfaces. The use of AI in red teaming further amplifies these capabilities, allowing organizations to validate security controls effectively against diverse adversarial scenarios. These advancements not only streamline testing processes but also bolster defense strategies, ensuring readiness against evolving cyber threats.
DefCamp_2016_Chemerkin_Yury-publish.pdf - Presentation by Yury Chemerkin at DefCamp 2016 discussing mobile app vulnerabilities, data protection issues, and analysis of security levels across different types of mobile applications.
Discovery Series - Zero to Hero - Task Mining Session 1DianaGray10
This session is focused on providing you with an introduction to task mining. We will go over different types of task mining and provide you with a real-world demo on each type of task mining in detail.
Demystifying Neural Networks And Building Cybersecurity ApplicationsPriyanka Aash
In today's rapidly evolving technological landscape, Artificial Neural Networks (ANNs) have emerged as a cornerstone of artificial intelligence, revolutionizing various fields including cybersecurity. Inspired by the intricacies of the human brain, ANNs have a rich history and a complex structure that enables them to learn and make decisions. This blog aims to unravel the mysteries of neural networks, explore their mathematical foundations, and demonstrate their practical applications, particularly in building robust malware detection systems using Convolutional Neural Networks (CNNs).
How UiPath Discovery Suite supports identification of Agentic Process Automat...DianaGray10
📚 Understand the basics of the newly persona-based LLM-powered Agentic Process Automation and discover how existing UiPath Discovery Suite products like Communication Mining, Process Mining, and Task Mining can be leveraged to identify APA candidates.
Topics Covered:
💡 Idea Behind APA: Explore the innovative concept of Agentic Process Automation and its significance in modern workflows.
🔄 How APA is Different from RPA: Learn the key differences between Agentic Process Automation and Robotic Process Automation.
🚀 Discover the Advantages of APA: Uncover the unique benefits of implementing APA in your organization.
🔍 Identifying APA Candidates with UiPath Discovery Products: See how UiPath's Communication Mining, Process Mining, and Task Mining tools can help pinpoint potential APA candidates.
🔮 Discussion on Expected Future Impacts: Engage in a discussion on the potential future impacts of APA on various industries and business processes.
Enhance your knowledge on the forefront of automation technology and stay ahead with Agentic Process Automation. 🧠💼✨
Speakers:
Arun Kumar Asokan, Delivery Director (US) @ qBotica and UiPath MVP
Naveen Chatlapalli, Solution Architect @ Ashling Partners and UiPath MVP
Generative AI technology is a fascinating field that focuses on creating comp...Nohoax Kanont
Generative AI technology is a fascinating field that focuses on creating computer models capable of generating new, original content. It leverages the power of large language models, neural networks, and machine learning to produce content that can mimic human creativity. This technology has seen a surge in innovation and adoption since the introduction of ChatGPT in 2022, leading to significant productivity benefits across various industries. With its ability to generate text, images, video, and audio, generative AI is transforming how we interact with technology and the types of tasks that can be automated.
Keynote : Presentation on SASE TechnologyPriyanka Aash
Secure Access Service Edge (SASE) solutions are revolutionizing enterprise networks by integrating SD-WAN with comprehensive security services. Traditionally, enterprises managed multiple point solutions for network and security needs, leading to complexity and resource-intensive operations. SASE, as defined by Gartner, consolidates these functions into a unified cloud-based service, offering SD-WAN capabilities alongside advanced security features like secure web gateways, CASB, and remote browser isolation. This convergence not only simplifies management but also enhances security posture and application performance across global networks and cloud environments. Discover how adopting SASE can streamline operations and fortify your enterprise's digital transformation strategy.
20. Azure Templates can:
• Ensure Idempotency
• Simplify Orchestration
• Simplify Roll-back
• Provide Cross-Resource Configuration
and Update Support
Azure Templates are:
• Source file, checked-in
• Specifies resources and dependencies
(VMs, WebSites, DBs) and connections
(config, LB sets)
• Parametized input/output
Instantiation of repeatable config.
Configuration Resource Group
Power of Repeatability
SQL - A Website
Virtual
Machines
SQL-A
Website
[SQL CONFIG] VM (2x)
DEPENDS ON SQLDEPENDS ON SQL
SQL CONFIG
21. Extending the power of your VM
Enable easier management
Support partner ecosystem
Full control still with you!
Azure VM Extensions
Curated
ExtensionsAgent
22. Thank you
Sean Usher
Office 365
Email: seusher@microsoft.com
Twitter: @seanushermsft
Mahesh Thiagarajan
Microsoft Azure
Email: mahthi@microsoft.com
Twitter: @_cloudguy
Ben Lackey
DataStax
Email: ben.lackey@datastax.com