This presentation gives an brief overview of the history of relational databases, ACID and SQL and presents some of the key strentgths and potential weaknesses. It introduces the rise of NoSQL - why it arose, what is entails, when to use it. The presentation focuses on MongoDB as prime example of NoSQL document store and it shows how to interact with MongoDB from JavaScript (NodeJS) and Java.
Anuj Singh Kanyal from BlazeClan technologies presents an overview of HTML5 and PhoneGap & how the next generation of web/mobile computing is going to change.
In the age of digital transformation and disruption, your ability to thrive depends on how you adapt to the constantly changing environment. MongoDB 3.4 is the latest release of the leading database for modern applications, a culmination of native database features and enhancements that will allow you to easily evolve your solutions to address emerging challenges and use cases.
In this webinar, we introduce you to what’s new, including:
- Multimodel Done Right. Native graph computation, faceted navigation, rich real-time analytics, and powerful connectors for BI and Apache Spark bring additional multimodel database support right into MongoDB.
- Mission-Critical Applications. Geo-distributed MongoDB zones, elastic clustering, tunable consistency, and enhanced security controls bring state-of-the-art database technology to your most mission-critical applications.
- Modernized Tooling. Enhanced DBA and DevOps tooling for schema management, fine-grained monitoring, and cloud-native integration allow engineering teams to ship applications faster, with less overhead and higher quality.
Responding to Digital Transformation With RDS Database TechnologyAlibaba Cloud
See Webinar Recording at https://resource.alibabacloud.com/webinar/detail.htm?webinarId=30
Learn how your business can gain an advantage and utilize ApsaraDB for RDS to enhance data flexibility, security, performance, and stability amid the backdrop of digital transformation.
Chen Zhaoshang of the Alibaba Cloud Database Product team will share how traditional industries can meet database challenges and leverage distributed architecture transformation using MySQL and other open technologies to reduce the overall cost of ownership and realize cross-data-center disaster tolerance deployment while ensuring data consistency. He will also share how to ensure database security databases and alleviate concerns about data migration to the cloud.
ApsaraDB for RDS: www.alibabacloud.com/product/apsaradb-for-rds
More Webinars: https://resource.alibabacloud.com/webinar/index.htm
This presentation talks about the Following -
-Working of AWS S3 & CloudFront Logs with respect to
Content Storing and Distribution.
-The hidden potential of your Stored S3 & CloudFront Logs
& Unlocking them with Cloudlytics
-Some of our Reports using Cloudlytics
Check the video embedded after the slideshare for a Live recording of our webinar conducted around this topic.
Building Complete Private Clouds with Apache CloudStack and Riak CSJohn Burwell
IT infrastructure rigidity has emerged as one of the leading barriers to achieving the cost efficiency and operational agility required to drive growth. While public cloud such as Amazon Web Services and Rackspace provide this agility, many organizations are precluded from utilizing them due to regulatory, security, performance, and/or existing investments. For these organizations to realize these agility benefits, they must transform their private infrastructures to embrace public cloud principles.
During this session, we will explore cloud system architecture principles and best practices. Using the Apache CloudStack cloud orchestration platform and Basho’s Riak CS object store, a complete, open source private cloud will be realized that creates the operational agility and cost-reduction benefits of public clouds.
Logging infrastructure for Microservices using StreamSets Data CollectorCask Data
This document discusses using StreamSets Data Collector (SDC) to build a logging infrastructure for microservices. SDC can ingest logs from microservices running in containers and handle issues like schema changes and new log formats. It processes and transforms the logs, sending them to destinations like Kafka. SDC pipelines can run on Spark clusters on Yarn and Mesos to handle large volumes of log data and load it into systems like HDFS, HBase and Elasticsearch for analysis.
Understanding Codenvy - for Containerized Developer WorkspacesLynn Langit
Codenvy provides containerized developer workspaces that allow for faster development in the cloud or on-premises. It aims to improve on current approaches like manual setup, desktop environments, and virtual machines that are administrative, error-prone, expensive, and not portable. Codenvy allows developers to modify, run, and debug code from anywhere through automated one-click provisioning of shareable and securely managed workspaces. It supports various programming languages and libraries through plug-ins and uses a factory/project model to create reusable templates and instances. Codenvy also enables contributions to open source through embedded buttons that allow direct code changes and pull requests on GitHub from within the Codenvy IDE.
When the Cloud is a Rockin: High Availability in Apache CloudStackJohn Burwell
CloudStack currently provides a variety bespoke high availability mechanisms for resources such as virtual machines, hosts, and virtual routers. Each of these implementations duplicates the HA check/recovery cycle, as well as, concurrency, persistence, and clustering required manage high available for any CloudStack resource. The High Availability Resource Management Service has been developed to consolidate these concerns -- providing a robust, extensible HA mechanism. Using this service, plugins only need to define health check, activity check, and fence operations.
Designing For Multicloud, CF Summit Frankfurt 2016Mark D'Cunha
Your carefully planned cloud strategy and technology architecture is useless, because multicloud changes everything. In this session, we will explore what multicloud means and why your business will force it upon you.
We provide examples of customers successfully using multicloud models, identify early patterns of usage and how to leverage them. You’ll learn about how Cloud Foundry provides unique capabilities to simplify and implement multicloud deployments. We’ll cover how you can use features like service brokers, service plans, asynchronous provisioning and arbitrary parameters to deploy muilticloud, while still maintaining a consistent experience for your application developers and IT operations staff.
Presentation on how developer roles change when meeting cloud infrastructure, and how a a "role driven"/template based VM deployment model helps this separation
RedisConf18 - Transforming Vulnerability Telemetry with Redis EnterpriseRedis Labs
This document summarizes Malwarebytes' transformation to leverage big data and AI by building a data and AI team over 18 months. It discusses challenges around scaling to big data volumes and varieties, handling stateful and real-time data, and massive data caching needs. It then outlines architectural goals and solutions developed using Redis Enterprise to build advanced data visualizations and analytics around malware detections, including infection maps, AV failure rates over time, and malware velocity trends. Redis Enterprise helped scale to large data volumes and processing needs cost effectively with high performance and availability.
An Introduction to the Speakers & What BlazeClan as an AWS Advanced Consulting Partners does and how it has Evolved. Varoon, Our Solution Architect, Specializing on Amazon Redshift, Talks about the Key differentiators of Amazon Redshift. Learn why & how Exactly Redshift can optimize your Time and Efforts & reduce costs by 1/10th the cost of a traditional warehouse solution. A Demo of Amazon Redshift in action, processing 2billion records in a matter of seconds! A casestudy of one of our products, Cloudlytics, and how it extensively user Amazon Redshift.
We had conducted a webinar on Amazon Redshift, you can also view the Video of the Webinar along with the Q & A at the end of the Slideshare.
An Introduction to Apache Ignite - Mandhir Gidda - Codemotion Rome 2017Codemotion
Apache Ignite is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies.
Serverless Architectures with AWS Lambda and MongoDB Atlas by Sig NarvaezData Con LA
Abstract:- It's easier than ever to power serverless architectures with managed database services like MongoDB Atlas. In this session, we will explore the rise of serverless architectures and how they've rapidly integrated into public and private cloud offerings. We will demonstrate how to build a simple REST API using AWS Lambda functions, create a highly available cluster in MongoDB Atlas, and connect both via VCP Peering. We will then simulate load and use the monitoring and scale features of MongoDB Atlas and use MongoDB Compass to browse our database.
Bridging Your Business Across the Enterprise and Cloud with MongoDB and NetAppMongoDB
This document discusses how NetApp solutions can help businesses bridge their MongoDB databases across on-premises and cloud environments. It provides an introduction to NetApp and describes how their storage solutions and data fabric can enable hybrid cloud for MongoDB. Specific solutions and technologies discussed include NetApp ONTAP for storage management and provisioning, FlexClone for development/testing, and SolidFire for high performance MongoDB deployments. Customer examples and performance benefits are also summarized.
Discovery Day 2019 Sofia - Big data clustersIvan Donev
This document provides an overview of the architecture and components of SQL Server 2019 Big Data Clusters. It describes the key Kubernetes concepts used in Big Data Clusters like pods, services, and nodes. It then explains the different planes (control, compute, data) and nodes that make up a Big Data Cluster and their roles. Components in each plane like the SQL master instance, compute pools, storage pools, and data pools are also outlined.
Discovery Day 2019 Sofia - What is new in SQL Server 2019Ivan Donev
This document summarizes new features in SQL Server 2019 including:
- Support for big data analytics using SQL Server, Spark, and data lakes for high volume data storage and access.
- Enhanced data virtualization capabilities to combine data from multiple sources.
- An integrated AI platform to ingest, prepare, train, store, and operationalize machine learning models.
- Improved security, performance, and high availability features like Always Encrypted with secure enclaves and accelerated database recovery.
- Continued improvements based on community feedback.
Bi and AI updates in the Microsoft Data Platform stackIvan Donev
This document provides an overview and agenda for a presentation on new features in the Microsoft data platform universe including SQL Server, Azure, and Power BI, with a focus on business intelligence (BI) and artificial intelligence (AI).
The presentation covers new features in SQL Server 2019 for BI like calculation groups and dynamic formatting in Analysis Services tabular mode. It also highlights important updates in Azure including new serverless and hyperscale options for Azure SQL Database, updates to data ingestion and storage with Azure Data Factory v2 and Azure Data Lake Gen 2, and changes to data modeling, preparation, and serving capabilities in Azure SQL Data Warehouse.
The document concludes by discussing options for incorporating AI and machine learning into BI solutions using either
Azure Cosmos DB: Features, Practical Use and Optimization "GlobalLogic Ukraine
This presentation is dedicated to Azure Cosmos DB, it's history, characteristics, tasks and solutions. The presentation deals with performance optimization, practical experience of usage and an overview of the news about Cosmos DB from Microsoft Build 2017 conference (https://build.microsoft.com).
This presentation by Andriy Gorda (Engineering Manager & Lead Software Engineer, Consultant, GlobalLogic Kharkiv) was delivered at GlobalLogic Kharkiv MS TechTalk on June 13, 2017.
- MongoDB is well-suited for systems of engagement that have demanding real-time requirements, diverse and mixed data sets, massive concurrency, global deployment, and no downtime tolerance.
- It performs well for workloads with mixed reads, writes, and updates and scales horizontally on demand. However, it is less suited for analytical workloads, data warehousing, business intelligence, or transaction processing workloads.
- MongoDB shines for use cases involving single views of data, mobile and geospatial applications, real-time analytics, catalogs, personalization, content management, and log aggregation. It is less optimal for workloads requiring joins, full collection scans, high-latency writes, or five nines u
This document provides a summary of Oracle OpenWorld 2014 discussions on database cloud, in-memory database, native JSON support, big data, and Internet of Things (IoT) technologies. Key points include:
- Database Cloud on Oracle offers pay-as-you-go pricing and self-service provisioning similar to on-premise databases.
- Oracle Database 12c includes an in-memory option that can provide up to 100x faster analytics queries and 2-4x faster transaction processing.
- Native JSON support in 12c allows storing and querying JSON documents within the database.
- Big data technologies like Oracle Big Data SQL and Oracle Big Data Discovery help analyze large and diverse data sets from sources like
The document discusses NoSQL databases and their advantages compared to SQL databases. It defines NoSQL as any database that is not relational and describes the main categories of NoSQL databases - key-value stores, document databases, wide column stores like BigTable, and graph databases. It also covers common use cases for different NoSQL databases and examples of companies using NoSQL technologies like MongoDB, Cassandra, and HBase.
The key points covered during the presentation were:
• An overview of PostgreSQL and Advanced Server
• EnterpriseDB’s support of community PostgreSQL including add-ons & support
• A high-level feature comparison of PostgreSQL and Advanced Server
• Working with Oracle and Advanced Server
• Summary characteristics for choosing one database or the other
If you have any questions please email info@enterprisedb.com.
Relational databases vs Non-relational databasesJames Serra
There is a lot of confusion about the place and purpose of the many recent non-relational database solutions ("NoSQL databases") compared to the relational database solutions that have been around for so many years. In this presentation I will first clarify what exactly these database solutions are, compare them, and discuss the best use cases for each. I'll discuss topics involving OLTP, scaling, data warehousing, polyglot persistence, and the CAP theorem. We will even touch on a new type of database solution called NewSQL. If you are building a new solution it is important to understand all your options so you take the right path to success.
This is my presentation "CDC to the Max" as presented at the EMEA PUG Challange 16-11-2017. Descibed is an approach how to utilize OpenEdge Change Data Capture (CDC) to offload information to external systems to enable possibilities typically not very well suited for relational databases.
NoSQL is a non-relational database approach that accommodates a wide variety of data models. It is non-relational, distributed, flexible and scalable. The four main types of NoSQL databases are document databases, key-value stores, column-oriented databases, and graph databases. MongoDB is an example of a document-oriented NoSQL database. NoSQL databases offer benefits over relational databases like flexible schemas, horizontal scalability, and fast queries. Hadoop is an open source framework for distributed storage and processing of large datasets across clusters of computers. It uses MapReduce as its parallel programming model and the Hadoop Distributed File System for storage.
Optymalizacja środowiska Open Source w celu zwiększenia oszczędności i kontroliEDB
The document discusses optimizing the Open Source environment to increase savings and control. It covers evolving database infrastructure models in enterprises to get more for less. Key areas discussed include where Postgres can be most easily implemented, Postgres advances that enable new data types and challenges, and how to assess whether and how to implement Postgres. Case studies are presented that demonstrate cost savings and performance benefits organizations achieved by adopting Postgres.
Slides from workshop held on 12/14 in Asbury Park, NJ
http://www.meetup.com/Jersey-Shore-Tech/events/148118762/?gj=ro2_e&a=ro2_gnl&rv=ro2_e&_af_eid=148118762&_af=event
Recent advances in Postgres have propelled the database forward to meet today’s data challenges. At some of the world’s largest companies, Postgres plays a major role in controlling costs and reducing dependence on traditional providers.
This presentation addresses:
* What workloads are best suited for introducing Postgres into your environment
* The success milestones for evaluating the ‘when and how’ of expanding Postgres deployments
* Key advances in recent Postgres releases that support new data types and evolving data challenges
This presentation is intended for strategic IT and Business Decision-Makers involved in data infrastructure decisions and cost-savings.
http://www.opitz-consulting.com/go/3-5-898
Smartphones haben unsere Welt im Schnellgang erobert. Die Tablets folgen nicht minder schnell nach. Was fasziniert uns so daran? Welche neuen Möglichkeiten bieten sich für das Business? Welchen Einfluss wird das allgegenwärtige HTML5 haben? Wie bekomme ich mobile Lösungen architektonisch optimal in meine SOA-Landschaft integriert, und welche Vorteile gewinne ich bei der Prozessautomatisierung? Diese Session liefert sowohl einen Überblick als auch Antworten für eine neue Klasse von Architekturfragen.
Die SOA-Experten Torsten Winterberg und Guido Schmutz hielten diesen Fachvortrag bei der DOAG Konferenz und Ausstellung am 20.11.2013 in Nürnberg.
--
Über uns:
Als führender Projektspezialist für ganzheitliche IT-Lösungen tragen wir zur Wertsteigerung der Organisationen unserer Kunden bei und bringen IT und Business in Einklang. Mit OPITZ CONSULTING als zuverlässigem Partner können sich unsere Kunden auf ihr Kerngeschäft konzentrieren und ihre Wettbewerbsvorteile nachhaltig absichern und ausbauen.
Über unsere IT-Beratung: http://www.opitz-consulting.com/go/3-8-10
Unser Leistungsangebot: http://www.opitz-consulting.com/go/3-8-874
Karriere bei OPITZ CONSULTING: http://www.opitz-consulting.com/go/3-8-5
http://www.opitz-consulting.com/go/3-5-898
Smartphones and tablets conquered our world. Which new opportunities are there for our businesses? Which influence has the omnipresent HTML5? How can I integrate mobile solutions in an optimal architectural way in my SOA landscapes and which kind of advantages do I gain for business process automation? This session delivers answers and puts current buzzwords like Big Data, Cloud, internet of things, HTML5 and mobile in the context of BPM and integration. Thereby we derive a reference architecture for Oracle SOA Suite, OSB, BPM Suite, Enterprise Gateway, Webcenter, ADF Mobile, etc., which makes all the buzzwords easily manageable in our daily IT work and prevents you from making mistakes others already did.
Torsten Winterberg und Guido Schmutz, both well-respected SOA Experts, presented this session at German Oracle User Communities’s Conference (DOAG Konferenz) at nov 20th 2013 in Nuremberg, Germany.
--
- - -
About us:
OPITZ CONSULTING is a leading project specialist for custom-build applications and individual business intelligence solutions in the German market. The company's ambition is to help organizations to be better than their competitors. To achieve this OPITZ CONSULTING analyses the individual competitive edge the customer has, optimizes business processes for process automation and IT-support, chooses and designs appropriate system architectures, develops and implements solutions and guarantees a 24/7 support and application maintenance. To ensure the necessary skill and qualification OPITZ CONSULTING has established a training center for customers and the internal staff.
Since 1990 over 600 customers have a long lasting and successful business relationship with OPITZ CONSULTING. Over 2/3 of the German stock index (DAX) companies rely on services from the 400+ OPITZ CONSULTING consultants. OPITZ CONSULTING maintains offices in Bad Homburg, Berlin, Essen, Gummersbach, Hamburg, Munich, Nuremberg and Kraków and Warsawa (Poland).
About us: http://www.opitz-consulting.com/en/about_us
Services: http://www.opitz-consulting.com/en/leistungsangebot
Career: http://www.opitz-consulting.com/en/career
Dropping ACID: Wrapping Your Mind Around NoSQL DatabasesKyle Banerjee
This document discusses NoSQL databases as an alternative to traditional relational databases. It provides an overview of different types of NoSQL databases like document stores, wide column stores, key-value stores and graph databases. It also discusses advantages of NoSQL databases like horizontal scalability and ease of use with large amounts of unstructured data, as well as disadvantages like lack of transactions and joins. The document recommends choosing a database based on the type of queries, data size, read/write needs, and whether the data needs to be accessed by other applications.
Solr Under the Hood at S&P Global- Sumit Vadhera, S&P Global Lucidworks
This document summarizes S&P Global's use of Solr for search capabilities across their large datasets. It discusses how S&P Global indexes over 50 million documents into Solr monthly and handles over 5 million queries per week. It outlines challenges faced with an on-premise Solr deployment and how migrating to Solr Cloud helped address issues like performance, availability, and scalability. Next steps discussed include improving relevancy through data science, continuing to leverage new Solr features, and exploring ways to integrate machine learning into search capabilities.
When it comes time to select database software for your project, there are a bewildering number of choices. How do you know if your project is a good fit for a relational database, or whether one of the many NoSQL options is a better choice?
In this webinar you will learn when to use MongoDB and how to evaluate if MongoDB is a fit for your project. You will see how MongoDB's flexible document model is solving business problems in ways that were not previously possible, and how MongoDB's built-in features allow running at scale.
Topics covered include:
Performance and Scalability
MongoDB's Data Model
Popular MongoDB Use Cases
Customer Stories
DocumentDB is a fast, globally distributed, multi-model NoSQL database service. It provides automatic scaling of storage and throughput, high availability across regions, flexible data models, and developer productivity with support for SQL and JavaScript queries. Customers can use DocumentDB for building scalable applications that need to handle large volumes of data across any number of regions worldwide with low latency and high availability.
This document provides an overview of several document database technologies including MongoDB, CouchDB, and RavenDB. It discusses key architectural considerations for using document databases such as their schema-free model, eventual consistency, ability to model object aggregates, scaling out through sharding and replication, need for queries to use indexes, and ongoing administration requirements. It also presents two case studies where document databases were used for a survey system and a CRM.
EPAS + Cloud = Oracle Compatible Postgres in MinutesEDB
On this webinar, EnterpriseDB's Jamie Watt, VP of Global Support Services and Ajay Patel, Cloud Services Delivery Manager, will demonstrate how to launch on-demand Oracle compatible Postgres Databases in minutes on EDB Cloud Database Service (CDS). Specifically, the webinar covers the following:
- Instant Provisioning, Scaling and Managing Postgres
- Built-in Monitoring, High Availability, and Backup tools
- Using Pre-defined templates for standard configurations
ГАННА КАПЛУН «noSQL vs SQL: порівняння використання реляційних та нереляційни...QADay
This document compares noSQL and SQL databases, providing examples of each. NoSQL databases are non-relational and have dynamic schemas while SQL databases are relational and have predefined schemas. Some common noSQL databases mentioned are MongoDB, DynamoDB, Cassandra, and Neo4j, while examples of SQL databases include Oracle, MySQL, PostgreSQL, and SQLite. The document then discusses using MongoDB for a production application that stores tree-structured and dynamic data more easily than a SQL database would. It also covers combining MongoDB and Oracle by storing documents in Oracle and metadata in MongoDB.
Similar to Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14-03-2017) (20)
Introduction to web application development with Vue (for absolute beginners)...Lucas Jellema
In this slide deck I show you how you can easily and quickly create quite rich web applications with Vue 3 – without having to study complex concepts or understand many technical details. I have only recently learned how to work with Vue 3 myself and now is the best time for me to share my learning experience (and my enthusiasm) with you. I know what I found essential to understand and what most got me excited in these early steps (what was a little bit hard to grasp). I believe that I can present my steps and guide you to experience the same fun and have a similarly gratifying experience. I am not an expert in this subject – I have barely learned how to walk and that is why I can help you with these first steps with Vue.
In this deck, I do not explain how Vue works. I do not really know that. I will show you how to work with it and how to create web applications that are functional, appealing, fast and responsive.
The approach I am taking is straightforward:
• I will tell you a little bit about web development, browsers and reactive frameworks
• I will show the hello world of Vue applications
• I will explain about components and nesting, events, data binding and reactive behavior and demonstrate these concepts
• I will introduce Vue UI Component libraries – and with no effort at all we will launch our application to the next level – with rich components to explore, manipulate, visualize data collections
• We will publish the web application from our development environment to where the whole world could see it – using GitHub Pages
• As bonus topic – we discuss state management
At the end of this session you will be able to quickly create a simple yet rich web application with Vue 3. You have a starting point to further evolve your skills with the many online resources I am convinced that you will enjoy your newfound powers and the simplicity and power of Vue 3.
Note: a tutorial accompanies this slide deck - see https://github.com/lucasjellema/code-face-vue3-intro-reactiive-webapps-aug2023/blob/main/README.md
Making the Shift Left - Bringing Ops to Dev before bringing applications to p...Lucas Jellema
The document discusses bringing operations considerations into the development process earlier, referred to as "shifting left." It advocates designing applications with operations in mind from the beginning. This includes understanding operational objectives, constraints, and service level agreements. Application telemetry and monitoring are also important to incorporate from the start. The document provides examples of how to implement operational practices like deployments, health checks, and incident response processes in a shifted left model where development and operations work more closely together.
Lightweight coding in powerful Cloud Development Environments (DigitalXchange...Lucas Jellema
The document discusses lightweight coding in powerful cloud development environments using Gitpod. It describes Gitpod as providing a preconfigured Linux development environment in the browser or on local machine. The document outlines key Gitpod features like open source project collaboration, costs which are free for 50 hours per month, and benefits like clean environments and efficient resource usage. It also briefly mentions other tools like GitHub Codespaces.
Apache Superset - open source data exploration and visualization (Conclusion ...Lucas Jellema
Introducing Apache Superset - an open source platform for data exploration, visualization and analysis - co-starring Trino and Steampipe for providing SQL access to many non-SQL data sources.
CONNECTING THE REAL WORLD TO ENTERPRISE IT – HOW IoT DRIVES OUR ENERGY TRANSI...Lucas Jellema
Enterprise IT systems are deaf, blind and highly insensitive. They do not know what is going on in the outside world. Through Internet of Things technology, we provide eyes, ears and hands that allow enterprises to learn about and react in real time to events in the physical world. The energy transition at a major Dutch energy company (Eneco) is powered by IoT technology – to steer and sometimes curtail windmills and solar farms and to coordinate local energy production and trade. This session shows you how the physical world was connected to the customer portals and apps, asset management systems and Kafka platform through the Azure cloud based IoT Hub en Edge, digital twin, serverless functions, timeseries datastores and streaming data analysis. It is a story about technological innovation on top of existing foundations and of a vision for business and our society at large.
Help me move away from Oracle - or not?! (Oracle Community Tour EMEA - LVOUG...Lucas Jellema
I hear this aspiration from a growing number of organizations. Sometimes as a quite literal question. This however is merely half of a wish. Apparently, organizations want to quit with one thing — but have not yet stipulated what they desire instead. What is the objective that is pursued here? Only to get rid of Oracle? It will become clear why you should give a considerable thought about dropping Oracle, or any other vendors’ technology, when you’re not pleased with your current IT situation. You need to focus on the actual problems and objectives and define the suitable roadmap to fit your real needs. It turns out that the quest is usually for modernization and flexibility - and Oracle can very well be a part of that future.
Organizations with decades of investment in Oracle technology sometimes (and increasingly) express a wish to move away from Oracle. In this session, we will first explore where the desire to move away from Oracle might come from. Then we describe what the term Oracle represents — more than 2.000 products on all layers in the technology stack and in different business areas. Finally, we map out what the ‘moving away from’ consists of: defining where you ‘move to’ and subsequently actually going there.
It will become clear why you should give considerable thought about dropping Oracle, or any other vendors’ technology, when you’re not pleased with your current IT situation. You need to focus on the actual problems and objectives and define the suitable roadmap to fit your real needs. It turns out that the quest is usually for modernization and flexibility - and Oracle can very well be a part of that future.
Original storyline in this Medium Article: https://medium.com/real-vox/what-if-companies-say-help-me-move-away-from-oracle-ffbbc95afc4f
IoT - from prototype to enterprise platform (DigitalXchange 2022)Lucas Jellema
In 2019 the company started a small scale IoT project: smart meters in consumer homes, a cloud based IoT platform for device management, metrics collecting, monitoring and real time data processing. From the initial 12 devices and this single use case, the initiative has rapidly scaled, to tens of thousands devices - including entire wind parks and solar farms - and seven substantial business cases, not just for harvesting data but increasingly for real time actuation. The IoT Platform is feeding the brain at the heart of the enterprise - through an event streaming platform and an API platform. It supports complex operations with anomaly detection on metrics streams and device and communication monitoring. This session tells about the eye catching business cases - what are business objectives and results - and explains the journey since the start. It continues the story presented at DigitalXchange 2020 - discussing technical challenges and solutions as well as organizational aspects. Areas of particular interest: edge processing, data analytics and machine learning.
Who Wants to Become an IT Architect-A Look at the Bigger Picture - DigitalXch...Lucas Jellema
Pitch: The movie The Matrix made it clear: The Architect is powerful. How to be(come) and IT architect? What do you do, what do you need to know, is it fun and why? Using real world examples, core principles and useful tools, this session introduces the subtle art of designing and realizing flexible IT architectures. </p><p>Taking a step back to get and create an overview, frequently asking why to get to the real intention, bringing aspects such as cost, scale, time and change and business strategy into the design and bridging the gap between business owners, process managers and technical specialists. One way to define the responsibility of an IT architect. In this session, we will discuss what is expected of the architect and what you need to do for that and what you could use to get it done. How do you get started as an architect, how to grow in that role? We discuss a number of real life architectural challenges and solution design. And discuss a number of architecture principles, patterns, and powers to apply. Never stop programming - but perhaps rise to the architecture challenge too.
Notes: Many IT professionals aspire to become architects. Many architects wonder what it is they have to do. After 27 years in IT I find I have slowly and steadily moved into a role that I can probably use the label architect for, although still with some reluctance. What exactly does that mean - IT architect? While I may not have all answers and the ultimate truth and wisdom, I do have many architectural challenges to discuss and some core principles to share and a number of tips, tricks and tools to recommend that will help anyone get started or grow in a role as architect for software and IT systems. Elements that make an appearance include cloud, agile, DevOps, microservices, persistence, business, powers of persuasion, diagramming, cost, security, software engineering, data.
Outline: - two real world examples (one new business initiative, one running and struggling project) and how to approach them with an architect's mind - core principles to apply , patterns to us, what to unearth (the power question of WHY) - architecture products: what do you deliver as an architect; how do you ensure agility? - how to be effective? bringing your design to life - communication with stakeholders/powers of persuasion, monitoring adherence, being pragmatic but not lose grip; - anecdotal evidence from several small and large product teams - the good and also the ugly (architectural oversights and the consequences)
some specific answers to address - how much technical knowledge and programming skills does an architect require? What other knowledge is required and how to stay on top of your game? how to get going: first steps towards be(com)ing and architect?
Steampipe - use SQL to retrieve data from cloud, platforms and files (Code Ca...Lucas Jellema
Introduction to Steampipe - a tool for retrieving data and metadata about cloud resources, platform resources and file content - all through SQL. Data from clouds, files and platforms can be joined, filtered, sorted, aggregated using regular SQL. Steampipe offers a very convenient way to get hold of data that describes the environment in detail.
Automation of Software Engineering with OCI DevOps Build and Deployment Pipel...Lucas Jellema
Automation of software delivery has several advantages. Prevention of human error is certainly one. Consistent and complete execution of tried and tested build and deployment tasks as the only way to apply changes in the live environment. Once the pipelines have been set up, the engineers can focus on the software and applying the required changes to it. To bring that software all the way to production is a breeze. Oracle Cloud Infrastructure offers the DevOps service, introduced in the Summer of 2021. This service comes with git style code repositories, build servers and build pipelines, artifact repositories as well as deployment pipelines. This session introduces OCI DevOps and demonstrates how software can be built and deployed on OKE Kubernetes, Compute Instance VMs and Oracle Functions. From simple source code an application is put in production without manual intervention in the build and deployment process.
Introducing Dapr.io - the open source personal assistant to microservices and...Lucas Jellema
Dapr.io is an open source product, originated from Microsoft and embraced by a broad coalition of cloud suppliers (part of CNFC) and open source projects. Dapr is a runtime framework that can support any application and that especially shines with distributed applications - for example microservices - that run in containers, spread over clouds and / or edge devices.
With Dapr you give an application a "sidecar" - a kind of personal assistant that takes care of all kinds of common responsibilities. Capturing and retrieving state, publishing and consuming messages or events. Reading secrets and configuration data. Shielding and load balancing over service endpoints. Calling and subscribing to all kinds of SaaS and PaaS facilities. Logging traces across all kinds of application components and logically routing calls between microservices and other application components. Dapr provides generic APIs to the application (HTTP and gRPC) for calling all these generic services – and provides implementations of these APIs for all public clouds and dozens of technology components. This means that your application can easily make use of a wide range of relevant features - with a strict separation between the language the application uses for this (generic, simple) and the configuration of the specific technology (e.g. Redis, MySQL, CosmosDB, Cassandra, PostgreSQL, Oracle Database, MongoDB, Azure SQL etc) that the Dapr sidecar uses. Changing technology does not affect the application, but affects the configuration of the Sidecar. Dapr can be used from applications in any technology - from Java and C#/.NET to Go, Python, Node, Rust and PHP. Or whatever can talk HTTP (or gRPC).
In this Code Café I will introduce you to Dapr.io. I will show you what Dapr can do for you (application) and how you can Dapr-izen an application. I'll show you how an asynchronously collaborative system of microservices - implemented in different technologies - can be easily connected to Dapr, first to Redis as a Pub/Sub mechanism and then also to Apache Kafka without modifications. Then we do - with the interested parties - also a hands-on in which you will apply Dapr yourself . In a short time you get a good feel for how you can use Dapr for different aspects of your applications. And if nothing else, Dapr is a very easy way to get your code with Kafka, S3, Redis, Azure EventGrid, HashiCorp Consul, Twillio, Pulsar, RabbitMQ, HashiCorp Vault, AWS Secret Manager, Azure KeyVault, Cron, SMTP, Twitter, AWS SQS & SNS, GCP Pub/Sub and dozens of other technology components talk.
How and Why you can and should Participate in Open Source Projects (AMIS, Sof...Lucas Jellema
For a long time I have been reluctant to actively contribute to an open source project. I thought it would be rather complicated and demanding – and that I didn't have the knowledge or skills for it or at the very least that they (the project team) weren't waiting for me.
In December 2021, I decided to have a serious input into the Dapr.io project – and now finally to determine how it works and whether it is really that complicated. In this session I want to tell you about my experiences. How Fork, Clone, Branch, Push (and PR) is the rhythm of contributing to an open source project and how you do that (these are all Git actions against GitHub repositories). How to learn how such a project functions and how to connect to it; which tools are needed, which communication channels are used. I tell how the standards of the project – largely automatically enforced – help me to become a better software engineer, with an eye for readability and testability of the code.
How the review process is quite exciting once you have offered your contribution. And how the final "merge to master" of my contribution and then the actual release (Dapr 1.6 contains my first contribution) are nice milestones.
I hope to motivate participants in this session to also take the step yourself and contribute to an open source project in the form of issues or samples, documentation or code. It's valuable to the community and the specific project and I think it's definitely a valuable experience for the "contributer". I looked up to it and now that I've done it gives me confidence – and it tastes like more (I could still use some help with the work on Dapr.io, by the way).
Microservices, Apache Kafka, Node, Dapr and more - Part Two (Fontys Hogeschoo...Lucas Jellema
Apache Kafka is one of the best known enterprise grade message brokers – created at LinkedIn, donated to the Apache software foundation and used in an ever growing number of organizations to provide a backbone for asynchronous communication. This session introduces Apache Kafka – history, concepts, community and tooling. In a hands on lab, participants will create topics, publish and consume messages and get a general feel for Kafka. Simple microservices are developed in NodeJS – publishing to and consuming from Apache Kafka.
Dapr.io has support for Apache Kafka. Using Kafka through Dapr is very straightforward as is explained and demonstrated and applied in a second handson lab – with applications in various programming languages. Participants will even be able to exchange events across their laptops – through a cloud based Kafka broker.
Use of Apache Kafka in several architecture patterns is discussed – such as data integration, microservices, CQRS, Event Sourcing – along with a number of real world use cases from several well known organizations. The Kafka Connector framework is introduced – a set of adapters that allow us to easily connect Kafka to sources and sinks – where respectively change events are captured from and messages are published to.
Bonus Lab: Apache Kafka is ran on Kubernetes as is Dapr.io. Multiple mutually interacting microservices are deployed on the same local Kubernetes cluster.
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)Lucas Jellema
This session does a quick recap of microservices: why do we want them, what problems do they solve and what are the principles around designing and implementing them? The Dapr.io runtime framework for distributed applications is introduced. Dapr provides a sidecar (almost like a personal assistant to a manager) to an application or microservice, a companion process that handles common tasks such as storing and retrieving state, consuming and publishing messages and events, invoking external services and other microservices as well as handling incoming requests. Participants will do a handson lab with Dapr.io and learn how to quickly implement interactions with various technologies, including Redis and MySQL.
Node(JS) is introduced – a server side JavaScript-based programming language that can be used well for implementing microservices. Some of the main characteristics of NodeJS are discussed (functional programming, asynchronous flows, NPM package manager) as well as common use cases (handle incoming HTTP requests, invoke REST APIs). In the second lab, Node and Dapr are used together to implement microservices that interact with databases and message brokers and each other – in a decoupled fashion.
6Reinventing Oracle Systems in a Cloudy World (RMOUG Trainingdays, February 2...Lucas Jellema
The cloud is changing many things. Even the decision to not (yet) adopt cloud is one to make explicitly. Now is a time for any organization to reconsider the IT landscape. For each system we should make a conscious ruling on its roadmap. The 6R model suggests six ways to move a system forward.
This session uses the 6R model and applies it specifically to Oracle technology based systems: what are the options and considerations for Oracle Database, Oracle Fusion Middleware, custom applications, and other red components? What future should we consider and how do we choose? The paths chosen by several Oracle-heavy users is presented to illustrate these options and the decision making process. Oracle Cloud Infrastructure and Autonomous Database play a role, as do Azure IaaS and Azure Managed Database as well as on premises systems. Latency, recovery, scalability, licenses, automation, lock-in, skills, and resources all make their appearance.
Help me move away from Oracle! (RMOUG Training Days 2022, February 2022)Lucas Jellema
Organizations with decades of investment in Oracle technology sometimes (and increasingly) express a wish to move away from Oracle. In this session, we will first explore where the desire to move away from Oracle might come from. Then we describe what the term Oracle represents -- more than 2.000 products on all layers in the technology stack and in different business areas. Finally, we map out what the 'moving away from' consists of: defining where you 'move to' and subsequently actually going there.
It will become clear why you should give considerable thought about dropping Oracle, or any other vendors' technology, when you're not pleased with your current IT situation. You need to focus on the actual problems and objectives and define the suitable roadmap to fit your real needs. It turns out that the quest is usually for modernization and flexibility - and Oracle can very well be a part of that future.
DevOps is a term used in many places and unfortunately also to mean many different things. This presentation (largely in Dutch) paints the DevOps picture. While it may not give a clear cut definition (there does not seem to be one) it certainly makes clear what DevOps is about, what objectives and origins are and which factors enable and drive DevOps.
Conclusion Code Cafe - Microcks for Mocking and Testing Async APIs (January 2...Lucas Jellema
Microcks is a tool for API Mocking and Testing. In this presentation an overview of the support in Microcks for asynchronous APIs - the event publishing and consuming behavior of services and applications
Cloud native applications offer scalability, flexibility, and optimal use of compute resources. Serverless functions interacting through events, leveraging cloud capabilities for persistent storage and automated operations take organization to the next level in IT. This session demonstrates polyglot Functions interacting with native cloud services for events and persistence (Object Storage and NoSQL Database) and leveraging the Key and Secrets Vault, Monitoring and Notifications services for operational control. A lightweight API Gateway is used to expose APIs to external consumers. Infrastructure as Code is the guiding principle in deploying both cloud resources and application components, through OCI CLI and Terraform. This session leverages many cloud native (enabling) services in Oracle Cloud Infrastructure. The session will introduce concepts, then spend most of the time on live demonstrations. All sources are shared with the audience, to allow participants to create the same application in their own cloud tenancy. What is so great about Cloud Native Applications? How do you create one? I will explain the first and demonstrate the second. On Oracle Cloud Infrastructure, using services that anyone can use for free, I will live create a cloud native application that streams, persists, notifies, scales, monitors Benefits: - get to know many different OCI services - understand the meaning, purpose and benefits of cloud native development - learn how to take your own first steps in OCI - for free!
A captivating AI chatbot PowerPoint presentation is made with a striking backdrop in order to attract a wider audience. Select this template featuring several AI chatbot visuals to boost audience engagement and spontaneity. With the aid of this multi-colored template, you may make a compelling presentation and get extra bonuses. To easily elucidate your ideas, choose a typeface with vibrant colors. You can include your data regarding utilizing the chatbot methodology to the remaining half of the template.
What is OCR Technology and How to Extract Text from Any Image for FreeTwisterTools
Discover the fascinating world of Optical Character Recognition (OCR) technology with our comprehensive presentation. Learn how OCR converts various types of documents, such as scanned paper documents, PDFs, or images captured by a digital camera, into editable and searchable data. Dive into the history, modern applications, and future trends of OCR technology. Get step-by-step instructions on how to extract text from any image online for free using a simple tool, along with best practices for OCR image preparation. Ideal for professionals, students, and tech enthusiasts looking to harness the power of OCR.
Ansys Mechanical enables you to solve complex structural engineering problems and make better, faster design decisions. With the finite element analysis (FEA) solvers available in the suite, you can customize and automate solutions for your structural mechanics problems and parameterize them to analyze multiple design scenarios. Ansys Mechanical is a dynamic tool that has a complete range of analysis tools.
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio, Inc.
Alluxio Webinar
June. 18, 2024
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
- Jianjian Xie (Staff Software Engineer, Alluxio)
As Trino users increasingly rely on cloud object storage for retrieving data, speed and cloud cost have become major challenges. The separation of compute and storage creates latency challenges when querying datasets; scanning data between storage and compute tiers becomes I/O bound. On the other hand, cloud API costs related to GET/LIST operations and cross-region data transfer add up quickly.
The newly introduced Trino file system cache by Alluxio aims to overcome the above challenges. In this session, Jianjian will dive into Trino data caching strategies, the latest test results, and discuss the multi-level caching architecture. This architecture makes Trino 10x faster for data lakes of any scale, from GB to EB.
What you will learn:
- Challenges relating to the speed and costs of running Trino in the cloud
- The new Trino file system cache feature overview, including the latest development status and test results
- A multi-level cache framework for maximized speed, including Trino file system cache and Alluxio distributed cache
- Real-world cases, including a large online payment firm and a top ridesharing company
- The future roadmap of Trino file system cache and Trino-Alluxio integration
4. TRADITIONAL APPROACH:
WITHOUT MUCH REGARD FOR
• What is it used for?
• By whom, where, in what way, using which tools
• Ratio between writes and reads
• Ad hoc access?
• What format is it in/should it be in?
• What confidentiality & integrity is required?
• How much of it?
• How long relevant (hot vs cold vs dead)?
• How fine grained and how accurate?
• How much change?
5. DATABASE: SINGLE POINT OF …
SCALABILITY | AVAILABILITY | CONSISTENCY
• Single Point of …
• What goes where …
6. RELATIONAL DATABASES
• Based on relational model of data (E.F. Codd), a mathematical
foundation
• Uses SQL for query, DML and DDL
• Transactions are ACID (Atomicity, Consistency, Isolation, Durability)
• All or nothing
• Constraint Compliant
• Individual experience [in a multi-session environment]
(aka concurrency)
• Down does not hurt
6
7. THE HOLY GRAIL OF NORMALIZATION
• Normalize to prevent data redundancy, discrepancies (split brain) and
storage waste
• Pragmatic approaches - recognizing the fact that some data is read far
more frequently than that it is created and modified
• Materialized View
• Virtual Column
• Index
• Sharding
• Replication
• Data Guard
• Data Warehouse & Data Marts
• Client Side Caching
• In Memory Database
8. ACID COMES AT A COST
• Transaction results have to be persisted in order to guarantee D
• Concurrency requires some degree of locking (and multi-versioning) in order to have I
• Constraint compliance (unique key, foreign key) means all data hangs together (as do all
transactions) in order to have C
8
9. ACID COMES AT A COST
• ACID means:
• All data closely together (and sometimes far from
applications and users)
• All transactions centrally organized
• All data persisted as part of the transaction
• ACID means:
• Hard to scale out (horizontally)
• Oracle offers scale up with Real Application Clusters
9
10. THE RELATIONAL MODEL IN PRACTICE
• Traditional Relational Data Model has severe impact on physical disk
performance
• Transaction Log => Sequential Write (append to file)
• Data Blocks require much more expensive Random Access disk writes
• Indexes (B-Tree, Bitmap, …) are used to speed up query (read) performance
• and slow down transactions
• Relational data does not [always] map naturally to data format required in
application (OO, JSON, XML)
• Capability to join and construct ad-hoc queries across the entire data model is
powerful
• Declarative integrity constraints allow for strict enforcement of data quality rules
• “the data may be non sensical, but at least it adheres to the rules, whatever the origin”
• SQL has been embraced by huge numbers of tools & technologies and IT/Data
professionals
11. DATABASES RE-EVALUATED
• Not all use cases require ACID (or can afford it)
• Read only (product catalog for web shops)
• Inserts only and no (inter-record) constraints
• Big Data collected and “dumped” in Data Lake (Hadoop) for subsequent
processing
• High performance demands
• Not all data needs structured formats or structured querying and JOINs
• Entire documents are stored and retrieved based on a single key
• Sometimes – scalable availability is more important than Consistency – and
ACID is sacrificed
• CAP-theorem states: Consistency [across nodes], Availability and Partition
tolerance can not all three be satisfied
11
12. USE CASE: WEBSHOP
• Webshop – 1M visitors per day
• Product catalog consists of millions of records
• The web shop presents: product description, images, reviews, pricing
details, related offerings, stock status
• Products are added and updated and removed every day
• Although most products do not change very frequently
• Some vendors do bulk manipulation of product details
Products Product updates
Webshop visits
- searches
- product details
- orders
13. USE CASE: WEBSHOP
TECHNOLOGY AND ARCHITECTURE
CONSIDERATIONS
Products Product updates
Webshop visits
- searches
- product details
- orders
firewall
Data manipulation
Data Quality (enforcement)
<10K transactions
Batch jobs next to online
Speed is nice
Read only
On line
Speed is crucial
XHTML & JSON
> 1M visits
14. Products
Products
USE CASE: WEBSHOP
TECHNOLOGY AND ARCHITECTURE CONSIDERATIONS
Products
Webshop visits
- searches
- product details
- orders
firewall
Data manipulation
Data Quality (enforcement)
<10K transactions
Batch jobs next to online
Speed is nice
Read only
On line
Speed is crucial
XHTML & JSON
> 1M visits
DMZ
Read only
JSON documents
Images
Text Search
Scale Horizontally
Products
Nightly generation
Product updates
15. A FEW OBSERVATIONS
• Redundancy
• for performance, scalability, availability
• Commodity hardware for scale out architecture
• Data (documents) prepared for specific application usage (super MV)
• Format
• Search paths
• Time lag in data modifications is accepted
• Perhaps even desirable to offer a consistent view to webshop users
• Transactions in one engine
• ACID still in place
• Transactions do not compete with reads
• vying for the same physical resources
16. NOSQL AND BASE
• NoSQL arose because of performance and scalability
challenges with Web Scale operations
• NoSQL is a label for a wide variety of databases that lack some
aspect of a true relational database
• ACID-ness, SQL, relational model, constraints
• The label has been used since 2009
• Perhaps NoREL would be more appropriate
• Some well known NoSQL products are
• Cassandra, MongoDB, Redis, CouchDB, …
• BASE as alternative to ACID:
• basically available, soft state, eventually consistent
(after a short duration)
16
17. TYPICAL FOR NOSQL
• Focus on speed, availability and scalability
• Horizontal scale out – distributed with load balancing and fail-over
• No (predefined) Data Structure
• Integrity primarily protected by application logic
• Open Source (most offerings are, not all: MarkLogic)
• Close(r) attention for how the data is used
• Application oriented data format and search paths and specialized database per
application (microservice, capability)
• Similar to the switch from SOA to API/Microservice
• Reads (far) more relevant than writes
• Data redundancy & denormalization
• No data access through SQL
18. NOSQL MEANS:
NO DATA ACCESS THROUGH SQL
• However
• Data Professionals and
Developers speak SQL
• Reporting, Dashboarding,
ETL, BI tools speak SQL
• There is no common query
language across NoSQL
products
19. NO DATA ACCESS THROUGH SQL
• However
• Data Professionals and
Developers speak SQL
• Reporting, Dashboarding,
ETL, BI tools speak SQL
• There is no common query
language across NoSQL
products
• Attempts from many vendors to create drivers that translate SQL
statements into NoSQL commands for the specific target database
• To protect existing investments in SQL – skills, tools, applications, reports, ..
20. TYPES OF NOSQL DATABASES
• Key-Value (from in-memory cache to hardware appliance –
Memcached, Hazelcast, Coherence, Redis, Oracle NoSQL/Berkeley
DB)
• Wide Column Store (Cassandra, Google BigTable, HBase)
• Document (JSON, XML, YAML and binary forms – MongoDB,
CouchDB, Couchbase, MarkLogic)
• Graph (networks of data, triplets, RDF stores – Neo4J)
• Miscellaneous: object, tabular, tuple, elastic search index, Hadoop
(hdfs) style, Kafka Topic
21. (LEADING) NOSQL DATABASE PRODUCTS
• MongoDB is (one of) the most popular (by any measure)
• Cloud (only):
• Google BigTable,
• AWS Dynamo
• Cache (in memory)
• ZooKeeper, Redis,
Memcached, …
• Hadoop/HDFS
• Oracle NoSQL
(fka Berkeley DB)
23. HISTORY OF MONGODB
• 10gen – startup from 2007 in New York City
• Developed database as component in Platform as a Service product
• Initial release of MongoDB: 2009
• 10gen changed its name to MongoDB Inc. in 2013
• Offers enterprise support, MongoDB Atlas cloud service, MongoDB Compass, Connectors
• Current stable release: 3.4.2 (3.5.2 is in preview)
• Open Source - GNU Affero General Public License and the Apache License.
• MongoDB == Humongous Database
• Evolution of MongoDB is still pretty fast
• Some crucial enterprise database capabilities were added fairly recently or are still lacking
• MongoDB is number one NoSQL database in terms of popularity
24. CORE ARCHITECTURE
• MongoDB stores JSON documents in a binary format (BSON)
• Documents are stored in collections (similar to row or records in tables)
• Interaction is through JavaScript
• taking the place of SQL for DML and query
• Written in C++
• Has the V8 JavaScript engine included
• Runs on Mac OS X, Windows, Solaris, and most flavors of Linux
• Source code is on GitHub
• Has GeoSpatial and Tekst indexes and search capabilities
• MongoDB, Inc. officially supports drivers for C, C++, C#, Erlang, Java, Node.js,
JavaScript, Perl, PHP, Python, Scala, and Ruby
25. ARCHITECTUUR, INSTALLATIE,
CONFIGURATIE, OPSLAG, SHELL
• Mongo draait op windows en linux.
• Windows install met msi file
• linux install met tgz file.
• Users en directories vrij te kiezen
• Mongod --dbpath db1-- port 27001
• Default path=datadb, default port=27017
• Mongo [-- port hostname:27001] [--shell some_json_file]
• Dbs, collections
25
26. IMPORT EXPORT
• Mongoimport
• Json
• Csv, tsv, maar hebben een descriptor file nodig
• mongoexport -h -d -c -o -q
• Leesbare output, minder geschikt voor backups
27. CRUD
• show dbs
• use db
• db.<collection>.find()
• db.<collection>.find().pretty()
• select: db.find({key:value,key:value})
• present: db.<collection>.find({key:value},{key:1,key:1,_id:0})
• explain: db.find({key:value,key:value}).explain("executionStats")
• db.companies.insert({Company:"AMIS",Event:"MongoDB Workshop"})
• db.companies.update({Company:"AMIS"},{$set: {Location:"Nieuwegein"}})
• Geen commit! Ofwel, autocommit.
28. CRUD
• db.companies.remove({"Company" : "AMIS"})
• $unset -> Om een veld uit een document te verwijderen.
• $inc -> om een numeriek veld te verhogen met een bepaalde waarde. Als het
veld niet bestaat wordt het toegevoegd (met de waarde van de verhoging)
• $push -> Om een waarde aan een array toe te voegen.
29. INDEXING, EXPLAIN
• _Id altijd geïndexeerd en unique
• Indexen helpen bij zoeken
• Indexen mogen compound zijn, dwz op meerdere keys gecombineerd, bv
index op woonplaats+leeftijd
• db.user.find().sort({ woonplaats:1, leeftijd: -1 }) De -1 betekent aflopend
geïndexeerd.
• Resultaten rechtstreeks uit index indien mogelijk (fat indexes in SQL)
• db.user.find({woonplaats:”Zoetermeer”}).explain(“executionStats”) geeft een
goed overzicht van het gevolgde plan
30. REPLICA SETS
• Replica sets zijn gedupliceerde collections
• Beveilliging tegen uitval
• De members besluiten onderling welk member primary is. Alleen
daarop kan DML plaatsvinden
• Altijd oneven aantal members, eventueel mbv een arbiter
• Secondary members kunnen wel voor select gebruikt worden, als je
eerst slaveOk() ingeeft
• Mogen op dezelfde host draaien, mogen ook op verschillende hosts
draaien.
• writeConcern: Je kan expliciet aangeven op hoeveel members je
DML uitgevoerd moet zijn voor je je prompt terug krijgt.
31. SHARDING
• Shard = splinter
• Range based partitioning
• Altijd index nodig op de shard key. Hoeft niet unique te zijn.
• Iedere shard is een proces. Bij voorkeur ieder proces zijn eigen host.
• Scale out proces
• Metadata staat in aparte config db. Typically 3 stuks.
• Clients verbinden met een mongos server. Dat is een instance
zonder database die uit de config haalt waar de data zich bevindt.
• Iedere shard kan bestaan uit replica sets.
32. SHARDING
• Data verdeelt zichzelf over chunks.
• Chunks kunnen splitten wanneer ze te groot worden (typically
100MB maar is aan te passen naar wens)
• Chunks kunnen migraten naar een andere chard. Mongo streeft
ernaar elke shard evenveel chunks te laten hebben.
• Tijdens migration kan DML gewoon doorgaan.
33. BACKUP & RESTORE
• Backup met mongodump
• Creëert een dump directory met daaronder een directory per
database en plaatst daarin 1 of meerdere .bson files, een binair
formaat.
• Kan vanaf ieder member van een replica set. Bij voorkeur eentje die
het niet druk heeft.
• Mongo kent geen transactions, dus backup is niet consistent. Het is
maar afwachten wat erin staat als er veel DML plaatsvindt.
• De tegenhanger heet mongorestore.
• Restore van bestaande database lukt niet: duplicate key op _id
34. MONGODB COMPASS
(AKA NOSQL DEVELOPER)
• The GUI for MongoDB
• Visually explore data
• Run ad hoc queries in seconds.
• Interact with your data with full CRUD functionality
• View and optimize your query performance.
• Separately installed product
• On Windows, OS X, Linux, …
• Can run against local and remote MongoDB servers
36. PROGRAMMING AGAINST MONGODB
• MongoDB officially supports drivers for C, C++, C#, Erlang, Java,
Node.js, JavaScript, Perl, PHP, Python, Scala, and Ruby
• Note: there is also a Connector for Hadoop
• Several REST APIs for MongoDB are available from the community
• RESTHeart (Java), Eve (Python),, Crest (Node), AMID, Kule, DreamFactory
Connect,
create,
update
Connect,
create,
update
37. MEAN STACK
• Modelled after LAMP
• End-to-End JavaScript/JSON
• Term coined in 2013 (by MongoDB)
38. INTERACTING WITH MONGODB
FROM NODE.JS
• MongoDB Node.js Driver 2.0
• Support for ECMAScript 6.0 – Promises for asynch
• http://mongodb.github.io/node-mongodb-native/2.0/
• Using mongodb in a NodeJS application
• npm install mongodb –save
• var MongoClient = require('mongodb').MongoClient;
Connect,
create,
update
39. CONNECTING TO SERVER
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
MongoClient.connect(url, function(err, db) {
console.log("Connected correctly to server.");
// DO YOUR THING WITH MONGODB
db.close();
console.log("Connection to database is closed.");
}) //connect()
40. RETRIEVE DATA (1)
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
MongoClient.connect(url, function(err, db) {
console.log("Connected correctly to server.");
db.collection('countries').find({},{"sort": [["area",-1]]})
.limit(20).toArray(function(err, results){
console.log("Name of Country Four " +results[3].name+ " and size: " +results[3].area);
// using cursors to retrieve data sets in a controlled fashion
// note: cursor implements NodeJS Stream - results can be piped
var cursor = db.collection('countries').find({"continent":"Asia"}, {"sort": "name"});
cursor.count(function(err, count){
console.log("Country count in Asia: "+count);
});
cursor.each(function(err, country){
console.log(country.name);
})//each cursor
41. RETRIEVE DATA (2)
Connect,
create,
update
var aggquery = [ {$sort: {area : -1}}
, {$group:{ _id: '$continent'
, largestCountry : {$first: "$name"}
}
}
];
var aggcursor = db.collection('countries').aggregate(aggquery);
aggcursor.each(function(err, result){
if (err) {
console.log(err);
} else if (result)
console.log(JSON.stringify(result));
}) //each aggcursor
var ccursor = db.collection('countries').find({});
// the cursor returned from find and aggregate implements a NodeJS (readable) Stream
ccursor.on('data', function(doc) {
console.log(doc);
});
ccursor.once('end', function() {
console.log(“Out of countries. Time to move on");
});
42. CONNECTING TO SERVER – ECMA 6
Connect,
create,
update
var MongoClient = require('mongodb').MongoClient;
var assert = require('assert'), co = require('co');
var mongodbHost = '127.0.0.1';
var mongodbPort = '27017';
var mongodbDatabase = 'world';
var url = 'mongodb://'+mongodbHost+':'+mongodbPort + '/' + mongodbDatabase;
co(function*() {
// Use connect method to connect to the Server
var db = yield MongoClient.connect(url);
console.log("Connected correctly to server");
// DO YOUR THING WITH MONGODB
// Close the connection
db.close();
}).catch(function(err) {
console.log(err.stack);
});
43. RETRIEVE DOCUMENTS – ECMA 6
Connect,
create,
update
co(function*() {
// Use connect method to connect to the Server
var db = yield MongoClient.connect(url);
console.log("Connected correctly to server");
// find top 20 countries by size
var results = yield db.collection('countries').find({}
,{"sort": [["area",-1]]}).limit(20).toArray();
console.log("Country One " +JSON.stringify(results[0]));
console.log("Name of Country Four " +results[3].name
+ " and size: " +results[3].area);
// use cursor to get the country count
var cursor = db.collection('countries').find({"continent":"Asia"}
, {"sort": "name"});
var count = yield cursor.count();
console.log("Country count in Asia: "+count);
while (yield cursor.hasNext()){
var country = yield cursor.next();
console.log(country.name);
}
44. RETRIEVE DOCUMENTS (2) – ECMA 6
Connect,
create,
update
// the largest country per continent
var aggquery = [ {$sort: {area : -1}}
, {$group:{ _id: '$continent'
, largestCountry : {$first: "$name"}
}
}
];
var aggcursor = db.collection('countries').aggregate(aggquery);
while (yield aggcursor.hasNext()){
var result = yield aggcursor.next();
console.log(JSON.stringify(result));
}
// Close the connection
db.close();
}).catch(function(err) {
console.log(err.stack);
});//co
45. CREATE DOCS IN MONGODB – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {
"continent" : "Europe",
"children" : [ {"name":"Belgium"}, {"name":"Luxemburg"}],
"someVariable" : 19123, "andmore" : "2716NK" };
var doc2 = {
"continent" : "Asia", "name" : "Johnny",
"nrs" : [ {"name":"China"}, {"name":"India"}, {"name":"Buthan"}],
"tree": {"branch": {"twig":{"leaf": 1}}}
};
var nameOfCollection = "myDocs"
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount
+" documents created into collection "+nameOfCollection);
...
46. CREATE & UPDATE DOCS – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“;
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount +" documents created);
var cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
result = yield db.collection(nameOfCollection).updateOne(
{"tree.branch.twig.leaf":1}, {$set: {name: "Hank", city:"Nieuwegein“
, "location.province":"Utrecht“
, "location.country":"The Netherlands“
, "tree.stem":5}});
console.log(">> Updated "+result.modifiedCount+" document(s)");
cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
47. CREATE, UPDATE AND DELETE – ES6
Connect,
create,
update
// define two documents, any structure you like
var doc = {...}, doc2 = {...} , nameOfCollection = "myDocs“;
var result = yield db.collection(nameOfCollection).insertMany([doc,doc2]);
console.log(">> "+result.insertedCount +" documents created);
var cursor = db.collection(nameOfCollection).find();
while (yield cursor.hasNext()){
var doc = yield cursor.next();
console.log("Document: " +JSON.stringify(doc));
}// while cursor
result = yield db.collection(nameOfCollection).updateOne(...);
...
result = yield db.collection(nameOfCollection).deleteMany({});
console.log(">> Deleted "+ result.deletedCount+" documents ");
// execute command to drop the collection
yield db.command({drop:nameOfCollection});
console.log(">> Dropped collection "+nameOfCollection);
// Close the connection
db.close();
console.log("Connection to database is closed.");
48. INTERACTING WITH MONGODB
FROM JAVA
• MongoDB Java Driver 3.x
• Supports Synchronous and Asynchronous operations
• https://docs.mongodb.com/ecosystem/drivers/java/
• Using MongoDB Driver in a Java Application
• Add dependency in Maven pom.xml file
• Library JAR file mongo-java-driver-3.4.0.jar
Connect,
create,
update
49. CONNECTING TO SERVER
package nl.amis.mongodb.countries;
import java.net.UnknownHostException;
import com.mongodb.MongoClient;
import com.mongodb.client.MongoDatabase;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
String mongodbHost = "127.0.0.1";
Integer mongodbPort = 27017;
String mongodbDatabase = "world";
MongoClient mongoClient = new MongoClient(mongodbHost, mongodbPort);
try {
System.out.println("Connected to server; now hook into database " + mongodbDatabase);
MongoDatabase db = mongoClient.getDatabase(mongodbDatabase);
System.out.println("Connected to database " + mongodbDatabase + " successfully");
} finally {
mongoClient.close();
System.out.println("Closed mongodb connection");
}
}
}
Connect,
create,
update
50. RETRIEVE DATA
...
import org.bson.Document;
import com.mongodb.client.MongoCollection;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
... Connect to world database
// get all documents from the countries collection
MongoCollection<Document> countries = db.getCollection("countries");
System.out.println("Got collection countries successfully");
System.out.println("Number of countries: " + countries.count());
System.out.println("All countries - name and continent");
for (Document doc : countries.find()) {
// System.out.println(doc.toJson());
System.out.println(doc.get("name") + " (" + doc.get("continent") + ")");
}//for
Connect,
create,
update
51. RETRIEVE DATA (2)
import com.mongodb.client.model.Sorts;
import com.mongodb.client.model.Filters;
import com.mongodb.Block;
public class AppConnect {
public static void main(String[] args) throws UnknownHostException {
... Connect to world database
System.out.println( "Retrieve all countries in Asia order alphabetically
by name and display full country document");
countries.find(Filters.eq("continent", "Asia"))
.sort(Sorts.ascending("name"))
.forEach(new Block<Document>() {
@Override
public void apply(final Document document) {
System.out.println(document.toJson());
}
});
Connect,
create,
update
52. RETRIEVE DATA - AGGREGATION
...
Block<Document> printBlock = new Block<Document>() {
@Override
public void apply(final Document document) {
System.out.println("Country: " + document.toJson());
}
};
...
System.out.println("List largest country in each continent");
countries.aggregate(
Arrays.asList(
Aggregates.sort(Sorts.descending("area")),
Aggregates.group("$continent", Accumulators.first("Largest Country", "$name"))
))
.forEach(printBlock);
Connect,
create,
update
53. RETRIEVE DATA – AGGREGATION (2)
System.out.println("List name in uppercase,sorted by size, delta between birthdate and deathrate, "
+" population density for countries in Oceania");
BsonArray subArgs=new BsonArray();
subArgs.add(new BsonString("$birthrate"));
subArgs.add(new BsonString("$deathrate"));
BsonArray divArgs=new BsonArray();
divArgs.add(new BsonString("$population"));
divArgs.add(new BsonString("$area"));
countries.aggregate
(Arrays.asList(
Aggregates.match(Filters.eq("continent", "Oceania")),
Aggregates.sort(Sorts.descending("area")),
Aggregates.project( Projections.fields(
Projections.excludeId(),
Projections.include("continent"),
Projections.computed("name",Projections.computed("$toUpper", "$name")),
Projections.computed("populationGrowthRate",Projections.computed("$subtract", subArgs)),
Projections.computed("populationDensity",
Projections.computed("$trunc", Projections.computed("$divide", divArgs)))
))
))
.forEach(printBlock);
Connect,
create,
update
54. JAVA: CREATE DOCUMENTS IN MONGODB
...
String nameOfCollection = "myDocs";
MongoCollection<Document> coll = db.getCollection(nameOfCollection);
Document doc1 = new Document("continent", "Europe")
.append("nrs", Arrays.asList(new Document("name", "Belgium"),
new Document("name", "Luxemburg")))
.append("someVariable", 19123)
.append("andmore", "kl;jdsfakhfsdahjfsdasjbsdahjbsdahgvsahjkZl;po");
Document doc2 = new Document("continent", "Asia").append("name", "Johnny")
.append("nrs", Arrays.asList(new Document("name", "China"),
new Document("name", "India"), new Document("name", "Buthan")))
.append("tree", new Document("branch",
new Document("twig", new Document("leaf", 1))));
List<Document> documents = new ArrayList<Document>();
documents.add(doc1);
documents.add(doc2);
coll.insertMany(documents);
System.out.println(">> Documents created into collection");
coll.find().forEach(printBlock);
Connect,
create,
update
55. JAVA: UPDATE AND DELETE DATA
...
coll.updateOne(
Filters.eq("tree.branch.twig.leaf", 1),
Updates.combine(
Updates.set("name", "Hank"), Updates.set("city", "Nieuwegein"),
Updates.set("location.province", "Utrecht"),
Updates.set("location.country", "The Netherlands"),
Updates.set("tree.stemp", 5),
Updates.currentDate("lastModified")
)
);
System.out.println(">> Updated one document in collection " + nameOfCollection);
coll.find().forEach(printBlock);
DeleteResult dr = coll.deleteMany(Filters.exists("_id"));
System.out.println(">> Deleted " + dr.getDeletedCount() + " documents from collection "
+ nameOfCollection);
db.runCommand(new Document("drop", nameOfCollection));
System.out.println(">> Dropped collection " + nameOfCollection);
Connect,
create,
update
56. INTERACTING WITH MONGODB
FROM JAVA – JSON OO MAPPING
• Libraries for mapping between BSON/JSON
documents and Java Objects
• Morphia - http://mongodb.github.io/morphia/1.3/getting-started/quick-tour/
• Jongo – Query in Java as in the MongoDB shell - http://jongo.org/
Connect,
create,
update
57. HANDS ON WORKSHOP
• Install MongoDB
• Create Database, Create Collections and Documents
• Retrieve Data
• Import Database
• More configuration and administration
• Development: interacting with MongoDB from
• Compass
• Java
• Node.js