This document discusses microservices, which are small focused tasks that can be invoked via events in Fedora to perform tasks like OCR, image manipulation, and file conversion. It describes two versions of microservices - a Python version and a PHP version. It outlines some advantages of microservices like reusability and consistency. It also describes a student project that integrated the old PHP microservices with Taverna workflow software to allow third party services. It provides instructions on installing, configuring, and running microservices including creating workflows in Taverna Workbench and triggering the services.
This document provides an introduction to microservices, including:
- Microservices are small, independently deployable services that work together and are modeled around business domains.
- They allow for independent scaling, technology diversity, and enable resiliency through failure design.
- Implementing microservices requires automation, high cohesion, loose coupling, and stable APIs. Identifying service boundaries and designing for orchestration and data management are also important aspects of microservices design.
- Microservices are not an end goal but a means to solve problems of scale; they must be adopted judiciously based on an organization's needs.
Kubernetes Architecture - beyond a black box - Part 1Hao H. Zhang
This is part 1 of my Kubernetes architecture deep-dive slide series.
I have been working with Kubernetes for more than a year, from v1.3.6 to v1.6.7, and I am a CNCF certified Kubernetes administrator. Before I move on to something else, I would like to summarize and share my knowledges and take-aways about Kubernetes, from a software engineer perspective.
This set of slides is a humble dig into one level below your running application in production, revealing how different components of Kubernetes work together to orchestrate containers and present your applications to the rest of the world.
The slides contains 80+ external links to Kubernetes documentations, blog posts, Github issues, discussions, design proposals, pull requests, papers, source code files I went through when I was working with Kubernetes - which I think are valuable for people to understand how Kubernetes works, Kubernetes design philosophies and why these design came into places.
The introduction covers the following
1. What are Microservices and why should be use this paradigm?
2. 12 factor apps and how Microservices make it easier to create them
3. Characteristics of Microservices
Note: Please download the slides to view animations.
The document discusses microservice architecture, including concepts, benefits, principles, and challenges. Microservices are an architectural style that structures an application as a collection of small, independent services that communicate with each other, often using RESTful API's. The approach aims to overcome limitations of monolithic architectures like scalability and allow for independent deployments. The key principles include organizing services around business domains, automating processes, and designing services to be independently deployable.
1) Kubernetes is an open-source system for managing containerized applications and services across multiple hosts. It was created by Google in 2014 to automate deployment, scaling, and operations of application containers.
2) Kubernetes allows for automatic deployment and scaling of applications. It makes applications portable and lightweight by running them in containers.
3) The document provides an overview of key Kubernetes concepts including pods, replication controllers, and services. Pods are the smallest deployable units that can contain one or more containers which share resources. Replication controllers ensure a specified number of pod replicas are running. Services define a policy to access pods through labels.
GitOps, Driving NGN Operations Teams 211127 #kcdgt 2021William Caban
The adoption of cloud-native principles brings new challenges. Scaling and evolving operations teams and staying up to date requires the adoption of new operational models and paradigms.
This deck presents how modern paradigms map to GitOps principles and the charactersitics that must be supported by any software used for GitOps.
(Draft) Kubernetes - A Comprehensive OverviewBob Killen
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery called pods. Its main components include a master node that manages the cluster and worker nodes that run the applications. It uses labels to organize resources and selectors to group related objects. Common concepts include pods, services for discovery/load balancing, replica controllers for scaling, and namespaces for isolation. It provides mechanisms for configuration, storage, security, and networking out of the box to ensure containers can run reliably and be easily managed at scale.
Kevin Huang: AWS San Francisco Startup Day, 9/7/17
Architecture: When, how, and if to adopt microservices - Microservices are not for everyone! If you're a small shop, a monolith provides a great amount of value and reduces the complexities involved. However as your company grows, this monolith becomes more difficult to maintain. We’ll look at how microservices allow you to easily deploy and debug atomic pieces of infrastructure which allows for increased velocity in reliable, tested, and consistent deploys. We’ll look into key metrics you can use to identify the right time to begin the transition from monolith to microservices.
Kubernetes for Beginners: An Introductory GuideBytemark
Kubernetes is an open-source tool for managing containerized workloads and services. It allows for deploying, maintaining, and scaling applications across clusters of servers. Kubernetes operates at the container level to automate tasks like deployment, availability, and load balancing. It uses a master-slave architecture with a master node controlling multiple worker nodes that host application pods, which are groups of containers that share resources. Kubernetes provides benefits like self-healing, high availability, simplified maintenance, and automatic scaling of containerized applications.
This document discusses the transition from monolithic architecture to microservices architecture. It begins by outlining challenges with monolithic systems like long development cycles and difficulties scaling. It then defines microservices as loosely coupled services that have bounded contexts. The document provides examples of how to evolve a monolith to microservices by starting with existing services and gradually decomposing the monolith. It acknowledges challenges in distributed systems and eventual consistency that come with microservices. Overall, the document presents microservices as enabling faster innovation, increased agility and delighted customers compared to monolithic systems.
The document discusses microservices architecture as an enhancement of service-oriented architecture. It describes microservices as a way to break up monolithic applications into smaller, interconnected services that implement distinct features. Each microservice has its own business logic and adapters, and may expose an API or user interface. The benefits of microservices include supporting agile development, reducing complexity, enabling independent scaling of services, and allowing choice of technologies. Potential drawbacks include increased complexity of distributed systems and coordinating changes across services. The document provides examples of how a monolithic taxi-hailing application could be decomposed into microservices for trip management, passenger management, driver management, and other functions.
Promgen is a Prometheus management tool that allows web-based management of server configurations and alerting rules. It addresses the need for an easier way to manage Prometheus server configurations than manually editing YAML files. Promgen stores configuration data in a MySQL database and generates YAML files from the stored configurations. It aims to provide a simple interface for configuring Prometheus exporters, ports, alerts and other settings across multiple servers and projects.
Some problems with monolithic architecture, the microservice's best practices and their drawbacks. How build microservices? How to prepare your Operations teams for microservices?
A basic introductory slide set on Kubernetes: What does Kubernetes do, what does Kubernetes not do, which terms are used (Containers, Pods, Services, Replica Sets, Deployments, etc...) and how basic interaction with a Kubernetes cluster is done.
This document provides an overview of Kubernetes including:
1) Kubernetes is an open-source platform for automating deployment, scaling, and operations of containerized applications. It provides container-centric infrastructure and allows for quickly deploying and scaling applications.
2) The main components of Kubernetes include Pods (groups of containers), Services (abstract access to pods), ReplicationControllers (maintain pod replicas), and a master node running key components like etcd, API server, scheduler, and controller manager.
3) The document demonstrates getting started with Kubernetes by enabling the master on one node and a worker on another node, then deploying and exposing a sample nginx application across the cluster.
Hands-On Introduction to Kubernetes at LISA17Ryan Jarvinen
This document provides an agenda and instructions for a hands-on introduction to Kubernetes tutorial. The tutorial will cover Kubernetes basics like pods, services, deployments and replica sets. It includes steps for setting up a local Kubernetes environment using Minikube and demonstrates features like rolling updates, rollbacks and self-healing. Attendees will learn how to develop container-based applications locally with Kubernetes and deploy changes to preview them before promoting to production.
DevOps is a movement to change how IT is done by promoting collaboration between development and operations teams. It aims to reduce waste and improve delivery of software by making development and operations processes more efficient through automation, monitoring, and communication. The DevOps philosophy advocates enhancing software design with operational knowledge, building feedback loops from production into development to improve systems, and fostering a culture of shared responsibility. Key DevOps practices include accelerating the flow of changes to production through continuous integration, delivery, and deployment; adding development practices to operations like automated testing; and empowering developers to do production work to break down barriers between teams. DevOps uses tooling throughout the development and operations process to measure and monitor systems and provide feedback.
These are the slides for a talk/workshop delivered to the Cloud Native Wales user group (@CloudNativeWal) on 2019-01-10.
In these slides, we go over some principles of gitops and a hands on session to apply these to manage a microservice.
You can find out more about GitOps online https://www.weave.works/technologies/gitops/
The document discusses various "islands of automation" which are individual automated systems used in manufacturing. These islands include computer-aided design (CAD), computer-aided manufacturing (CAM), computer numerically controlled (CNC) machines, flexible manufacturing systems (FMS), robotics, automated material handling systems, group technology, computer aided process planning, manufacturing resource planning, and computer control systems. When these various islands are integrated and interconnected, they form a computer integrated manufacturing system.
Crossroads of Asynchrony and Graceful DegradationC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1VmbI3t.
Nitesh Kant describes how embracing asynchrony in the Netflix applications, from networking to business processing, creates gracefully degrading and highly resilient applications. Filmed at qconsf.com.
Nitesh Kant is an engineer in Netflix’s Edge Gateway team, working on Netflix’s asynchronous Inter Process Communication stack. He is the author of RxNetty which forms the core of this stack and is currently moving Zuul to this new architecture.
The document discusses consciousness as a limitation. It begins by reviewing concepts covered so far, and introduces the idea that consciousness arises from limitations of our mental capabilities.
It then tells two stories to illustrate its point. The first is about an alien visiting special needs children and mistakenly thinking their condition results from something extra they possess rather than deficits. The second introduces the concept of a "fully aware being".
The document argues that traditional views of the mind are mistaken in assuming consciousness results from something added rather than limitations. It asserts consciousness arises from the distributed and limited nature of information processing in the brain, not from any single structure or region.
Building a useful target architecture - Myth or reality2Regine Deleu
The document discusses the value of developing a target enterprise architecture and how to do so successfully. It emphasizes the importance of having an "enterprise" mindset that is willing to take risks and invest in innovations. Key tools for informing decisions about transformation include a business capability model, investment portfolio, and enterprise architecture with aligned strategies, standards, and perspectives. The architecture should be based on key events and guide the organization flexibly towards its goals.
Art Nouveau was a total art style that emerged in the late 19th century, incorporating architecture, design and fine arts. It took inspiration from natural, organic forms like vines and flowers. Two key figures were Antonio Gaudi, a Spanish architect known for unique structures like Casa Mila and Parque Güell that featured curving shapes, and Charles Rennie MacKintosh, a Scottish designer who pioneered the Art Nouveau interior style using flowing lines and nature-inspired motifs. Art Nouveau emphasized harmony and rejected historical influences in favor of a modern aesthetic focused on the natural world.
Does Current Advertising Cause Future Sales?Trieu Nguyen
findings from a large-scale field experiment that allows us to study whether
there is a causal relationship between current advertising and future sales. The
experimental design overcomes limitations that have affected previous investigations of
this issue. We find that current advertising does affect future sales but the sign of the
effect varies depending on the customers targeted. For the firm’s best customers the
long-run effect of increases in current advertising is actually negative, while for other
customers the effect is positive. We argue that these outcomes reflect two competing
effects: brand-switching and inter-temporal substitution. Furthermore, our data suggest a way to distinguish between the informative and persuasive roles of advertising, providing insight into the mechanism by which advertising differentially affects various customer subsets
This document discusses using RFX (Reactive Function X), a design pattern and collection of open source tools, to solve fast data problems. It presents an example of using RFX for web analytics to count pageviews and unique users and detect DDOS attacks. The RFX approach applies the BEAM methodology for agile data warehousing. It demonstrates RFX concepts like event data actors, agents, collectors, routers, processors, storage and reactors using a pageview analytics demo with source code on GitHub.
Slide 2 collecting, storing and analyzing big dataTrieu Nguyen
This document discusses the process of collecting, storing, processing and analyzing big data. It covers the key concepts and technologies for collecting data using tools like Apache Sqoop and Kafka, storing data using clusters, file systems, NoSQL databases and concepts like sharding and replication. It also discusses processing data using parallel and distributed processing with Hadoop, and analyzing data using Apache Phoenix which provides a SQL interface to query HBase databases.
Application-oriented ping-pong benchmarking: how to assess the real communica...Trieu Nguyen
Moving data between processes has often been discussed as one of the
major bottlenecks in parallel computing—there is a large body of research, striving
to improve communication latency and bandwidth on different networks, measured
with ping-pong benchmarks of different message sizes. In practice, the data to be
communicated generally originates from application data structures and needs to be
serialized before communicating it over serial network channels.
This document introduces the Reactive Data System (RDS) framework called RFX for solving fast data problems reactively. It discusses how RFX was developed to handle common issues like counting pageviews, unique users, and real-time marketing. RFX is an open source, full stack framework that uses various tools like Kafka, Spark, and Redis to process high volumes of event data in real-time for applications like analytics, advertising, and monitoring. The document provides an example architecture and topology for collecting tracking data, processing it through RFX components, and generating reports.
A Day in the Life of a Hadoop AdministratorEdureka!
This document outlines the daily tasks of a Hadoop administrator, which include monitoring the cluster, planning maintenance tasks, executing regular utility tasks like backups and file merging, upgrading systems, assisting developers, and troubleshooting issues. It also provides demonstrations on achieving high availability in Hadoop and YARN clusters, and discusses tools for monitoring cluster resources, user permissions, and common error messages. The document promotes an online Hadoop administration certification course from Edureka that teaches skills for planning, deploying, monitoring, tuning and securing Hadoop clusters.
Where is my next jobs in the age of Big Data and AutomationTrieu Nguyen
The document discusses how automation is impacting knowledge work jobs and proposes that the best approach is augmentation, where humans and machines work together. It provides examples of how different knowledge work jobs like teachers, lawyers, and financial advisors could take steps to augment their work with automation. The key steps include humans mastering automated systems, identifying new areas for automation, focusing on tasks they currently do best, finding niche roles, and building automated systems. The implications are that organizations should adopt an augmentation perspective, select the right technologies, design work for humans and machines, provide transition options for employees, and appoint a leader to manage workplace changes.
Introduction and overview ArangoDB query language AQLArangoDB Database
This document provides an overview of the ArangoDB query language (AQL). It discusses how AQL was created to fit the document-oriented data model of ArangoDB better than SQL. Key features of AQL covered include: using FOR loops to iterate over documents in collections, applying FILTER conditions to restrict results, and using RETURN to select the output fields. Scopes are also explained, with AQL using FILTER in place of SQL's ON and WHERE clauses through consistent scoping.
This document summarizes the key findings from the 2016 O'Reilly Data Science Salary Survey, which collected responses from 983 data professionals. Some of the main findings include: Python and Spark contribute most to salary; those who code more earn higher salaries; SQL, Excel, R and Python are the most commonly used tools; attending more meetings correlates with higher pay; women earn less than men for the same work; and geographic location, as measured by GDP, serves as a proxy for salary variation. The report also clusters respondents based on their tool usage and tasks to identify subgroups.
Big Data Warehousing Meetup: Real-time Trade Data Monitoring with Storm & Cas...Caserta
Caserta Concepts' implementation team presented a solution that performs big data analytics on active trade data in real-time. They presented the core components – Storm for the real-time ingest, Cassandra, a NoSQL database, and others. For more information on future events, please check out http://www.casertaconcepts.com/.
Slide 3 Fast Data processing with kafka, rfx and redisTrieu Nguyen
1. The document discusses using the RFX (Reactive Function X) framework to solve problems with fast data processing.
2. RFX is a design pattern and collection of open source tools that can be used to quickly build data products and implement an agile data pipeline.
3. Examples of how RFX could be used for web analytics are presented, including counting pageviews and unique users in near real-time and detecting DDOS attacks.
Eduardo Silva is an open source engineer at Treasure Data working on projects like Fluentd and Fluent Bit. He created the Monkey HTTP server, which is optimized for embedded Linux and has a modular plugin architecture. He also created Duda I/O, a scalable web services stack built on top of Monkey using a friendly C API. Both projects aim to provide lightweight, high performance solutions for collecting and processing data from IoT and embedded devices.
Bighead is Airbnb's machine learning infrastructure that was created to:
1) Standardize and simplify the ML development workflow;
2) Reduce the time and effort to build ML models from weeks/months to days/weeks; and
3) Enable more teams at Airbnb to utilize ML.
It provides services for data management, model training/scoring, production deployment, and model management to make the ML process more efficient and consistent across teams. Bighead is built on open source technologies like Spark, TensorFlow, and Kubernetes but addresses gaps to fully support the end-to-end ML pipeline.
Bighead: Airbnb’s End-to-End Machine Learning Platform with Krishna Puttaswa...Databricks
Bighead is Airbnb's machine learning infrastructure that was created to:
- Standardize and simplify the ML development workflow;
- Reduce the time and effort to build ML models from weeks/months to days/weeks; and
- Enable more teams at Airbnb to utilize ML.
It provides shared services and tools for data management, model training/inference, and model management to make the ML process more efficient and production-ready. This includes services like Zipline for feature storage, Redspot for notebook environments, Deep Thought for online inference, and the Bighead UI for model monitoring.
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Ambassador Labs
1. The presentation introduces Docker, Kubernetes, and Envoy as foundational tools for building microservices. Docker allows packaging applications into portable containers, Kubernetes provides a platform to manage containers across clusters of hosts, and Envoy handles traffic routing and resilience at the application layer.
2. The presenters demonstrate how to build a simple Python web application into a Docker container image. They then deploy the containerized application to a Kubernetes cluster using Kubernetes objects like deployments and services. This allows the application to scale across multiple pods and be accessed via a stable service endpoint.
3. Finally, the presenters note that as applications become distributed across microservices, failures at the application layer (L7) become more common and
Enter the world of cloud computing and software development with PaaS. What it takes to create a production ready application with Heroku and how to run it?
Do you know what your Drupal is doing_ Observe it!sparkfabrik
Our Drupal 8 websites are true applications, often very complex ones.
More and more workload is delegated to external systems, usually microservices, that are used for many different tasks.
Architectures are always more distributed and fragmented.
To trace the lifecycle of a single request that origins in a client, passes throught all Drupal subsytems, reaches external (micro)services and comes back will become mandatory to track down problems and to optimize for performances. This is often time consuming and without the right tools may became very difficult.
A simple unstructured log stream isn't enough anymore, we need to find a way to observe the details of what is going on.
Observability is all about this and is based on structured logs, metrics and traces. In this talk we will see how to implement these tecniques in Drupal, which tools and which modules to use to trace and log all requests that reach our website and how to expose and display useful metrics.
We will integrate Drupal with OpenTracing, Prometheus, Monolog, Grafana and many more.
The document provides information about an upcoming Montreal MuleSoft Meetup event that will introduce attendees to CloudHub 2.0. The agenda includes introductions, a presentation on CloudHub 2.0, a demo, and a Q&A session. Attendees are asked to provide feedback. The speaker will discuss what CloudHub 2.0 is, its architecture, features like replicas and security, differences from CloudHub 1.0, and limitations. A live demo is also planned.
This document provides an overview and summary of OpenShift v3 and containers. It discusses how OpenShift v3 uses Docker containers and Kubernetes for orchestration instead of the previous "Gears" system. It also summarizes the key architectural changes in OpenShift v3, including using immutable Docker images, separating development and operations, and abstracting operational complexity.
msnos: a cool and cozy blanket for your microservices - Bruno Bossola - Codem...Codemotion
Codemotion Rome 2015 - Since two years in Workshare we moved to a microservices based architecture and it's proved to be challenging in several different ways. Traditional configuration based mechanisms failed because of the very dynamic nature of such architecture. At any point in time you should be able to deploy a new microservice, kill one, upgrade one, this while preserving things like load balancing and session affinity, and being sure at the same time that everything is healthy. Workshare built an open source library, msnos, that tries to address the problem on the three main platform used (ruby, java, .net)
This talk describes the current state of the Veil-Framework and the different tools included in it such as Veil-Evasion, Veil-Catapult, Veil-Powerview, Veil-Pillage, Veil-Ordnance
Heroku is a platform as a service (PaaS) that allows developers to build, run, and operate applications entirely in the cloud. With Heroku, developers can spend their time on their application code instead of managing servers, deployment, ongoing operations, or scaling. The document provides an overview of cloud computing and Heroku's features which include easy deployment with Git, automatic scaling, comprehensive monitoring tools, and an intuitive dashboard. It also describes how to set up a Heroku account, understand key concepts like the Procfile and package.json, and deploy a sample Node.js application to Heroku with Git.
Improve your infrastructure security with FusionDirectory, Benoit Mortier, OW...OW2
This document discusses how OW2 improved its infrastructure security by implementing FusionDirectory. Previously, user data was spread across different databases and directories. FusionDirectory was chosen in 2016 to centralize user management in a single LDAP directory. It required work to model the data and migrate existing user data. Applications were then connected to the LDAP directory for authentication. FusionDirectory provides tools for user, group, and role management that improved security and administration for OW2.
This short document discusses the Puppet configuration management tool. It begins with a quote about controlling the spice and universe from Dune and includes the author's name and date. The document then lists various Puppet features like scalability, flexibility, simple installation, and modules. It provides examples of large sites using Puppet like Google and describes Puppet's client-server and masterless models. In closing, it emphasizes Puppet's flexibility, community support, and over 1200 pre-built configurations.
This workshop was given at Crikeycon 2019 in Brisbane. It introduces Velociraptor and explains some of the design goals and implementation.
Note - this slide deck is outdated but might still be useful. The tool has evolved significantly since Crikeycon.
This document provides an overview of machine learning in cyber security. It discusses definitions of machine learning, cyber security, and how machine learning can be used for cyber security tasks like malware detection. It also covers theoretical concepts, hands-on materials like necessary software and lab setup, and guidance for projects. Specific machine learning and security tools are mentioned, like Docker for containerization. The document aims to explain the importance and applications of machine learning in cyber security.
Canonical established Ubuntu in 2004 and released the long-term support version 12.04 "Precise Pangolin" in April 2012. Ubuntu is positioned as the leading Linux operating system for servers, clouds, and service orchestration. Canonical's Juju service orchestration tool allows easy deployment and management of services on Ubuntu through reusable "charms" that encapsulate best practices.
1) Docker Swarm provides a simple way to orchestrate containers across multiple nodes using built-in tools for scheduling, load balancing, and configuration through environment variables.
2) Kubernetes and Mesosphere are more fully featured orchestration platforms that can schedule both containers and tasks, but have additional dependencies and more complex configurations.
3) When moving to microservices, the document recommends containerizing the application components rather than the infrastructure, starting small, having contingency plans for failure, and managing orchestration without hosted solutions.
Building Your Customer Data Platform with LEO CDP in Travel Industry.pdfTrieu Nguyen
1. The document outlines the Chief Platform Engineer's background and introduces LEO CDP, a customer data platform for the travel industry.
2. It discusses 5 challenges companies face related to customer growth, journeys, data platforms, communication and understanding customers with big data.
3. A case study shows how LEO CDP can be used to create a customer journey map for a travel agency, including personalized promotions and offers sent via email.
How to track and improve Customer Experience with LEO CDPTrieu Nguyen
This document discusses how to track and improve customer experience using LEO CDP. It begins by explaining why measuring customer experience is important, then introduces four key metrics: Customer Feedback Score, Customer Effort Score, Customer Satisfaction Score, and Net Promoter Score. It describes using journey maps to manage customer experience data and visualize the customer journey. Finally, it presents LEO CDP as a software solution for collecting customer experience data, building surveys, and generating reports to gain insights to improve products, services, and the overall customer experience.
[Notes] Customer 360 Analytics with LEO CDPTrieu Nguyen
Part 1: Why should every business need to deploy a CDP ?
1. Big data is the reality of business today
2. What are technologies to manage customer data ?
3. The rise of first-party data and new technologies for Digital Marketing
4. How to apply USPA mindset to build your CDP for data-driven business
Part 2: How to use LEO CDP for your business
1. Core functions of LEO CDP for marketers and IT managers
2. Data Unification for Customer 360 Analytics
3. Data Segmentation
4. Customer Personalization
5. Customer Data Activation
Part 3: Case study in O2O Retail and Ecommerce
1. How to build customer journey map for ecommerce and retail
2. How to do customer analytics to find ideal customer profiles
The ideal customer profile in a B2B context
The ideal customer profile in a B2C context
3. Manage product catalog for customer personalization
4. Monitoring Data of Customer Experience (CX Analytics)
CX Data Flow
CX Rating plugin is embedded in the website, to collect feedback data
An overview of CX Report
A CX Report in a customer profile
5. Monitoring data with real-time event tracking reports
Event Data Flow
Summary Event Data Report
Event Data Report in a Customer Profile
Part 4: How to setup an instance of LEO CDP for free
1. Technical architecture
2. Server infrastructure
3. Setup middlewares: Nginx, ArangoDB, Redis, Java and Python
Network requirements
Software requirements for new server
ArangoDB
Nginx Proxy
SSL for Nginx Server
Java 8 JVM
Redis
Install Notes for Linux Server
Clone binary code for new server
Set DNS hosts for LEO CDP workers
4. Setup data for testing and system verification
Part 5: Summary all key ideas
Why should you invest in LEO CDP ?
Purpose: Big data and AI democracy for SMEs companies
Problem: Customer Analytics and Customer Personalization
Solutions: CDP + CX + Personalization Engine
Product demo: LEO CDP for Ecommerce and Fintech
Business model: Freemium → Ecosystem → Subscription
Market size: 20 billion USD in 2026 and CAGR 34.6%
Differentiation: cloud-native software
Go-to-market approach: Community → Free → Paid
Team: 1 full-stack dev, 1 data scientist and 12,000 fans of BigDataVietnam.org Community
Need 150,000 USD for scaling business (you get 20% share)
The document outlines new features and updates for 2022 from USPA Technology Company, including a new dedicated dashboard for CMOs, updated UI for Customer 360 Insights, and a focus on data-driven business processes and digital marketing in B2B through standardizing data-driven processes and focusing on customer insights.
Lộ trình triển khai LEO CDP cho ngành bất động sảnTrieu Nguyen
1) Hiểu bài toán số hoá trải nghiệm khách hàng
2) Nghiên cứu giải pháp LEO CDP
3) Lộ trình triển khai
Phát triển / số hoá điểm chạm khách hàng
Xây dựng bản đồ hành trình khách hàng
Định nghĩa các metrics và KPI quan trọng
Xây dựng web portal và mobile data hub
Xây dựng kế hoạch Digital Marketing
Triển khai CDP và Marketing Automation
Xây dựng đội Analytics để phân tích dữ liệu
From Dataism to Customer Data PlatformTrieu Nguyen
1) How to think in the age of Dataism with LEO CDP ?
2) Why is Dataism for human, business and society ?
3) How should LEO Customer Data Platform (LEO CDP) work ?
4) How to use LEO CDP for your business ?
Data collection, processing & organization with USPA frameworkTrieu Nguyen
1) How to think in the age of Dataism with USPA framework ?
2) How to collect customer data
3) Data Segmentation Processing for flexibility and scalability
4) Data Organization for personalization and business activation
Part 1: Introduction to digital marketing technologyTrieu Nguyen
This document provides an overview of a mini-course on data-driven marketing using the USPA framework presented by Trieu Nguyen. It includes biographical information about Trieu Nguyen's background and experience in big data projects, machine learning, and digital marketing roles. The document also outlines the topics that will be covered in the mini-course, including digital media models, search engine marketing, social media marketing, advertising technology, customer data platforms, and case studies. Key terms like omnichannel strategy, customer experience strategy, and artificial intelligence strategies for marketing are also defined.
Transform your marketing and sales capabilities with Big Data and A.I
1) Why is Customer Data Platform (CDP) ?
Case study: Enhancing the revenue of your restaurant with CDP and mobile app marketing
Question: Why can CDP disrupt business model for restaurant industry (B2C) ?
2) How would CDP work in practice ?
Introducing USPA.tech as logical framework for implementing CDP in practice
How Can a Customer Data Platform Enhance Your Account-Based Marketing Strategy (B2B) ?
3) How can we implement CDP for business?
Introducing the CDP as customer-first marketing platform for all industries (my key idea in this slide)
How to build a Personalized News Recommendation PlatformTrieu Nguyen
This document discusses how to build a personalized news recommendation platform. It explains that recommendation systems are needed to retain users, increase traffic, and improve the content experience. It describes popular techniques like collaborative filtering, content-based filtering, and hybrid systems. Specifically, it outlines a case study using a USPA framework with real social news data. Key factors for a news recommendation system are discussed like novelty, user history, and location. The document also provides a simple example of building a recommendation engine with Apache Spark.
How to grow your business in the age of digital marketing 4.0Trieu Nguyen
1. The document discusses how businesses can grow in the digital marketing age using technologies like cloud services, big data, AI, and headless CMS platforms.
2. It introduces LeoCloudCMS as a headless API CMS that is built for digital marketing 4.0 and can run scalably on cloud computing.
3. The key idea is to think of your entire business as a "box" and use LeoCloudCMS to attract internet users into the box and offer valuable services.
Video Ecosystem and some ideas about video big dataTrieu Nguyen
Introduction to Video Ecosystem Mind Map
Video Streaming Platform
Video Ad Tech Platform
Video Player Platform
Video Content Distribution Platform
Video Analytics Platform
Summary of key ideas
Q & A
Concepts, use cases and principles to build big data systems (1)Trieu Nguyen
1) Introduction to the key Big Data concepts
1.1 The Origins of Big Data
1.2 What is Big Data ?
1.3 Why is Big Data So Important ?
1.4 How Is Big Data Used In Practice ?
2) Introduction to the key principles of Big Data Systems
2.1 How to design Data Pipeline in 6 steps
2.2 Using Lambda Architecture for big data processing
3) Practical case study : Chat bot with Video Recommendation Engine
4) FAQ for student
This document discusses open over-the-top (OTT) video content platforms. It defines OTT as streaming media distributed directly over the internet bypassing traditional distribution methods. The document then covers OTT market drivers and business models. It examines the most popular OTT platform in Vietnam and challenges for successful OTT platforms including scalability, content acquisition and management, audience engagement, and business models. Finally, it proposes a modular technical architecture for an open OTT video platform using open source technologies.
Apache Hadoop and Spark: Introduction and Use Cases for Data AnalysisTrieu Nguyen
This document provides an introduction to Apache Hadoop and Spark for data analysis. It discusses the growth of big data from sources like the internet, science, and IoT. Hadoop is introduced as providing scalability on commodity hardware to handle large, diverse data types with fault tolerance. Key Hadoop components are HDFS for storage, MapReduce for processing, and HBase for non-relational databases. Spark is presented as improving on MapReduce by using in-memory computing for iterative jobs like machine learning. Real-world use cases of Spark at companies like Uber, Pinterest, and Netflix are briefly described.
Introduction to Recommendation Systems (Vietnam Web Submit)Trieu Nguyen
1) Why do we need recommendation systems ?
2) How can we think with recommendation systems ?
3) How can we implement a recommendation system with open source technologies ?
RFX framework https://github.com/rfxlab
Apache Kafka: https://kafka.apache.org
Apache Spark: https://spark.apache.org
How to Secure Your Kubernetes Software Supply Chain at ScaleAnchore
Achieving comprehensive security visibility in Kubernetes environments is essential for maintaining robust and compliant cloud-native applications. In this exclusive webinar, Anchore and Spectro Cloud team up to showcase how to enhance your Kubernetes security posture with SBOM (Software Bill of Materials) management and vulnerability scanning.
Join Cornelia Davis, VP of Product, Spectro Cloud and Alan Pope, Director of Developer Relations, Anchore to learn how to elevate your Kubernetes security visibility and protect your cloud-native applications effectively.
—Discover how Anchore can be integrated with Spectro Cloud Palette to take SBOM scanning to the next level, delivering fully automated software compliance
—Gain valuable insights into best practices for securing your Kubernetes workloads, ensuring compliance, and improving your DevSecOps processes.
Predicting Test Results without Execution (FSE 2024)Andre Hora
As software systems grow, test suites may become complex, making it challenging to run the tests frequently and locally. Recently, Large Language Models (LLMs) have been adopted in multiple software engineering tasks. It has demonstrated great results in code generation, however, it is not yet clear whether these models understand code execution. Particularly, it is unclear whether LLMs can be used to predict test results, and, potentially, overcome the issues of running real-world tests. To shed some light on this problem, in this paper, we explore the capability of LLMs to predict test results without execution. We evaluate the performance of the state-of-the-art GPT-4 in predicting the execution of 200 test cases of the Python Standard Library. Among these 200 test cases, 100 are passing and 100 are failing ones. Overall, we find that GPT-4 has a precision of 88.8%, recall of 71%, and accuracy of 81% in the test result prediction. However, the results vary depending on the test complexity: GPT-4 presented better precision and recall when predicting simpler tests (93.2% and 82%) than complex ones (83.3% and 60%). We also find differences among the analyzed test suites, with the precision ranging from 77.8% to 94.7% and recall between 60% and 90%. Our findings suggest that GPT-4 still needs significant progress in predicting test results.
iBirds Services - Comprehensive Salesforce CRM and Software Development Solut...vijayatibirds
Unlock the full potential of your business with iBirds Services. As a trusted Salesforce Consulting Partner, iBirds Software Pvt. Ltd. offers a wide range of customer-centric consulting services to help you seamlessly integrate, customize, and optimize your Salesforce CRM. Our team of experts specializes in delivering innovative software development solutions tailored to meet your unique business needs.
In this document, you will discover:
An overview of iBirds Services and our expertise in Salesforce CRM implementation.
Detailed insights into our software development services, including custom applications, integrations, and automation.
Case studies highlighting our successful projects and satisfied clients.
Key benefits of partnering with iBirds Services for your CRM and software development needs.
Whether you are a small business or a large enterprise, our proven strategies and cutting-edge technologies ensure your business stays ahead of the competition. Explore our services and learn how iBirds can transform your business operations with scalable and efficient solutions.
What is Micro Frontends and Why Use it.pdflead93317
🚀 Let's Deep Dive into 𝐖𝐡𝐲 𝐌𝐢𝐜𝐫𝐨 𝐅𝐫𝐨𝐧𝐭𝐞𝐧𝐝𝐬 𝐢𝐬 𝐭𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞 𝐨𝐟 𝐅𝐫𝐨𝐧𝐭𝐞𝐧𝐝 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 🚀
In today's fast-paced tech landscape, agility, scalability, and maintainability are more crucial than ever. Traditional monolithic frontend architectures often struggle to keep up with these demands. Enter Micro Frontends: a revolutionary approach that's transforming the way we build web applications.
Three available editions of Windows Servers crucial to your organization’s op...Q-Advise
Three available editions of Windows Servers crucial to your organization’s operations
Windows Server, Microsoft’s robust operating system, is the cornerstone of enterprise IT infrastructure, tailored for mission-critical operations. It helps in managing enterprise-level tasks, including data storage, applications, and communication.
Proper licensing of Windows Server is essential for both legal compliance and optimal functionality within business environments.
Windows Server comes in various edition and before any edition is used in your organization, it is required you license them appropriately. The licensing can be complex and capital demanding when you don’t know what you want or understand the licensing requirements.
Even if successfully licensed, there are various activities you can practice as an organization to make sure your Server is operating optimally and there is real value for money. This requires a deeper understanding of best practices and our team of cloud and licensing experts can be of support.
Send the team an email, info@q-advise.com let’s have a look at your needs, together with you decide which licensing model will best work in your case, assist you with savings options and share with you how pre-owned licensing can help you get licensed adequately also.
Literals - A Machine Independent Feature21h16charis
Introduction to Literals, A machine independent feature. The presentation is based on the prescribed textbook for System Software and Compiler Design, Computer Science and Engineering - System Software by Leland. L. Beck,
D Manjula.
PathSpotter: Exploring Tested Paths to Discover Missing Tests (FSE 2024)Andre Hora
When creating test cases, ideally, developers should test both the expected and unexpected behaviors of the program to catch more bugs and avoid regressions. However, the literature has provided evidence that developers are more likely to test expected behaviors than unexpected ones. In this paper, we propose PathSpotter, a tool to automatically identify tested paths and support the detection of missing tests. Based on PathSpotter, we provide an approach to guide us in detecting missing tests. To evaluate it, we submitted pull requests with test improvements to open-source projects. As a result, 6 out of 8 pull requests were accepted and merged in relevant systems, including CPython, Pylint, and Jupyter Client. These pull requests created/updated 32 tests and added 80 novel assertions covering untested cases. This indicates that our test improvement solution is well received by open-source projects.
BitLocker Data Recovery | BLR Tools Data Recovery SolutionsAlina Tait
BLR Tools provides an advanced BitLocker Data Recovery Tool specifically engineered to recover lost or inaccessible data from BitLocker-encrypted drives. Whether you're dealing with accidental deletion, encryption key problems, or system crashes, our cutting-edge software guarantees a secure and efficient recovery process. Rely on BLR Tools for dependable BitLocker data recovery and effortlessly restore access to your essential files.
The code is written and the tests pass. I just have to commit this last round of changes to my branch. Wait, why does that say committed to main? Did I commit all those changes to main? Arghh! I can’t redo all of this!
Committing changes to the wrong branch, forgetting files, misspelling the commit message, and needing to undo commits are some of the “advanced” features of Git that we normal people run into way too often and need help with. The fixes are often easy – once you know what they are. But in the heat of the moment, with the deadline (or Friday afternoon) approaching, it isn’t always easy to figure out what magic spell to cast to get Git to do what you need.
We’ll spend some time looking at typical Git situations people get themselves into, and then we’ll demonstrate how to get out of them. This isn’t about Git internals or a Git master’s class – this real-world Git when things aren’t going right. And there will be plenty of time for questions, so bring your “best” Git nightmare scenarios so we can figure out how to recover.
Unlocking value with event-driven architecture by Confluentconfluent
Sfrutta il potere dello streaming di dati in tempo reale e dei microservizi basati su eventi per il futuro di Sky con Confluent e Kafka®.
In questo tech talk esploreremo le potenzialità di Confluent e Apache Kafka® per rivoluzionare l'architettura aziendale e sbloccare nuove opportunità di business. Ne approfondiremo i concetti chiave, guidandoti nella creazione di applicazioni scalabili, resilienti e fruibili in tempo reale per lo streaming di dati.
Scoprirai come costruire microservizi basati su eventi con Confluent, sfruttando i vantaggi di un'architettura moderna e reattiva.
Il talk presenterà inoltre casi d'uso reali di Confluent e Kafka®, dimostrando come queste tecnologie possano ottimizzare i processi aziendali e generare valore concreto.
Alluxio Webinar | What’s new in Alluxio Enterprise AI 3.2: Leverage GPU Anywh...Alluxio, Inc.
Alluxio Webinar
July.23, 2024
For more Alluxio Events: https://www.alluxio.io/events/
Speaker:
- Shouwei Chen (core maintainer and product manager, Alluxio)
In today's AI-driven world, organizations face unprecedented demands for powerful AI infrastructure to fuel their model training and serving workloads. Performance bottlenecks, cost inefficiencies, and management complexities pose significant challenges for AI platform teams supporting large-scale model training and serving. On July 9, 2024, we introduced Alluxio Enterprise AI 3.2, a groundbreaking solution designed to address these critical issues in the ever-evolving AI landscape.
In this webinar, Shouwei Chen will introduce exciting new features of Alluxio Enterprise AI 3.2:
- Leveraging GPU resources anywhere accessing remote data with the same local performance
- Enhanced I/O performance with 97%+ GPU utilization for popular language model training benchmarks
- Achieving the same performance as HPC storage on existing data lake without additional HPC storage infrastructure
- New Python FileSystem API to seamlessly integrate with Python applications like Ray
- Other new features, include advanced cache management, rolling upgrades, and CSI failover
2. What do we mean when we say
microservices?
● Small focused tasks (in our case they are
often used for creating derivatives)
● Our Microservices are invoked via events
in Fedora (via JMS/Stomp messages)
Microservices
3. Currently we have two versions.
● Python version - the original version.
○ https://github.
com/Islandora/islandora_microservices
● PHP version - developed so we could leverage
existing Islandora php code including tuque
○ https://github.com/roblib/php_listeners
○ This version has been updated to use
Taverna.
Microservices
4. ● OCR
● Image manipulation - thumbnails, image
conversion (tiff to jp2)
● File conversion - convert Word or Open
Office files to PDF, convert PDF to PDFa
● Technical metadata, crosswalks
● Handles
● Some security policy management
(publishing an object)
● Other?
Microservices common usage
5. ● Services and their dependencies can be
installed, configured and maintained
once.
● UPEI has a number of Drupal servers and instead
of installing Tesseract, Imagemagick etc. on each
one we can install and maintain these packages
in one place but use them with various Fedora or
Drupal installs
● Consistency - Microservices will be
invoked if an object is modified by an
external script, Islandora, Hydra or the
Fedora admin client.
Microservices Advantages
6. ● With Taverna we can also leverage third
party external services.
Advantages Continued
8. CS482 UPEI student project to integrate the
old PHP microservices with Taverna.
Goals of this project:
● Create and manage workflow using
Taverna Workbench
● Store Taverna T2flow in Fedora content
models
Current Microservices
implementation
9. ● When our listener receive a message we
will get the corresponding T2flow
document from the objects content
model and pass that on to Taverna
Server.
● Taverna server then dispatches the
workflow to the correct locations
● Taverna workbench for
creating/designing workflow
Current Microservices cont’d
13. To create a workflow you need Taverna
Workbench installed.
● Click import new services and choose
WSDL Service and enter a url to the
services soap endpoint.
● Drag and drop functions to the canvas
● Configure the functions
○ don’t forget looping and security.
● Documentation
● Demo workbench
Create a workflow
15. There are two config.xml files.
One file contains configuration information
for the services exposed via SOAP.
● /var/www/html/php_listeners
The other is the configuration information
for the listener.
● /opt/php_listeners
Configure microservices
17. ● on centos
○ cd /opt/php_listeners
○ php listener.php
○ tail -f listener.log
● add a tiff a page to a book to trigger
services.
Demo microservices