Advanced Scala reflection & runtime meta-programming. The Scala compiler toolbox. Reading Scala Annotations and overcoming type erasure with some real world use cases.
The Scala programming language has been gaining momentum recently as an alternative (and some might say successor) to Java on the JVM. This talk will start with an introduction to basic Scala syntax and concepts, then delve into some of Scala's more interesting and unique features. At the end we'll show a brief example of how Scala is used by the Lift web framework to simplify dynamic web apps.
Scala is an object-oriented and functional programming language that runs on the Java Virtual Machine. It was designed by Martin Odersky and developed at EPFL in Switzerland. Scala combines object-oriented and functional programming principles, including support for immutable data structures, pattern matching, and closures. It interoperates seamlessly with existing Java code and libraries.
This document provides an overview of the Scala programming language. Some key points:
- Scala runs on the Java Virtual Machine and was created by Martin Odersky at EPFL.
- It has been around since 2003 and the current stable release is 2.7.7. Release 2.8 beta 1 is due out soon.
- Scala combines object-oriented and functional programming. It has features like pattern matching, actors, XML literals, and more that differ from Java. Everything in Scala is an object.
Scala Intro training @ Lohika, Odessa, UA.
This is a basic Scala Programming Language overview intended to evangelize the language among any-language programmers.
I used these slides for a Scala workshop that I gave. They are based on these: http://www.scala-lang.org/node/4454. Thanks to Alf Kristian Støyle and Fredrik Vraalsen for sharing!
This document provides an overview of Scala fundamentals including:
- Scala is a programming language for the JVM that supports both object-oriented and functional paradigms.
- It defines variables, values, lazy values, functions, types, classes, objects, traits, and higher-order functions.
- Classes can extend other classes and traits, allowing for multiple inheritance. Objects are used as singletons.
- Functional concepts like immutability, anonymous functions, and higher-order functions are supported.
Scala is a multi-paradigm programming language that runs on the Java Virtual Machine. It is intended to be both object-oriented and functional. Scala is compatible with Java and allows Java code to interoperate with Scala code. Some key features of Scala include type inference, lazy evaluation, treating methods as variables, and support for both object-oriented and functional programming paradigms.
Functional Objects & Function and ClosuresSandip Kumar
Scala functions are objects that implement traits like Function1. Functions are treated as objects with an apply method. When a function is defined as a method in a class, it is treated differently than a standalone function. Functions can take variable arguments using a * notation and have default parameter values specified.
This document introduces developing a Scala DSL for Apache Camel. It discusses using Scala techniques like implicit conversion, passing functions as parameters, by-name parameters and currying to build a DSL. It provides examples of building simple routes in the Scala DSL using these techniques and compares them to the Java DSL. It also covers some caveats like interactions between Java and Scala generics and operator precedence. The goal is to introduce basic Scala concepts and syntax that can be used to build a Scala DSL, using Apache Camel as an example.
Scala is a multi-paradigm language that runs on the JVM and interoperates with Java code and libraries. It combines object-oriented and functional programming by allowing functions to be treated as objects and supports features like traits, pattern matching, and immutable data structures. The Scala compiler infers types and generates boilerplate code like getters/setters, making development more productive compared to Java. While Scala has a learning curve, it allows a more concise and scalable language for building applications.
Starting with Scala : Frontier Developer's Meetup December 2010Derek Chen-Becker
This document provides an overview of the Scala programming language presented at a meetup event. It discusses Scala's history and pedigree, being created by Martin Odersky. It outlines some key aspects of Scala like being object-oriented, functional, and working on the JVM. The talk covers Scala fundamentals like immutable values, mutable variables, functions, and objects. It also discusses traits, classes, case classes and more Scala concepts.
The document discusses several core Java concepts including:
1) Comments in Java code can be single-line or multiline javadoc comments.
2) Classes are fundamental in Java and describe data objects and methods that can be applied to objects.
3) Variables and methods have scopes determined by curly braces and a variable is only available within its scope.
This document provides an overview of the Scala programming language and discusses its usage on Android. Scala is a multi-paradigm language that integrates object-oriented and functional programming. It runs on the Java Virtual Machine and is used by companies like LinkedIn, Twitter, and The Guardian. Scala's features include being object-oriented, functional, statically typed, and expressive while allowing concise code. It supports classes, traits, pattern matching, and lazy evaluation among other things.
Scaladroids: Developing Android Apps with ScalaOstap Andrusiv
Android is a fast-growing mobile operating system with millions of users worldwide.
Scala is a modern programming language designed to express common programming patterns in a concise, elegant, and type-safe way. Why not combine both of them?
Presentation provides introduction and detailed explanation of the Java 8 Lambda and Streams. Lambda covers with Method references, default methods and Streams covers with stream operations,types of streams, collectors. Also streams are elaborated with parallel streams and benchmarking comparison of sequential and parallel streams.
Additional slides are covered with Optional, Splitators, certain projects based on lambda and streams
This document provides an introduction to Scala concepts and features compared to Java, including how to set up Scala, the Simple Build Tool (SBT), case classes, lazy definitions, imports, objects, pattern matching, collections, higher-order functions, partial functions, currying, implicit conversions, and implicit parameters. Useful Scala resources are also listed.
This document provides an introduction to the Scala programming language. It begins with an overview of Scala's motivation and history. It then covers the basics of Scala including simple data structures, loops, objects, types and generics. More advanced topics such as traits, mixins, implicit conversions and sealed classes are also discussed. The document concludes with references for further reading.
This document provides an introduction to Scala reflection. It discusses reflection concepts like the Universe, Mirrors, Symbols, and examples of using reflection. The Universe is the entry point and provides types, symbols, trees, names, and other metadata. Mirrors provide reflective information through different types like ClassMirrors and ModuleMirrors. Symbols represent bindings between names and entities like classes and methods. Examples demonstrate getting mirrors for classes and objects, invoking constructors and methods reflectively, and retrieving symbols.
This document summarizes a presentation about practical type mining in Scala using reflection. It discusses how Salat used pickled Scala signatures before 2.10 to mine types without runtime reflection. It covers benefits of 2.10 reflection like choosing runtime vs compile-time reflection. It provides an overview of universes, mirrors, symbols and trees, and how to navigate and inspect them. It also discusses limitations like non-thread-safe reflection and provides examples of macros libraries.
An internal presesentation of the new features in the scaladoc tool in the 2.10 release of the Scala programming language compiler.
It explains new features and gives usage examples tailored for documenting the new reflection part of the Scala library.
Metaprogramming involves writing programs that act on other programs. It is used in applications like compiler construction, domain-specific optimization, and generative programming. Effective metaprogramming tools are needed to make the technique accessible to more programmers. This document discusses requirements for metaprogramming tools, including representations of programs, methods for analysis and transformation, and interfaces to external analysis tools. It also outlines proposals for logical frameworks and metalanguages to formally define and orchestrate metaprogramming operations.
Update:
Video available from Oredev: vimeo.com/53147485
Metaprogramming is the dirty little secret behind the success of many Java frameworks such as Spring and Struts2, and constitutes the backbone of many of the most fundamental APIs across the JEE technology stack. This session aims introduce the topic and highlight, with code examples, the different mechanisms and techniques to take advantage of this underused feature of the Java Programming Language.
This session will adopt a learn-by-example approach that combines the philosophy and theory behind metaprogramming with concrete code examples. The audience will be walked real-life scenarios to highlight the benefits of this technique such as minimizing the number of lines of code, reduced development time, and greater flexibility, etc… Design patterns, and best practices will be picked up along the way.
No previous knowledge or exposure to the topic is required, but an intermediate understanding of Java SE is expected.
Spark Summit EU talk by Stavros kontopoulos and Justin PihonySpark Summit
This document provides recipes for solving common Spark problems including Out of Memory errors, NoSuchMethod errors, slow joins, issues with large datasets, speculation failures, unsafe stream recovery, and handling data from S3. It discusses causes and solutions for each problem type. For Out of Memory errors, it recommends tuning memory parameters, planning cluster resources, and avoiding repartitioning. For NoSuchMethod errors, it suggests shading libraries, enforcing load order, upgrading dependencies, or downgrading libraries. For slow joins, it advises broadcast joins, explain plans, and join strategies.
Introduction to Annotations.
What are Annotations➔ Annotations are structured information added to program source code➔ Annotations associated meta-information with definitions➔ They can be attached to any variable, method, expression, or another program element Like comments, they can be sprinkled throughout a program➔ Unlike comments, they have structure, thus making them easier to machine process.
Luca Canali presented on using flame graphs to investigate performance improvements in Spark 2.0 over Spark 1.6 for a CPU-intensive workload. Flame graphs of the Spark 1.6 and 2.0 executions showed Spark 2.0 spending less time in core Spark functions and more time in whole stage code generation functions, indicating improved optimizations. Additional tools like Linux perf confirmed Spark 2.0 utilized CPU and memory throughput better. The presentation demonstrated how flame graphs and other profiling tools can help pinpoint performance bottlenecks and understand the impact of changes like Spark 2.0's code generation optimizations.
Akka Cluster allows distributing actors across multiple JVMs with no single point of failure. The document discusses challenges faced with unreachable members and journal lifecycles when operating an Akka Cluster application at scale for 10 months. For unreachables, triggering a scale-in to mark nodes as down and automating restarts addressed the issue. Journals stored in Redis required cleanup to avoid inconsistencies, as deleting messages did not remove the highest sequence number. Straying from event sourcing's complete event history model weakened ecosystem support.
After the construction of several datalakes and large business intelligence pipelines, we now know that the use of Scala and its principles were essential to the success of those large undertakings.
In this talk, we will go through the 7 key scala-based architectures and methodologies that were used in real-life projects. More specifically, we will see the impact of these recipes on Spark performances, and how it enabled the rapid growth of those projects.
- Apache Spark is an open-source cluster computing framework for large-scale data processing. It was originally developed at the University of California, Berkeley in 2009 and is used for distributed tasks like data mining, streaming and machine learning.
- Spark utilizes in-memory computing to optimize performance. It keeps data in memory across tasks to allow for faster analytics compared to disk-based computing. Spark also supports caching data in memory to optimize repeated computations.
- Proper configuration of Spark's memory options is important to avoid out of memory errors. Options like storage fraction, execution fraction, on-heap memory size and off-heap memory size control how Spark allocates and uses memory across executors.
Join operations in Apache Spark is often the biggest source of performance problems and even full-blown exceptions in Spark. After this talk, you will understand the two most basic methods Spark employs for joining DataFrames – to the level of detail of how Spark distributes the data within the cluster. You’ll also find out how to work out common errors and even handle the trickiest corner cases we’ve encountered! After this talk, you should be able to write performance joins in Spark SQL that scale and are zippy fast!
This session will cover different ways of joining tables in Apache Spark.
Speaker: Vida Ha
This talk was originally presented at Spark Summit East 2017.
The document discusses Scala reflection, which provides a unified API to access information about a program at both compile-time and runtime. This reflection API exposes compiler trees, symbols, and types through universes, mirrors, and symbols. It allows programs to inspect and analyze themselves, defeat erasure, and even compile code at runtime or compile-time through macros. The reflection features are available in Scala 2.10 and aim to give programs almost as much insight into themselves as the compiler has.
The document discusses Scala reflection, which provides a unified API to access information about a program at both compile-time and runtime. This API allows programs to inspect their structure, types, and symbols using trees, mirrors, and universes. It facilitates tasks like runtime compilation, reifying types, and writing macros. The current implementation is available in Scala 2.10 M3, with runtime reflection improved for the upcoming M4 release.
Qcon2011 functions rockpresentation_scalaMichael Stal
Scala functions provide powerful ways to decompose problems and harness the benefits of functional programming. The presentation introduces core concepts of functional programming using Scala as an example language, highlighting how Scala combines object-oriented and functional programming. It presents benefits like immutability, higher-order functions, and improved concurrency while avoiding past performance issues through modern virtual machines and libraries.
This document summarizes a talk given about Nokia's migration to Scala for its Places API. The key points are:
1) Nokia migrated its Places API codebase to Scala to take advantage of Scala's features like its powerful type system, immutable data structures, and functional programming capabilities.
2) The migration was done gradually over time while continuing to develop new features. They discovered many benefits of Scala along the way like improved test readability and JSON parsing capabilities.
3) Nokia uses Scala features like case classes, options, and functions to model data and add type safety to its codebase. This uncovered bugs that would have been hard to find in Java.
This document provides a taxonomy of Scala concepts including object-oriented features, pattern matching, functional programming, actors, futures, implicits, type theory, macros, and category theory. It aims to serve as a reference for many of the terms used in the Scala community. The document covers topics such as case classes, lazy definitions, imports, objects, tuples, pattern matching examples, immutable collections, higher order functions, parallel collections, partial functions, currying, actors, futures, implicit conversions, implicit parameters, implicit classes, type inference, type classes, higher kinded types, algebraic data types, macros, concepts and arrows in category theory, morphisms, and functors.
This document provides an introduction to Scala for Java developers. It discusses that Scala is a hybrid object-oriented and functional language that runs on the JVM and interoperates well with Java. It highlights several features of Scala that allow for more concise code compared to Java, such as type inference, expressions instead of statements, higher-order functions, and case classes.
A short introduction (with many examples) to the Scala programming language and also an introduction to using the Play! Framework for modern, safe, efffcient and reactive web applications.
The document discusses Scala and why some developers think it could replace Java on the JVM. It provides quotes from several influential developers, including the creator of Java and Groovy, expressing their view that Scala is the best candidate to replace Java in the long run. Specifically, James Gosling says that if he had to pick another language on the JVM besides Java, it would be Scala. Charlie Nutter describes Scala as the current heir apparent to the Java throne and the momentum behind Scala is now unquestionable. James Strachan says that if he had seen the Programming in Scala book in 2003, he may have never created Groovy and that his tip for a long term replacement for Java is
Martin Odersky received his PhD in 1989 and began designing Scala in 2001 at EPFL. Scala is a functional and object-oriented programming language that runs on the Java Virtual Machine. It is concise, high-level, statically typed, and supports both immutable and mutable data structures. Many large companies use Scala including LinkedIn, Twitter, and Ebay. Scala supports both object-oriented programming with classes, traits, and objects as well as functional programming with immutable data, higher-order functions, and algebraic data types.
This document introduces Scala and compares it to Java. It discusses Scala's features like being statically typed yet having type inference, being object-oriented and functional, and being concise. It provides examples of Scala code and how Scala can simplify tasks like parsing files, working with collections, pattern matching, and XML processing. It also discusses how Scala is used in industry, tools for using Scala, and resources for learning Scala.
Apache Spark is a fast and general cluster computing system that improves efficiency through in-memory computing and usability through rich APIs. Spark SQL provides a way to work with structured data and transform RDDs using SQL. It can read data from sources like Parquet and JSON files, Hive, and write query results to Parquet for efficient querying. Spark SQL also allows machine learning pipelines to be built by connecting SQL queries to MLlib algorithms.
This document provides an introduction to the Scala programming language. It discusses what Scala is, how to get started, basic concepts like mutability and functions, and Scala features like classes, traits, pattern matching, and collections. Scala combines object-oriented and functional programming. It runs on the Java Virtual Machine and is compatible with Java. The document provides code examples to demonstrate Scala concepts and features.
This document introduces higher order functions (HOFs) in Scala. It provides examples of how HOFs such as map and filter can transform collections in more idiomatic and functional ways compared to imperative approaches using loops. Key benefits of HOFs include producing immutable and thread-safe results without needing to manually manage intermediate data structures. The document also briefly outlines some other powerful HOFs like reduce, partition, min, max and parallel collections.
Spring Day | Spring and Scala | Eberhard WolffJAX London
2011-10-31 | 09:45 AM - 10:30 AM
Spring is widely used in the Java world - but does it make any sense to combine it with Scala? This talk gives an answer and shows how and why Spring is useful in the Scala world. All areas of Spring such as Dependency Injection, Aspect-Oriented Programming and the Portable Service Abstraction as well as Spring MVC are covered.
Fuel Up JavaScript with Functional ProgrammingShine Xavier
JavaScript is the lingua franca of web development for over a decade. It has evolved tremendously along with the Web and has entrenched in modern browsers, complex Web applications, mobile development, server-side programming, and in emerging platforms like the Internet of Things.
Eventhough JavaScript has come a long way, a reinforced makeover to it will help build concurrent and massive systems that handle Big Data, IoT peripherals and many other complex eco systems. Functional Programming is the programming paradigm that could empower JavaScript to to enable more effective, robust, and flexible software development.
These days, Functional Programming is at the heart of every new generation programming technologies. The inclusion of Functional Programming in JavaScript will lead to advanced and futuristic systems.
The need of the hour is to unwrap the underlying concepts and its implementation in the software development process.
The 46th edition of FAYA:80 provides a unique opportunity for the JavaScript developers and technology enthusiasts to shed light on the functional programming paradigm and on writing efficient functional code in JavaScript.
Join us for the session to know more.
Topics Covered:
· Functional Programming Core Concepts
· Function Compositions & Pipelines
· Use of JS in Functional Programming
· Techniques for Functional Coding in JS
· Live Demo
This document provides an overview of Scala-ActiveRecord, a type-safe Active Record model library for Scala. It discusses features such as being type-safe, having Rails ActiveRecord-like functionality, automatic transaction control, and support for associations and validations. The document also covers getting started, defining schemas, CRUD operations, queries, caching queries, validations, callbacks, and relationships.
Functional programming uses immutable values, pure functions, and precise types to simplify code, enable free concurrency and parallelism, and make code easier to reason about and reuse. It aims to reduce complexity by eliminating state changes and side effects. Key aspects include immutable values that can be safely shared, pure functions without side effects, recursion to loop, and higher order functions that take or return other functions. Functional programming benefits include simple, reusable, and parallelizable code that is easier to reason about due to the absence of mutation and side effects.
During the talk, we will build a simple web app using Lift and then introduce Akka ( http://akkasource.org) to help scale it. Specifically, we will demonstrate Remote Actors, "Let it crash" fail over, and Dispatcher. Other Scala oriented tools we will use include sbt and ENSIME mode for emacs.
This document summarizes a presentation about scaling web applications with Akka. It discusses how Akka uses an actor model of computation with message passing between lightweight processes to enable safe concurrency. Key features of Akka that help with scaling include fault tolerance through supervision, flexible dispatch strategies to leverage multiple cores, and support for NoSQL databases through pluggable storage backends. The presentation provides code examples of implementing actors in Akka and other frameworks and concludes by taking questions about Akka.
Similar to Scala Reflection & Runtime MetaProgramming (20)
Vijay Engineering and Machinery Company (VEMC) is a leading company in the field of electromechanical engineering products and services, with over 70 years of experience.
Bell Crank Lever.pptxDesign of Bell Crank Leverssuser110cda
In a bell crank lever, the two arms of the lever are at right angles.
Such type of levers are used in railway signalling, governors of Hartnell type, the drive for the air pump of condensers etc.
The bell crank lever is designed in a similar way as discussed earlier.
Agricultural Profitability through Resilience: Smallholder Farmers' Strategie...IJAEMSJORNAL
This study investigated the knowledge strategies and coping utilized by smallholder farmers in Guimba, Nueva Ecija to reduce and adjust to the effects of climate change. Smallholder farmers, who are frequently susceptible to climate change, utilize various traditional and innovative methods to strengthen their ability to withstand and recover from these consequences. Based on the results of this study, farmers in Guimba, Nueva Ecija demonstrate a profound comprehension of the adverse weather conditions, such as typhoons, droughts, and excessive rainfall, which they ascribe to climate change. While they have a fundamental understanding of climate change and its effects, their knowledge of scientific intricacies is restricted, indicating a need for information that is particular to the context. Although farmers possess knowledge about climate change, they are not actively engaging in proactive actions to adapt to it. Instead, they rely on reactive coping mechanisms. This highlights the necessity for targeted educational and communicative endeavors to promote the acceptance and implementation of approaches. Furthermore, the absence of available resources poses a significant barrier to achieving successful adaptation, highlighting the importance of pushing for inexpensive and feasible measures for adaptation. Farmers recognize the benefits of agroforestry and have started integrating the growth of fruit trees, particularly mangoes, into their coping techniques.
Predicting damage in notched functionally graded materials plates thr...Barhm Mohamad
Presently, Functionally Graded Materials (FGMs) are extensively utilised in several industrial sectors, and the modelling of their mechanical behaviour is consistently advancing. Most studies investigate the impact of layers on the mechanical characteristics, resulting in a discontinuity in the material. In the present study, the extended Finite Element Method (XFEM) technique is used to analyse the damage in a Metal/Ceramic plate (FGM-Al/SiC) with a circular central notch. The plate is subjected to a uniaxial tensile force. The maximum stress criterion was employed for fracture initiation and the energy criterion for its propagation and evolution. The FGM (Al/SiC) structure is graded based on its thickness using a modified power law. The plastic characteristics of the structure were estimated using the Tamura-Tomota-Ozawa (TTO) model in a user-defined field variables (USDFLD) subroutine. Validation of the numerical model in the form of a stress-strain curve with the findings of the experimental tests was established following a mesh sensitivity investigation and demonstrated good convergence. The influence of the notch dimensions and gradation exponent on the structural response and damage development was also explored. Additionally, force-displacement curves were employed to display the data, highlighting the fracture propagation pattern within the FGM structure.
Structural Dynamics and Earthquake Engineeringtushardatta
Slides are prepared with a lot of text material to help young teachers to teach the course for the first time. This also includes solved problems. This can be used to teach a first course on structural dynamics and earthquake engineering. The lecture notes based on which slides are prepared are available in SCRIBD.
,*$/?!~00971508021841^(سعر حبوب الإجهاض في دبيnafizanafzal
,*$/?!~00971508021841^(سعر حبوب الإجهاض في دبي)حبوب سايتوتك في ام القيوينالاجهاض للبيع في الامارات اسقاط الجنين بدبي حبوب الحمل للبيع # بيع؟ ؟ #شراء؟ ؟ #حبوب؟ ؟ #الاجهاض؟ #سايتوتك؟ #في؟ ؟ #دبي؟ ؟ #الشارقه؟ ؟ #عجمان؟ ؟ #العين؟ ؟ #ابوظبي؟ #الجنين؟ #سايتوتك؟ ؟ #للبيع؟ Cytotec # # الامارات # في؟ #دبي؟ # سايتوتك للبيع من داخل # دبي # شارقه # عجمان للطلب من باقي الدول في الخل #Data Opennesيتضمن قرار الإجهاض في عيادة الإجهاض في أبو ظبي ، الإمارات العربية المتحدة ، اعتبارات أخلاقية وأخلاقية ودينية وعائلية ومالية وصحية وعصر. شراء حبوب الإجهاض في دبي ، شراء حبوب الإجهاض في عمان ، شراء حبوب الإجهاض في أبو ظبي ، شراء حبوب الإجهاض في الشارقة ، شراء حبوب الإجهاض في رأس الخيمة ( RAK ), شراء حبوب الإجهاض في # عجمان ، شراء حبوب الإجهاض في العين ، شراء حبوب الإجهاض في أم القيوين حبوب الإجهاض الحصرية للبيع في دبي.
أين يمكنني شراء حبوب الإجهاض في دبي / الإمارات العربية المتحدة?
هل يمكنني الحصول على حبوب الإجهاض في دبي?
عيادة إجهاض النساء في الإمارات / دبي
أين يتم الإجهاض في الإمارات / دبي / أبو ظبي
عيادة الإجهاض الآمن في الإمارات / دبي / أبو ظبي.
أفضل عيادة إجهاض في الإمارات / دبي / قطر
حبوب الإجهاض عبر الإنترنت AMAZON / DUBAI / الإمارات العربية المتحدة.
حبوب الإجهاض في DISC HEM في دبي.
تكلفة حبوب الإجهاض في أبو ظبي / الإمارات.
حبوب الإجهاض بسعر الخصم الإمارات / دبي.
حبوب الإجهاض تظهر في دبي.
سعر حبوب الإجهاض في دبي.
حبوب الإجهاض في قطر.
حبوب الإجهاض آثار جانبية.
أنا حبوب الإجهاض في أبو ظبي.
أطقم أطقم غير مرغوب فيها في دبي / الإمارات العربية المتحدة
أطقم أطقم غير مرغوب فيها في أبو ظبي
أطقم أطقم غير مرغوب فيها في أجمان
أطقم أطقم غير مرغوب فيها في الكويت
أطقم أطقم غير مرغوب فيها في قطر / الدوحة
حبوب الإجهاض الإماراتية.
حبوب الإجهاض 1MG KUWAIT.
حبوب الإجهاض لمدة 12 أسبوعًا في دبي.
حبوب الإجهاض 24 ساعة في الإمارات / دبي.
حبوب الإجهاض بعد شهرين في هندي.
حبوب الإجهاض بعد شهرين في دبي.
حبوب الإجهاض تصل إلى 3 أشهر في دبي.
486 حبوب الإجهاض.
أفضل مجموعة في دبي / الإمارات.
حبوب الإجهاض 500.الإمارات العربية المتحدة
حبوب الإجهاض غير مرغوب فيها 72 دبي
2. About Me
Meir Maor
Chief Architect @ SparkBeyond
At SparkBeyond we leverage the collective
human knowledge to solve the world's toughest
problems
3. This Talk
Showcase Scala reflection, and compiler toolbox
Use a real guiding example to see how these are
used to solve a real problem
4. SparkBeyond
SparkBeyond generates code to solve machine
learning problems, leveraging world knowledge
Allows crafting deeply embedded pipelines to
explicitly transform your data
5. Guiding Example
● We would like to build a deeply embedded
framework to transform data.
● Multiple classes define actions
● Can be stacked together
● Embed user generated Scala code
● Auto generate a UI to input parameters for
actions.
● Allow annotating fields to guide UI generation
6. case class AddColumn(columnName: String,codeExpression: String) extends Action
{...}
case class SpecialParam(a: Int,b : Int => Int)
object SpecialAction {
val myMap = Map("mul2" -> SpecialParam(1,_ * 2),
"add1" -> SpecialParam(2,_ + 1),
"ident" -> SpecialParam(3, identity)
)
}
case class SpecialAction(@PossibleValuesMap(SpecialAction.myMap) foo:
SpecialParam) extends Action
7. Intro to Scala Meta Programming
Creating and using mirror
import reflect.runtime.universe._
val cm =
runtimeMirror(getClass.getClassLoader)
8. Know the players
● Type
● Symbol
● Tree
documentation reference: http://docs.scala-lang.org/overviews/reflection/symbols-trees-types.html
10. Symbol
● If it has a name it has a Symbol
– Types, Members, parameters
Not a type, even TypeSymbol is less than a Type
val ts=t.typeSymbol
ts: reflect.runtime.universe.Symbol =
class List
11. Tree
● An Abstract Syntax Tree
val tree = Apply(Select(Ident(TermName("x")),
TermName("$plus")), List(Literal(Constant(2))))
tree: scala.reflect.runtime.universe.Apply = x.$plus(2)
val expr = reify { class Flower { def name = "Rose" } }
expr: scala.reflect.runtime.universe.Expr[Unit] = …
expr.tree
12. Get param list from primary ctr
scala> val typ = typeOf[AddColumn]
typ: reflect.runtime.universe.Type = AddColumn
val primary=typ.typeSymbol.asClass.primaryConstructor
primary: reflect.runtime.universe.Symbol = constructor AddColumn
//recall a method can have multiple param lists, hence flaten
val paramList = primary.asMethod.paramLists.flatten
paramList: List[reflect.runtime.universe.Symbol] = List(value
columnName, value codeExpression)
13. Code Generation
● Cglib – byte code generation library
● Compiling externally and loading with a fresh
classloader
● The unsafe way - mutate the existing
classloader
● Scala Compiler toolbox
14. UrlClassLoader
def getClassInstanceFromJar[T](jarFile: File, className:
String): T = {
val classLoader = new
scala.reflect.internal.util.ScalaClassLoader.URLClassLoader(Seq(ja
rFile.toURI.toURL), this.getClass.getClassLoader)
val mirror =
scala.reflect.runtime.universe.runtimeMirror(classLoader)
val classSymbol = mirror.staticClass(className)
val ctorSymbol = classSymbol.primaryConstructor.asMethod
val ctor =
mirror.reflectClass(classSymbol).reflectConstructor(ctorSymbol)
ctor.apply().asInstanceOf[T]
}
15. Mutate existing ClassLoader
public static void addURL(URL u) throws IOException {
URLClassLoader sysloader = (URLClassLoader)
ClassLoader.getSystemClassLoader();
Class sysclass = URLClassLoader.class;
try {
Method method = sysclass.getDeclaredMethod("addURL", parameters);
method.setAccessible(true);
method.invoke(sysloader, new Object[]{u});
} catch (Throwable t) {
t.printStackTrace();
throw new IOException("Error, could not add URL to system
classloader");
}//end try catch
}//end method
16. Compiler ToolBox
● Experimental (even more so than the rest of the reflection api)
import scala.tools.reflect.ToolBox
val tb = cm.mkToolBox()
● Parse, typecheck, eval
def run(code: String) = {
synchronized {
val tree = tb parse s"import scala._nimport Predef._n $code"
tb eval tree
}
}
17. Using the ToolBox
● Get User code fragment
● Add boilerplate function definition, useful
variables
● Parse and eval
● Apply function on new data
18. Annotations
● In Java you can easily read annotations reflectively at
runtime
● Annotations can only hold specific types:
– primitive
– String
– Class
– an Enum
– another Annotation
– an array of any of the above
19. Scala annotations
● Scala annotations are not Java annotations
● You can put anything in a Scala annotation
● But How do you read it?
Documentation: http://docs.scala-lang.org/overviews/reflection/annotations-names-scopes.html
20. Defining Scala Annotation
Extend StaticAnnotation to make it available at
Runtime
case class ValuesMap(vals: Map[String, Any]) extends
StaticAnnotation
21. Get annotations from a symbol
● Many things can be annotated
– A type, A Method, A parameter, a val/var
● Recall our example:
case class SpecialAction(@ValuesMap(SpecialAction.myMap) foo:
SpecialParam) extends Action
22. Get annotations from a symbol
val param: Symbol = longestParmList.head
param: reflect.runtime.universe.Symbol = value foo
val annotations = param.annotations
scala> annotations: List[reflect.runtime.universe.Annotation] =
List(com.sparkbeyond.runtime.util.misc.ValuesMap(SpecialAction.m
yMap)
23. Recreate Annotation Instance
scala> val tree=annotations.head.tree
tree: reflect.runtime.universe.Tree = new
com.sparkbeyond.runtime.util.misc.ValuesMap(SpecialAction.myMap)
scala> tb.eval(tb.untypecheck(tree))
res15: Any = ValuesMap(Map(mul2 -> SpecialParam(1,<function1>), add1 ->
SpecialParam(2,<function1>), ident -> SpecialParam(3,<function1>)))
24. Overcoming Erasure - RTTI
● Java Generics for better or worse get erased in
runtime
● We would like to be able to tell the Type of an
instance including Type Parameters
● In most cases this is possible in a meaningful
way.
25. What is my type?
case class OneIntSeq(a: Int) extends Seq[Int] {...}
case class MyCaseClass[T,U](e: T,o: Seq[U])
val v = MyCaseClass("foo",OneIntSeq(1))
We want a method, given v to produce:
MyCaseClass[String,Int]
26. val topType=cm.reflect(v).symbol.toTypeConstructor
topType: reflect.runtime.universe.Type = MyCaseClass
//As Before
val primaryParams = topType.typeSymbol.asClass.primaryConstructor...
primaryParams: List[reflect.runtime.universe.Symbol] = List(value elem,
value other)
primaryParams.map(_.typeSignature)
res29: List[reflect.runtime.universe.Type] = List(T, scala.Seq[U])
27. Inspecting the Type
topType.decls.foreach{case s => println(s.toString + "t" + s.typeSignature) }
value elem => T
value elem T
value other => scala.Seq[U]
value other scala.Seq[U]
constructor MyCaseClass (elem: T, other: scala.Seq[U])MyCaseClass[T,U]
method copy [T, U](elem: T, other: scala.Seq[U])MyCaseClass[T,U]
method copy$default$1 [T, U]=> T @scala.annotation.unchecked.uncheckedVariance
method copy$default$2 [T, U]=> scala.Seq[U] @scala.annotation.unchecked.uncheckedVariance
method productPrefix => java.lang.String
method productArity => scala.Int
method productElement (x$1: scala.Int)scala.Any
method productIterator => Iterator[scala.Any]
method canEqual (x$1: scala.Any)scala.Boolean
method hashCode ()scala.Int
method toString ()java.lang.String
method equals (x$1: scala.Any)scala.Boolean
28. val instanceMirror = cm.reflect(v)
instanceMirror: reflect.runtime.universe.InstanceMirror = instance
mirror for MyCaseClass(foo,(1))
val elemField=topType.decls.filterNot(x => x.isMethod).head
elemField: reflect.runtime.universe.Symbol = value elem
val elemValue=instanceMirror.reflectField(elemField.asTerm).get
elemValue: Any = foo
val elemTyp=cm.reflect(elemValue).symbol.toTypeConstructor
elemTyp: reflect.runtime.universe.Type = java.lang.String
29. Similarly for second param other: Seq[U]
val otherTyp= cm.reflect(otherValue).symbol.toTypeConstructor
otherTyp: reflect.runtime.universe.Type = OneIntSeq
val asBase=otherTyp.baseType(otherField.typeSignature.typeSymbol)
asBase: reflect.runtime.universe.Type = Seq[scala.Int]
val uType = asBase.typeArgs.head
uType: reflect.runtime.universe.Type = scala.Int
30. Combining Things together
val finalType=appliedType(topType,List(elemTyp,uType))
finalType: reflect.runtime.universe.Type =
MyCaseClass[java.lang.String,scala.Int]
Q.E.D