๐๐ฒ๐๐ฒ๐น๐ผ๐ฝ๐บ๐ฒ๐ป๐ ๐ง๐ถ๐ฝ ๐๐ผ ๐ฎ๐๐ผ๐ถ๐ฑ ๐น๐ฒ๐ฎ๐ธ๐ถ๐ป๐ด ๐๐ฒ๐ป๐๐ถ๐๐ถ๐๐ฒ ๐ฑ๐ฎ๐๐ฎ ๐ผ๐ป ๐ถ๐ข๐ฆ Filter properties in your logging system at runtime. ๐๐ถ๐น๐๐ฒ๐ฟ ๐ฝ๐ฟ๐ผ๐ฝ๐ฒ๐ฟ๐๐ถ๐ฒ๐ ๐๐๐ถ๐ป๐ด ๐ฅ๐ฒ๐ด๐ฒ๐ - use Regex to define specific patterns you can filter your logs with and redact values if any matches are found. ๐ฃ๐ฟ๐ผ: you can define intricate patterns to search for ๐๐ผ๐ป: you have to handle localisation & formatting yourself ๐๐ถ๐น๐๐ฒ๐ฟ ๐ฝ๐ฟ๐ผ๐ฝ๐ฒ๐ฟ๐๐ถ๐ฒ๐ ๐๐๐ถ๐ป๐ด ๐ก๐ฆ๐๐ฎ๐๐ฎ๐๐ฒ๐๐ฒ๐ฐ๐๐ผ๐ฟ - filter common data types (phone numbers, URLs, addresses). It is an approach that requires less work than regex but can fail if the data doesnโt follow common structures. ๐ฃ๐ฟ๐ผ: it handles localisation and formatting for the data types it detects (nicer for international use) ๐๐ผ๐ป: Less flexible and sometimes won't pick up on patterns if it doesnโt follow a well-known structure (e.g. it will not detect addresses from some countries/areas that don't follow common address formats) ๐๐ถ๐น๐๐ฒ๐ฟ ๐ฝ๐ฟ๐ผ๐ฝ๐ฒ๐ฟ๐๐ถ๐ฒ๐ ๐๐๐ถ๐ป๐ด ๐ฎ ๐๐น๐ฎ๐ฐ๐ธ๐น๐ถ๐๐ - Blacklist specific properties you do not want to log e.g. firstname or lastname and apply this filter across all logs. ๐ฃ๐ฟ๐ผ: very specific matching for the properties you don't want ๐๐ผ๐ป: very specific matching for the properties you don't want ๐ฌ #iosdevelopment #security
Alex Lotsuโs Post
More Relevant Posts
-
The credentials to an organization's systems and data are prime targets for malicious actors.Traditional secret detetions that are used to detect credentials in code within developer workflows are limited by regex capabilities and canโt leverage any form of data-flow analysis for detecting secrets. To overcome this, one of my secrets in code analysis is manual semantic analysis, not just regex. Well, Semgrep added this to their workflow. Semgrep Secrets can reason about how data is used within your code. Its awesome you can try it.
To view or add a comment, sign in
-
-
Data analytics & Product Implementation, ELK Stack,Tableau,Oracle database,Linux,Microsoft Dynamics AX,ElasticSearch,APM, Apache Kafka.
Need to explore how useful it is,if this can be alternative to DSL for dev tools or can be used in painless scripting.This would help many folks struggling with JSON query syntax๐
2 days ago, Elastic presented her new query language - ES|QL(ElasticSearch Query Language). This query language will allow a single, consolidated way to interact with Elasticsearch โ one that brings comprehensive compute capabilities close to the data and eliminates the need of expensive transfers to external systems for custom processing. In one of the examples that can be found below, we can see that in single query preform filtering, processing, grouping, renaming, sorting, look-ups and column pruning. ES|QL brings a new execution engine designed with performance in mind โ one that operates on blocks at a time instead of per row, targets vectorization and cache locality, and embraces specialization and multi-threading. feel free to contact us for more info about this game changer ๐ฏ Ofir Adi Idan Moshe Arthur Gimpel Ido Friedman Nick Van Dyk Lior Halfin Eran Frid Miki Levinshtain Meytal Balas Elad Shalvi Dror Mor โ๏ธLeah Brand Zeev Shavit Sagi Kelner Max Totney KPMG Israel
To view or add a comment, sign in
-
-
Senior DevOps Engineer @ Solirius | Deengineers Tech Community Founder ๐ | Founder of CoderCo โก๏ธ
๐งฒ ๐ฅ๐ฒ๐ด๐๐น๐ฎ๐ฟ ๐๐ ๐ฝ๐ฟ๐ฒ๐๐๐ถ๐ผ๐ป๐ (๐ฅ๐ฒ๐ด๐ฒ๐ ) Regular expressions, or regex, might seem daunting at first glance, but they're an incredibly powerful tool for searching, parsing, and manipulating text data. Let's decode the basics: ๐ ๐๐๐๐ฉ ๐๐จ ๐ง๐๐๐๐ญ? Regex is a sequence of characters that forms a search pattern. It can be used for 'find' or 'find and replace' operations, syntax validation, and more. ๐ก ๐พ๐ค๐ข๐ข๐ค๐ฃ ๐๐๐๐๐ญ ๐๐ฎ๐ฃ๐ฉ๐๐ญ: ๐น.: Matches any character except newline ๐น``: Matches 0 or more repetitions of the preceding character ๐น+: Matches 1 or more repetitions of the preceding character ๐น?: Matches 0 or 1 repetition of the preceding character ๐น[abc]: Matches any of the characters enclosed in the brackets ๐น[^abc]: Matches any character NOT enclosed in the brackets (abc|def): Matches abc or def ๐ฏ ๐ผ๐ฅ๐ฅ๐ก๐๐๐๐ฉ๐๐ค๐ฃ๐จ ๐ค๐ ๐ง๐๐๐๐ญ: ๐ธ Extracting emails or URLs from a block of text ๐ธ Validating input formats (like phone numbers, emails) ๐ธ Parsing logs or large data files ๐ธ Code refactoring Mastering regex can save you hours of manual search and manipulation, and bring your data manipulation skills to the next level. #regex #data #linux #techdeepdive โ ๐ Follow me on LinkedIn and Subscribe to my newsletter for more detailed articles on System Design, DevOps, and Software Engineering (see comments ๐๐ฝ)
To view or add a comment, sign in
-
-
2 days ago, Elastic presented her new query language - ES|QL(ElasticSearch Query Language). This query language will allow a single, consolidated way to interact with Elasticsearch โ one that brings comprehensive compute capabilities close to the data and eliminates the need of expensive transfers to external systems for custom processing. In one of the examples that can be found below, we can see that in single query preform filtering, processing, grouping, renaming, sorting, look-ups and column pruning. ES|QL brings a new execution engine designed with performance in mind โ one that operates on blocks at a time instead of per row, targets vectorization and cache locality, and embraces specialization and multi-threading. feel free to contact us for more info about this game changer ๐ฏ Ofir Adi Idan Moshe Arthur Gimpel Ido Friedman Nick Van Dyk Lior Halfin Eran Frid Miki Levinshtain Meytal Balas Elad Shalvi Dror Mor โ๏ธLeah Brand Zeev Shavit Sagi Kelner Max Totney KPMG Israel
To view or add a comment, sign in
-
-
๐ Natural Language => LLM => SQL ๐ One of the most prevalent uses of LLMs in production among our prospects at Prompt Security nowadays is translating natural language into SQL. This awesome tutorial demonstrates how to create a natural language to SQL code generator using LLM in a few minutes. With the OpenAI API, coupled with a touch of Pandas and DuckDB, it's remarkable how straightforward it is to build - producing consistently accurate results most of the time. You can access the notebook here: https://lnkd.in/d3Ja_KkQ ๐ The entire repository is available here: https://lnkd.in/dUkf386S ๐ Upcoming on their roadmap, they plan to integrate support for open-source LLMs such as Llama2 and Huggingface Transformers ๐ค Huge thanks to my incredible colleague, Rami Krispin, who is the driving force behind this project ๐๐จ๐ป.
To view or add a comment, sign in
-
The ability to ask Observe's O11y Extract to write regexes for you is a game changer in terms of productivity (and your sanity). https://lnkd.in/eAKnHv4Q
To view or add a comment, sign in
-
Effortless Base64 Encoding & Decoding. SmartToolsAI.com's Base64 Encode Decode tool is the perfect solution for swiftly converting data using the Base64 encoding scheme. Base64 encoding simplifies data transfer and binary storage, making it easily readable and transferable. Simple and User-Friendly: 1. Encode: Enter text or upload a file for encoding 2. Click "Encode" 3. Get your Base64-encoded result โฌ ๏ธ OR โก๏ธ 1. Decode: Input encoded text or upload a file for decoding 2. Click "Decode" 3. Watch your Base64 data restored to its original form Perfect for Developers and webmasters: Save time and effort with our easy-to-use tool, catering to those handling data encoding and decoding tasks. Your data's security and privacy are our top priority. Your input is handled with the strictest confidentiality. Effortlessly handle Base64 data with Smart Tools AI's Base64 Encode Decode tool. Simplify data tasks, securely transfer data, and elevate your workflow. Explore Now: https://lnkd.in/gyrw3Mnw #Base64 #DataEncoding #SmartToolsAI
To view or add a comment, sign in
-
-
Algorithms | Talk's_About_DSA | Developmet | Problem_Solving| Tech| GFG Institute Rank #1|MERN STACK DEVELOPER
#365DaysOfCoding Hey #LinkedInConnections, today I'm excited to dive into the fascinating world of Trie data structures. ๐โจ Trie, pronounced as "try," stands for retrieval and is an efficient tree-like data structure that excels at handling and searching for strings. ๐๐ Whether you're building a search engine, autocomplete feature, or spell checker, Tries can be your go-to solution. ๐ Key Features: Efficient Searching: Tries are designed to make searching for strings lightning-fast. By breaking down words into their individual characters and storing them hierarchically, Tries dramatically reduce search time. Autocomplete and Suggestions: Ever wondered how your keyboard suggests words while typing? Tries are the magic behind this feature, providing real-time suggestions as you type. Space Efficiency: Tries are memory-efficient when dealing with a large number of words that share common prefixes. Traditional search structures like hash tables or binary search trees can't match this space-saving ability. Prefix Matching: Tries are perfect for tasks that involve finding all strings with a common prefix, such as IP routing or dictionary lookups. Insertion and Deletion: Tries can easily handle dynamic datasets. Insertions and deletions are efficient, making them suitable for applications with changing content. ๐ Real-World Applications: Search Engines: Google's search engine harnesses the power of Tries to deliver instant search results as you type, even for complex queries. Spell Checkers: Tries enable spell checkers to quickly suggest correct spellings or alternatives, enhancing user experience. Phone Book Apps: When you search for a contact by typing the first few letters, you're benefiting from Tries' prefix matching capabilities. IP Routing: Routers use Tries to efficiently match network prefixes and determine the best path for data packets. ๐ Let's Connect the Dots: Trie data structures might appear complex at first glance, but they offer a brilliant solution to various real-world challenges. By implementing Tries in your projects, you can unlock the potential for lightning-fast searches, autocomplete features, and more. ๐ Have you had experience working with Tries? Or are you intrigued to explore their applications? Let's chat! Comment below or connect with me to exchange thoughts and ideas. ๐ค๐ฃ๏ธ #TrieDataStructure #DataStructures #CodingMagic #SearchAlgorithms #TechInnovation
To view or add a comment, sign in
-
-
GEN AI | RAI | Intelligent Automation | NLP |Conversation AI | AI Assist | AI Ethics | Enterprise Search | VX | AWS | Azure | Servicenow | SFDC | SAP | Microservices | Cloud Integration SME/ Data Architect
API Response time is critical for User Experience :)
How to improve API response time and performance? One of the things that may help is... Response compression. Firstly, what is it? Response compression is a technique that reduces the size of API data sent from the server to the client. How it works? When a client makes a request to an API, the server can compress the response data before sending it back to the client. It can utilize a few algorithms for that, such as: - Gzip - Brotli Gzip is the most widely used algorithm with a balance between compression ratio and speed. It uses the DEFLATE compression algorithm, which combines the LZ77 algorithm and Huffman coding. Brotli is a faster alternative to Gzip, developed by Google. It uses the LZ77 algorithm, Huffman coding, and 2nd-order context modeling. But if you want to use custom algorithms, itโs also possible. The question that may arise here is: Will the response be corrupted? No, it will not. On the contrary, the amount of data is minimized without corruption, leading to benefits like: - Fast data transfer - Reduced bandwidth usage But when should you consider adding response compression to your application? A few use cases are: - In scenarios where bandwidth is limited or expensive - If your API responses contain large amounts of data - Environments with slow or unreliable network connections Otherwise, it may be overkill. What is your opinion on this subject? โ
To view or add a comment, sign in