FLaNK Stack 26 February 2024

Apache NiFi, Apache Kafka, Apache Flink, Open Source, LLM, Generative AI, ML, DL, IoT

26-February-2024

FLaNK Stack Weekly

Tim Spann @PaaSDev

https://pebble.is/PaaSDev

https://vimeo.com/flankstack

https://www.youtube.com/@FLaNK-Stack

https://www.threads.net/@tspannhw

https://medium.com/@tspann/subscribe

https://www.cloudera.com/campaign/apache-nifi-for-dummies.html

https://ossinsight.io/analyze/tspannhw

CODE + COMMUNITY

Please join my meetup group NJ/NYC/Philly/Virtual.

http://www.meetup.com/futureofdata-princeton/

https://www.meetup.com/futureofdata-newyork/

https://www.meetup.com/futureofdata-philadelphia/

This is Issue #126

https://github.com/tspannhw/FLiPStackWeekly

https://www.cloudera.com/solutions/dim-developer.html

Articles

Using Google Gemma

NYC Traffic?? (NiFi, Kafka, Flink)

Subways and Transit Updates in Real-Time

Open Source Data Infrastructure Meetup - Feb 2024

https://sap1ens.com/blog/2024/02/18/customizing-flink-class-shadowing/

https://www.projectpro.io/recipes/use-nifi-extract-and-parse-data-from-http-endpoints-and-store-data-persistent-storage

https://engineering.grab.com/attribution-platform

Why Are LLMs So Gullible?

https://huggingface.co/blog/gemma

https://developer.nvidia.com/blog/build-an-llm-powered-data-agent-for-data-analysis/

https://thenewstack.io/the-rise-of-small-language-models/

https://www.infoq.com/news/2024/02/pinterest-pubsub-kafka-flink/

https://www.infoq.com/news/2024/01/doordash-service-mesh/

https://thenewstack.io/demo-use-webassembly-to-run-llms-on-your-own-device-with-wasmedge

https://www.eleuther.ai/releases

https://www.microsoft.com/en-us/research/blog/orca-2-teaching-small-language-models-how-to-reason/

https://www.baeldung.com/ops/docker-remove-dangling-unused-images

AI + More required for startup https://www.nfx.com/post/ai-like-water

https://explainextended.com/2023/12/31/happy-new-year-15/

https://medium.com/sids-tech-cafe/event-driven-systems-lessons-from-the-trenches-107c07b3fc1d

From Samza to Flink: A Decade of Stream Processing

https://exaspark.medium.com/the-ultimate-guide-to-postgresql-data-change-tracking-c3fa88779572

https://www.wired.com/story/17-tips-better-chatgpt-prompts

https://github.com/microsoft/generative-ai-for-beginners/

Videos

Continuous SQL with Kafka and Flink

Building Real-time Pipelines: A Case Study by Transit Data

Unlocking Financial Data with Real-Time Pipelines (OSACon 2023)

The Never Landing Stream

February 8, 2024 NYC Meetup

February 20, 2024 Virtual Meetup

Feb 22, 2024 NYC Meetup

Events

Feb 28, 2024: NYC. Cloudera Meetup. Flink

Feb 29, 2024: Virtual. Conf42 Python.

March 6, 2024: Princeton. New Location. Meetup. GenAI. https://www.meetup.com/applied-generative-artificial-intelligence-applications/

March 15, 2024: TCF Pro. Princeton, NJ. IT Professional Conference at Trenton Computer Festival IEEE Information Technology Professional Conference on Friday, March 15th, 2024 https://princetonacm.acm.org/tcfpro/

March 28, 2024: Pinot + NiFi + Flink + Kafka Meetup NYC https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/

April 2024: XtremeJ 2024. Virtual.

https://xtremej.dev/2023/schedule/

April 11, 2024: Conf42 LLM. Virtual.

https://www.conf42.com/llms2024

May 8-9, 2024: Data Summit 2024. Boston, MA. https://www.dbta.com/DataSummit/2024/default.aspx

Cloudera Events

https://www.cloudera.com/about/events.html

More Events:

https://www.linkedin.com/pulse/schedule-2024-tim-spann--y4coe

Code

• https://github.com/tspannhw/FLaNK-python-watsonx-processor

• https://github.com/thammuio/doc-genius-ai

• https://github.com/tspannhw/FLaNK-python-processors

Models

• https://github.com/ncbi/GeneGPT

• https://www.arxiv.org/abs/2402.03405

• https://huggingface.co/foduucom/stockmarket-pattern-detection-yolov8

• https://github.com/WongKinYiu/yolov9

Tools

• https://github.com/photopea/photopea

https://redash.io/

• https://lookatme.readthedocs.io/en/latest/getting_started.html

Electron apps

• Marp: Markdown presentation ecosystem.

• muryoimpl/slippr: markdown presentation app by Electron

• joe-re/cafe-pitch Markdown-driven presentation tool built on Electron.

Slide deck frameworks supporting Markdown source

• hakimel/reveal.js: The HTML Presentation Framework

• jxnblk/mdx-deck: React MDX-based presentation decks

• gnab/remark: A simple, in-browser, markdown-driven slideshow tool.

• FormidableLabs/spectacle: ReactJS based Presentation Library

• deckjs/deck: Deck is a markdown driven content presentation system

Hosted services

• Slides.com: Online UI for creating RevealJS decks (no markdown support yet, but maybe someday)

• Platon.io: Simple markdown presentations, right in the browser, using remarkjs

• Swipe: Create Interactive Online Presentations On Any Device

• HackMD: The best way to share knowledge in markdown

• Presenta: Make slides fast!

macOS apps

• Slidium: Beautiful and Easy Markdown Presentation - Neomobili

• Deckset: Presentations from Markdown in No Time

• Quiver: Developer notebook, with a presentation mode

• Slideas: Slideas is the easiest way to create a beautiful Markdown Presentation, with all the features you need.

• Obsidian: Obsidian is a powerful knowledge base that works on top of a local folder of plain text Markdown files.

CLI markdown ⇒ slides conversion tools

• Pandoc

• Fusuma: A tool to create slides easily for you

• present: A terminal-based presentation tool with colors and effects.

• backslide: CLI tool for making HTML presentations with Remark.js using Markdown

• patat: (Presentations Atop The ANSI Terminal) is a small tool that allows you to show presentations using only an ANSI terminal. It does not require ncurses

• markdown-slides: Using markdown, write simple but beautiful presentations with math, animations and media, which can be visualized in a web browser even without an internet connection, or exported to PDF.

• S9: Write Your Slides in Plain Text w/ Markdown Formatting Conventions - Free Web Alternative to PowerPoint and Keynote

• Cleaver: 30-second Slideshows for Hackers

view raw markdown-for-slide-decks.md hosted with ❤ by GitHub

https://prql-lang.org/

• https://fonts.google.com/selection

• https://www.kineticedge.io/blog/ktools-kafka-topic-truncate/

https://htmx.org/

• https://deervo.itch.io/diskclick

• https://leanrada.com/htmz/

https://groq.com/

• https://news.mit.edu/2024/tiny-tamper-proof-id-tag-can-authenticate-almost-anything-0218

• https://github.com/awslabs/llrt

• https://observablehq.com/framework/getting-started

• https://academy.datawrapper.de/article/384-how-to-create-small-multiple-line-charts

• https://github.com/enjalot/latent-scope

• https://github.com/IntelSoftware/Python-Loop-Replacement-with-NumPy-and-PyTorch

https://dmarcchecker.app/

• https://github.com/gcarmix/HexWalk

• https://markmap.js.org/repl

• https://github.com/plantuml/plantuml

• https://predibase.com/blog/lora-land-fine-tuned-open-source-llms-that-outperform-gpt-4

• https://github.com/microsoft/UFO

• https://github.com/datadreamer-dev/datadreamer

• https://thealliance.ai/news

• https://engineering.fb.com/2022/03/10/security/code-verify/

• https://www.sciencedaily.com/releases/2024/02/240216135820.htm

https://atuin.sh/

• https://github.com/simulaiofficial/simulai

https://hyperdiv.io/

• https://github.com/OpenMOSS/AnyGPT

• https://github.com/Dashibase/lotion

• https://github.com/microsoft/JARVIS

• https://github.com/ariya/pico-jarvis

• https://github.com/ibis-project/ibis

• https://www.sivalabs.in/langchain4j-ai-services-tutorial/

• https://github.com/weaviate/weaviate-examples/tree/main

• https://github.com/weaviate/weaviate-examples/tree/main/clip-multi-modal-text-image-search

• https://huggingface.co/docs/transformers/model_doc/gptj

• https://github.com/EleutherAI/gpt-neox/

• https://github.com/weaviate-tutorials/DEMO-multimodal-search

• https://github.com/cloudera/CML_llm-hol

• https://github.com/Mozilla-Ocho/llamafile

https://pagescms.org/

• https://github.com/erfanzar/EasyDeL

• https://github.com/bots-garden/pi-genai-stack

• https://spring.io/blog/2024/02/23/spring-ai-0-8-0-released

• https://github.com/Azure/PyRIT

• https://github.com/amithkoujalgi/ollama4j

• https://github.com/dustinblackman/oatmeal

• https://docs.spring.io/spring-ai/reference/api/clients/ollama-chat.html

• https://opensource.expediagroup.com/stream-registry/

• https://github.com/ExpediaGroup/beekeeper

• https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html

• https://github.com/Frimkron/mud-pi

• https://pylint.readthedocs.io/en/latest/pyreverse.html

https://electric-sql.com/

• https://medium.com/hashmapinc/nifi-nar-files-explained-14113f7796fd

• https://github.com/OpenCodeInterpreter/OpenCodeInterpreter

• https://github.com/tstack/lnav

• https://github.com/microsoft/FASTER

• https://github.com/ok-robot/ok-robot

• https://github.com/google/gemma.cpp

• https://github.com/Victormeriqui/Consol3

• https://github.com/chand1012/sq

• https://github.com/mukovnin/psfiles

Notable Tools

Postgresql + MySQL Cache https://github.com/readysettech/readyset

NVIDIA GPU LLM https://github.com/NVIDIA/TensorRT-LLM

Configuration Management Server https://caddyserver.com/features

Fast Text to Image

https://fastsdxl.ai/

Very Interesting Remote tool for OBS

https://vdo.ninja/

Commands Du Jour

docker system prune -a docker image prune -a docker system df docker ps docker logs name

© 2020-2024 Tim Spann