How can one use this for logging? Won't the LLM performance lag grind your application to a halt?

They're using MITIE[1] models for NER. And MITIE models aren't LLMs. They're way lighter: word embeddings with SVM classifiers.

[1]: https://github.com/mit-nlp/MITIE

about 18 months ago, our non-Google / Bing bot traffic went from single digits per cent to over 99.9% bot traffic. We tried some home-spun solutions at first, but eventually threw in the towel and put Cloudflare in front of all our publicly accessible pages. On a long term basis, this was probably the right move for us, but we felt forced into this. And the Cloudflare Managed Ruleset definitely blocks some legit traffic such that it requires a fair amount of manual tuning.

An order of magnitude less scrutiny, but also an order of magnitude in size of investor base. The private markets trade at Palantir levels so why go public. Also the private markets are now routinely doing secondary transactions so even less reason to go public.

> toy for media consumption, not a serious work device.

But this is still largely true.

If you have serious work to do, do you stay on your phone or do you open your laptop?

I think they where talking about phones with keyboard vs modern smartphones.

They're selling shares at a $500B valuation. The market is telling them everything they are doing is amazing.

Is it possible to differentiate the feedback of the initial success of chatgpt from whatever came after it?

It's possible those investments are just the oai owners selling their 2023 chatgpt success and its profit share.

The distance between OpenAI and its competitors has been shrinking for years (and across some metrics is now negative), yet the price keeps going on up. And I cannot figure out Thrive. Weren't they an early investor, yet they keep chasing the price higher?

It's not the tech that matters, it's the amount of users. See snapchat vs instagram.

Also, the business models are incentivized towards efficient token usage.

I dunno. Depending on the writer and their particularly axe to grind the definition can vary widely. I would like it to mean, "any fixes I needed to make were minimal and not time intensive."

It's more of "yeah it worked, but I had to do a lot of hand-holding" and "it passes the tests but I cannot tell if the code has memory leaks".

Actually, I can tell; I ran split on the C source and got things like this:

disk_space.c:144:16: Only storage bin.ref_bin (type void *) derived from variable declared in this scope is not released (memory leak)

So I'm looking into a Rust version with Rustler now.

I'd also recommend having a look at the Zig library for Elixir. For simple stuff, you can just inline, but support for pretty complex setups too.

Also comes with a built-in BEAM allocator so the runtime properly reports NIF memory consumption.

https://hexdocs.pm/zigler/Zig.html

Thanks, one day I'll look into Zig too, because I've been impressed with what Tigerbeetle has achieved with it! For now, I'm all-in on Elixir.

you can use zigler for a c nif, using easy_c (or c_src) options.

the big advantage is that it will automatically box/unbox to/from c values for you and generate sane error messages (which rustler does not, last i checked) when you pass incompatible terms in to the function.

on the other hand rustler lets you precompile (which is coming in a future version of Zigler)

Perhaps the poster we referring to Simon not Gemma.

> All LLMs should be treated as potentially compromised and handled accordingly.

There are no agentic tools if one follows this proviso.

I’ve been doing all my claude coding on a hetzner, if it breaks out of that and into the other vms, or somehow crawls back through the ssh connection into my machine, then I guess I would have a problem.