Functional semantics in imperative clothing

https://rtfeldman.com/imperative-clothing

67 points by lwboswell on 2024-04-29 | 69 comments

Automated Summary

In this article, Richard Feldman discusses his experience with porting an I/O-heavy shell script from Bash to Roc, a purely functional programming language. He expresses his initial dissatisfaction with the code's feel when using Roc's standard features for I/O. However, a new operator, the '!' suffix, implemented by a contributor, significantly improved the experience, making purely functional I/O in Roc enjoyable. The '!' suffix works similarly to the 'await' keyword in other languages, desugaring into a call to Task.await and an anonymous function. Feldman emphasizes that the Roc version of the script, despite its imperative appearance, desugars to a big pile of 100% statically type-checked pure functions.

Comments

rmgk on 2024-04-29

I wonder about the core value of these things.

As far as I know, one can create a denotational semantics for any well-defined language. In practical terms, you could pick, say, JavaScript and do a similar representation as explained in the article: Every “non-pure function” is evaluated to a description of the state change that is caused by that function.

What is different for approaches as mentioned in the article is that all denotations (what we represent the result of applying a possibly effectful function) of the formal semantics, are also values in the language itself.

What this gains us is hinted at in the article: We can now implement different ways to execute these effects (i.e., for different execution environments or testing) »as a library« (without modifying the language interpreter/compiler).

However, this flexibility seems to be primarily used by language/framework implementers themselves. Not by application developers. Library implementations of “effect systems” seem to create their own DSL for denotations (i.e., having their own IO type, maybe reusing some closeby syntax from the host language).

For application developers, in case of this article, it seems that the main improvement is that effectful computations are marked with an ! so they are easier to spot. Don’t get me wrong, I think that is very valuable for very little additional syntactic complexity, so it’s a great thing to do.

However, the more I look into this the less I buy into the whole “functional semantics” for side-effects argument. Immutable data is great, explicit dependencies are great, but those are about minimizing the use of side-effects, not about dressing them up.

grayrest on 2024-04-29

> However, this flexibility seems to be primarily used by language/framework implementers themselves. Not by application developers.

In the case of Roc this is determined by the platform and the platform corresponds to the framework level. A webserver platform is going to have a different set of execution choices than a cli platform or a platform for defining user functions in a db. So I wouldn't expect every application developer to make a platform but it's not going to be a compiler/stdlib only sort of thing.

As for the fractured IO, I'm pretty sure that every single person who has dealt with any functional language is aware of the propensity towards fractured IO ecosystems. The exact details of sharing code across different platforms in a useful way is somewhat hand waved at the moment in that there's a rough, unimplemented plan. I kind of expect that there's going to be some piggybacking on the WASM interface definitions but there are a couple changes around Tasks and the module system that are in the pipeline before that'd come up.

Note: I'm mostly an interested bystander

rtfeldman on 2024-04-29

I'm not sure whether you've seen this, but we have settled on a specific design for sharing effectful code across platforms!

https://docs.google.com/document/d/110MwQi7Dpo1Y69ECFXyyvDWz...

As you noted, a few other changes to tasks and modules need to land before we can ship that. There are currently PRs in progress to implement those changes, e.g. https://github.com/roc-lang/roc/pull/6658 and https://github.com/roc-lang/roc/pull/6683

mrkeen on 2024-04-29

> As far as I know, one can create a denotational semantics for any well-defined language. In practical terms, you could pick, say, JavaScript and do a similar representation as explained in the article: Every “non-pure function” is evaluated to a description of the state change that is caused by that function.

Yes this argument has been done before [1], so it's important not to lose sight of the real value proposition:

I want the ability to write a function which returns the same output for the same input, regardless of when/where it's called, number of threads, whether it's in a test or in prod, and so on. But I can already do that in an any programming language, so the real value here is being able to mark the function as such, and have the compiler reject me if I'm wrong.

> However, the more I look into this the less I buy into the whole “functional semantics” for side-effects argument. Immutable data is great, explicit dependencies are great, but those are about minimizing the use of side-effects, not about dressing them up.

A function with minimal side-effects is a function with side-effects [2].

> effectful computations are marked with an ! so they are easier to spot

There's two important steps that you've rolled into one. The first step is the 'functional semantics (with explicit effects)'. E.g. you take your straight-line imperative program:

  html = Http.getUtf8(url);
  path = Path.fromStr(filename);
  File.writeUtf8(path, html);
  Stdout.line ("Wrote HTML to: $(filename)");
and model it monadically:

  Http.getUtf8(url) >>= \html ->
      path = Path.fromStr(filename)
      File.writeUtf8(path, html) >>= \_ ->
          Stdout.line ("Wrote HTML to: $(filename)");
This is where you get your nice functional properties meaning that your compiler won't let you mix them up. Detractors call them red/blue functions. The second step is wrapping it back up into a palatable syntax again.

Haskell and Scala use <- for this. Idris and Roc use !. Js uses await.

[1] http://conal.net/blog/posts/the-c-language-is-purely-functio...

[2] https://queue.acm.org/detail.cfm?id=2611829

rmgk on 2024-04-29

> [1]

Yes, Conal Elliots arguments are very relevant here.

> A function with minimal side-effects

Clarification, what I mean with “minimizing the use of side effects” is not about within a function, but in a program. Exaggerated example: You could write Haskell in a style where everything returns an IO and mutates some global state. Not doing that, and having most things pure is useful due to the simplified reasoning in the pure parts.

> There's two important steps that you've rolled into one.

Have I? What I want to say is that I do believe the syntactic distinction between calling side-effecting functions vs pure functions is useful.

Do I need to care about how the compiler enforces that? Does the specific encoding need to be part of the language semantics? Do I need to be able to model it monadically within the language?

> This is where you get your nice functional properties meaning that your compiler won't let you mix them up.

I may have formulated this a bit flippantly, but I think you mean the same here as I did when I said that “marked with an ! so they are easier to spot” I did have in mind that this is checked by the compiler. And I think we also agree that this separation is a good thing.

Terr_ on 2024-04-30

> A function with minimal side-effects is a function with side-effects [2].

Perhaps the real issue is being able to categorize those side-effects at design-time and control behavior based on category.

In particular, logging and telemetry which (ideally) do not matter to the core behavior of the application, versus web-calls (which probably do) and local memory changes (which almost always matter.)

jerf on 2024-04-29

"However, this flexibility seems to be primarily used by language/framework implementers themselves. Not by application developers."

Yes, but the question is whether that is because it isn't any good for application developers or because application developers are unaware that it is an extremely useful idea, and if done properly, doesn't even carry that much marginal cost.

I work primarily in imperative languages, like most people, but almost all the code I write now I separate into an "IO driver" and relatively pure code that uses the driver as a parameter. Like, everywhere, in every language I use now. The proximal reason is that testability skyrockets when you do this, because what makes tests hard is typically the intermixing of these two concerns. When you isolate them they are almost always vastly easier to test in isolation from each other, and this even contributes to integration level testing higher up.

I say it isn't that expensive because once you get over an initial period of practice and learning how to do this, the difference between (in some random psuedocode)

     func DoUserInput():
         x = userInputLine()
         y = bigPureComputation(x)
         outputResult(y)
and something like

     interface IO:
         inputLine() string
         outputResult(string)
     
     func DoUserInput(IO IO):
         x = IO.inputLine()
         y = bigPureComputation(x)
         IO.outputResult()

     type RealIO:

     RealIO.inputLine = inputLine // the global version here
     RealIO.outputResult = outputResult
really isn't that great; here you see all the overhead front and center but at scale this tends to be less of a big deal on a percentage basis than this would imply, and it pays off immediately in testability... yes, what I wrote implies that bigPureComputation must be easy to test, but now I can provide drivers that allow me to essentially push that purity up into DoUserInput and test it as if it were a pure function, because when I combine that with a testing implementation of IO the whole DoUserInput+TestIO essentially does become externally pure, even if you can look on the inside and see things like input being consumed and such. Testing RealIO is also easier because now it's all broken into its component pieces, and of course at real code scale all of these little snippets can expand into arbitrarily complicated and otherwise difficult-to-test functionality.

In my toolbelt, this technique is basic, fundamental, and desperately underutilized by application developers. It is sooo easy to write code up front in this style and such an amazing pain to refactor into it after the fact. Doable, absolutely, but an amazing pain.

rmgk on 2024-04-29

First of, I do like that style.

In the context of my argument its quite interesting because you essentially argue that one can get the benefits of separate runtime implementations without actually needing pure functions (in a very strict sense) or any of the IO encodings.

What I had in mind when I said application specific handling, I meant things like the application developer writing a custom optimization pass on the returned IO values to make the execution more efficient for the specific application. Or adding custom way of handling/reporting errors. And I meant doing so while not making their own interfaces and instead working with whatever the common IO abstraction is.

o_nate on 2024-05-01

I agree this is a great pattern and one I try to use in all my code. I believe it is often known as the "Functional Core, Imperative Shell" pattern.

taeric on 2024-04-29

This is an interesting analysis, in many ways. Take it further down the stack and see that functional semantics are always executed by imperative machines. Such that we have systems of imperative machines that we build functional abstractions on top of, so that we can have readable imperative code for the users. :D (An amusement being that both the machine and the people prefer imperative, it seems.)

mrkeen on 2024-04-29

> Take it further down the stack and see that functional semantics are always executed by imperative machines

And the other way around too. Even your C compiler can't reason about mutability, so it un-imperatives everything first:

  int foo(int i, int j) {
    while (i < 10) {
      j += i;
      i++;
    }
    return j;
  }
That (j += i) line becomes:

  %9 = load i32, ptr %3, align 4, !dbg !23                  // read i into %9
  %10 = load i32, ptr %4, align 4, !dbg !25                 // read j into %10
  %11 = add nsw i32 %10, %9, !dbg !25                       // sum %9 and %10 into %11
(i++;) becomes:

  %12 = load i32, ptr %3, align 4, !dbg !26           // read i into %12
  %13 = add nsw i32 %12, 1, !dbg !26                  // store i+1 into %13

Kamq on 2024-05-02

I'll admit to not being super great at getting down into the nitty gritty with internals of compilers. Are you talking about some sort of IR? Because if we're talking about the output, mutation definitely happens.

Checking on godbolt, there's definitely mutation at least sometimes. Using gcc 13.2, I get the following output.

j += i; becomes:

    movl    -4(%rbp), %eax     // move i into %eax
    addl    %eax, -8(%rbp)     // add i to j
With i++ becoming:

    addl    $1, -4(%rbp).      // add 1 to i
If your example is IR, do you happen to know how it crunches everything "back down" to mutability? Does it just check that the previous value of i and j were never used and use the same registers as an optimization?

taeric on 2024-04-29

I would consider the compiler to be working in the functional realm, for most of its ideas. Such that I would think it is between the imperative of the machine and the imperative of what a person is thinking?

That said, I confess I'm curious how on what the constraints are on this one. I thought there was an increment instruction, specifically. I'm guessing safety and general debugging concerns? And it occurs to me that you are showing an intermediate language in this? Static Single-Assignment (SSA) is definitely in the vein of moving imperative to functional for easy manipulation there.

Definitely fun topics to explore here.

amw-zero on 2024-04-30

I've come to the conclusion that executability and reasoning are separate axes. If we view everything in terms of languages with interpreters, of course functional semantics must be executed by imperative machines. But to reason about either kind of program, we need functional semantics, because functional programming is the language of pure reasoning.

More on the subject: https://concerningquality.com/quality-and-paradigm/

taeric on 2024-04-30

But folks have reasoned about imperative things for a long time, as well? It is certainly easier for many analytics to get into a functional form. But would be a stretch to say it is always easier. Fractals are often my go to example where imperative is easier for many constructs.

I view it like coordinate systems. Polar versus Cartesian coordinates can clearly both do the same things. They each have domains where they have advantages.

You could argue that time based logic systems are the end goal, maybe? Still feels like throwing babies in bathwater.

Edit: for a fun example of reasoning without programming "functional" logic, look up juggling diagrams/notation. Not all logic is symbolic in the same way.

amw-zero on 2024-04-30

This doesn't feel in contrast to what I was saying. You can reason about imperative code, but you have to add things to it to reason about it, most importantly the notion of a memory store. This is because imperative code assumes an underlying CPU + RAM which holds state.

It's not that it's easier, it's that imperative code has implicit state.

taeric on 2024-04-30

Fair that I don't intend my question there as a full rebuttal. I do question if functional semantics are fully needed any more so than other tools. In particular, I recall formal methods was more about prepositional logic than it was functional semantics.

And yes, you have to add and remove things to any model to really be able to reason about it. Is why engineering dynamics is harder than engineering statics. Things that change are hard. Reducing this to state being the enemy, though, I think is wrong in a lot of functional advocacy. State can be reasoned about without having to boil the ocean, as it were.

gumby on 2024-04-29

> There's an old joke about programming with pure functions: “Eventually you have to do some effects. Otherwise you're just heating up the CPU.”

I prefer to reframe the old sexist joke: "Side effects: can't live with them, can't live without them".

fifticon on 2024-04-29

For a few precious first seconds when I parsed the title, I was contemplating a style of clothing called 'Imperative Clothing', and why the latest fashion for those was to add a dash of 'Functional Semantics' to that clothings line. Then the boring proper way to parse it popped into my head, and the fun was over :-/

Akronymus on 2024-04-29

This looks a LOT like computation expressions in f#, specifically the async one. https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref...

Does Roc have something similar?

grayrest on 2024-04-30

Roc does not have computation expressions and this specific syntax is the "something similar". Many other comments have noted that this feature corresponds to the IO Monad with do notation in Haskell and Computation Expressions are F#'s take on monads & friends. Roc's take on this is basically "why should someone need to know category theory to write to a file" and the relatively simple "use ! when you want to yield control to the platform" suffices until the programmer decides to dig into a full explanation.

More generally Roc is following in the footsteps of Elm in that it's an exercise in functional language minimalism. I personally like computation expressions but I think they're beyond Roc's preferred feature complexity limit. There are proposals to expand this feature either using a `?` in a `with` section that would allow for Rust-like error handling with `Result`s or a type-dependent version of `!` so the syntax could be used on something other than `Task`s. Regardless, it's expected that the IO use case is going to be the primary use case regardless so I expect they're giving people some time to play around with it and provide feedback before making further changes.

Akronymus on 2024-04-30

> "use ! when you want to yield control to the platform"

Thats quite the elegant solution indeed.

and I can certainly see the benefits to keeping it simple

bradrn on 2024-04-29

I find it amusing that the system described in this article is simply what Haskell calls the ‘IO monad’. That is to say, it describes an IO system with the following ingredients:

1. Each IO operation returns a special value which denotes an IO action

2. These IO actions can be sequenced using a function which feeds an IO action to a continuation

3. Syntax sugar to remove the annoyance of explicitly writing the sequencing function

Together, (1) and (2) make this a monad; (3) in this article is just ‘do’-notation. The only major difference from Haskell is that Roc doesn’t generalise the idea to other monads, which makes it far less powerful than Haskell’s Monad typeclass.

garrinm on 2024-04-29

This gets discussed in the Roc community. They are exploring designing a language without higher kinded polymorphism.

Here's a snippet from the Roc FAQ.

> It's impossible for a programming language to be neutral on this. If the language doesn't support HKP, nobody can implement a Monad typeclass (or equivalent) in any way that can be expected to catch on. Advocacy to add HKP to the language will inevitably follow. If the language does support HKP, one or more alternate standard libraries built around monads will inevitably follow, along with corresponding cultural changes. (See Scala for example.) Culturally, to support HKP is to take a side, and to decline to support it is also to take a side.

https://www.roc-lang.org/faq.html#higher-kinded-polymorphism

Cyrus_T on 2024-04-29

I feel like adding HKP isn't "taking a side" nearly as much as not adding HKP to turn people away because you don't like their cultural contribution.

By adding the feature you aren't actually excluding the people who don't want to use it. They are free to avoid it and maintain parallel ecosystems where HKP is avoided.

You are only excluding them if the standard library forces them to interact with HKP code, which arguably Haskell does, and it makes a PITA for teaching it. Haskell however isn't doing it with the intention of exclusion.

On the other hand, not adding the feature because you don't like the culture it brings is intentionally exclusionary. You're intentionally taking a side against certain kinds of people and the certain kinds of code they want to be able to express without repetition... and you're hamstringing the expressibility of your language to do it.

I completely understand wanting to avoid HKP for teaching purposes, but maybe that is a problem that could better be resolved by always providing monomorphic options for standard library functions....

But avoiding HKP to avoid a certain kind of culture? Well now it's personal!

rtfeldman on 2024-04-29

As the FAQ entry notes, one of the "sides" in the design space here is "a fragmented ecosystem is best because there's no other way to accommodate the variety of different styles people want to use" - this is a totally reasonable preference to have!

Still, hopefully it's clear why a language designer might want to go down a different path. After all, it's also totally reasonable to prefer an ecosystem which isn't fragmented along these lines (the way it is in, say, Scala) even though it means fewer different programming styles are supported.

tkz1312 on 2024-04-29

Roc has algebraic effects instead of Monads for effect tracking. The two systems are exactly as expressive as each other [1]. Algebraic effects compose better than Monads, and are probably more beginner friendly. Imo Monadic code can be a little easier to follow sometimes since there’s less non-local control flow. They are both great things to have in a language and I’m super excited to see how much progress Roc is making in both approachability and performance when it comes to pure functional programming.

[1]: https://homepages.inf.ed.ac.uk/slindley/papers/effmondel-jfp...

rtfeldman on 2024-04-29

Roc actually doesn't use algebraic effects under the hood, although errors can naturally accumulate in a way that makes the code look more like error handling in algebraic effects systems:

https://www.roc-lang.org/tutorial#task-failure

The implementation detail that makes this error handling possible is that Roc uses anonymous sum types called "tags" (OCaml calls them polymorphic variants) that can accumulate on the fly:

https://www.roc-lang.org/tutorial#tags

tkz1312 on 2024-04-29

Very interesting, thanks for the clarification. What is the difference between algebraic effects and rocs approach?

rtfeldman on 2024-04-29

In Haskell terms, Roc's `Task ok err` is essentially an `IO (Either err ok)` - the only difference between how Roc does it and how Haskell does it is that `Task` bakes in error handling whereas `IO` doesn't. (Plus the syntax sugar being different, of course.)

So the biggest difference is that algebraic effects are a separate language feature, whereas Task (like IO in Haskell) is a plain old type in the standard library that happens to represent effeects.

Incidentally, the Task.await function in Roc is based on Elm's Task.andThen - we just tried out a different name and argument order for it:

https://package.elm-lang.org/packages/elm/core/latest/Task#a...

bedobi on 2024-04-29

can someone explain in plain English and with some easy to understand code examples?

ngruhn on 2024-04-29

But do algebraic effects preserve referential transparency? It’s hard for me to see that they do.

posix_monad on 2024-04-29

IIUIR, Roc's syntax is more ergonomic than do-notation.

In Roc, you can put a "!" (bind) pretty much anywhere, whereas in do-notation it must be part of a binding.

    current_user <- fetch_user
    manager <- fetch_manager current_user
Compared to

    manager <- fetch_manager (!fetch_user)
Please correct me if I'm wrong!

nyssos on 2024-04-30

You don't need to bind everything. The corresponding Haskell code is

    manager <- fetch_manager =<< fetch_user

posix_monad on 2024-04-30

Why have do-notation if we can use fancy operators for everything?

I think this version would is less approachable for most developers.

skulk on 2024-04-29

one advantage of this over do-notation is brought up in the article (though not explicitly): you can bind multiple things in a single expression. In Haskell, you'd have to do

    do x <- getx; y <- gety; x+y
In roc you can do

    getx! + gety!
which looks a bit cleaner.

bradrn on 2024-04-29

This is really just a matter of how much syntax sugar you want to implement. Idris already has precisely this syntax, and there’s a proposal to add it to Haskell too [0]. But none of this changes the core properties of the system which make it monadic.

[0] https://github.com/ghc-proposals/ghc-proposals/issues/527

rtfeldman on 2024-04-29

TIL that Idris also has a very similar `!` operator (in the prefix position instead of suffix) - thank you for sharing!

alipang on 2024-04-29

Even Javascript has a similar syntax, though `yield` / `await` only works for certain "monads" (promises seem close enough)

mrkeen on 2024-04-29

  (+) <$> getx <*> gety

superlopuh on 2024-04-29

Definitely doesn't look as clean as the example above, or as clean as the Swift equivalent of

    await x + y

mrkeen on 2024-04-30

The same syntax works across a range of constructs in Haskell, Scala and Idris.

  (+) <$> getx <*> gety
This could be:

* summing two reads in a Transaction,

* summing two parsed numbers in a Parser,

* summing two Nullable values into one Nullable value,

* producing a Validated<T> by validating two of its parts which require validation,

* and on and on.

It's even the cartesian product when run on lists:

  let suits = ["♠", "♥", "♦", "♣"]
  let nums  = concat [["A"], map show [2..10], ["J","Q","K"]]
  let cards = (++) <$> nums <*> suits

  putStrLn (unwords (take 15 cards))
  A♠ A♥ A♦ A♣ 2♠ 2♥ 2♦ 2♣ 3♠ 3♥ 3♦ 3♣ 4♠ 4♥ 4♦
Swift, Roc, Rust and JS have specialised their syntax for just one of these concerns.

Also, it looks like you can just do it with ! in Haskell: https://hackage.haskell.org/package/monadic-bang

tome on 2024-04-30

> it looks like you can just do it with ! in Haskell

Yes, although it's slightly clumsy.

    example = do
      let suits = ["♠", "♥", "♦", "♣"]
      let nums  = concat [["A"], map show [2..10], ["J","Q","K"]]
      let cards = do pure (!nums ++ !suits)
      putStrLn (unwords (take 15 cards))

    ghci> example 
    A♠ A♥ A♦ A♣ 2♠ 2♥ 2♦ 2♣ 3♠ 3♥ 3♦ 3♣ 4♠ 4♥ 4♦

jimbokun on 2024-04-29

The Roc version is still cleaner.

C3POXTC on 2024-04-29

This is such a good explanation. It's incredible what you can do with such a simple concept.

continuational on 2024-04-29

There doesn't seem to be much about the limitations of this approach, e.g. will this work?

    list.map (lambda f => readFile! f)

still_grokking on 2024-04-29

It would be even more "funny" if you'd wrap that `readFile` inside a thunk that gets captured from the environment by the lambda body. Than you'd have a reference to an effect that is supposed to run only inside the lambda but could be executed by external code. Depending on the effect this can have unexpected up to catastrophic outcomes.

That's why Scala introduces so called "capture checking" to make direct style effect systems safe.

https://docs.scala-lang.org/scala3/reference/experimental/cc...

To understand the problem better see:

https://www.inner-product.com/posts/direct-style-effects/#ca...

sixbrx on 2024-04-30

I'm pretty sure the thunk would be tagged by the effects of the inner function, so couldn't be called from an external context not declaring those effects. That's what the effect tracking is for.

mrkeen on 2024-04-29

> It would be even more "funny" if you'd wrap that `readFile` inside a thunk that gets captured from the environment by the lambda body. Than you'd have a reference to an effect that is supposed to run only inside the lambda but could be executed by external code. Depending on the effect this can have unexpected up to catastrophic outcomes.

Well... yeah? It looks like readFile (a Task) is sufficiently thunk-like to guess that that's the case.

I program in this still regularly. What bad outcomes should I expect?

mrkeen on 2024-04-29

Probably! (I've never touched Roc but I've done a bunch of FP).

I imagine if you did your function, it would return a list of unexecuted Tasks.

Just by guessing, the way to do it (actually read the files) would probably look like:

  list.mapTask (lambda f => readFile! f)
or

  Tasks.map list (lambda f => readFile! f)

rtfeldman on 2024-04-29

That's correct - if you wanted to actually run all of them, you could use Task.seq (to run them sequentially, as opposed to concurrently) like so:

    Task.seq \f -> File.read! f
https://www.roc-lang.org/packages/basic-cli/Task#seq

still_grokking on 2024-04-29

Where is the list of files in that example?

I also don't get how running

  list.map (lambda f => readFile! f)
would result in a list of Tasks. `readFile` needs already to return a Task as it's an I/O operation, right? Appending the exclamation mark will CPS transform that Task value by wrapping it into (possibly nested) calls to `Task.await`, as I understand the article. But CPS inside lambdas is problematic as you need to also CPS transform the lambda, more or less rewriting your whole program "inside out" (all up the whole call stack!). Also every higher order function would need to be there in two variants. HOFs need to get "shifted" manually for that. That doesn't scale.

I mean, it's not impossible to implement something like that. But it's not trivial. The problem with the HOFs remains still. See for example CPS-async in Scala:

https://github.com/rssh/dotty-cps-async

rtfeldman on 2024-04-29

> Appending the exclamation mark will CPS transform that Task value by wrapping it into (possibly nested) calls to `Task.await`, as I understand the article.

The article briefly mentioned this edge case:

> Since this is the last task in a chain, the ! doesn't do anything and isn't necessary…so we just drop it during desugaring instead of giving a compiler error or generating an unnecessary Task.await. This allows for a more consistent visual style, where async I/O operations always end in the ! suffix, but doesn't have any runtime cost.

That's why it would return a list of Tasks.

still_grokking on 2024-04-29

So the semantics of this program would change if I added a line containing an exclamation mark somewhere after this expression? That smells like the worst kind of action at a distance…

Also, how is the expression `doSomething!` typed then? Does it have different types depending on where it's written?

And how do I actually write something `traverse`-like? How to actually write

  list.forEach (lambda f => readFile! f)
than, if I really want the effect to run inside the lambda body?

mrkeen on 2024-04-29

> I also don't get how running ... would result in a list of Tasks

That's the contract of 'map' in any FP. Turn a container of A into a container of B, and do nothing else (in this case - no telling tasks to start executing).

> Appending the exclamation mark will CPS transform that Task value ... is problematic as you need to also CPS transform the lambda

You may be overthinking it. I believe (!) is a syntax-only transformation which happens very early in the compiler pipeline. And I don't think you need to invoke CPS here:

  CPS From [https://matt.might.net/articles/cps-conversion/]
  The M function only has to watch for lambda terms. When it sees a lambda term, it adds a fresh continuation parameter, $k, and then transforms the body of the lambda term into continuation passing style, asking it to invoke $k on the result. Variables are unchanged
In Roc (and Haskell, Scala, Idris, ...) you don't need to generate a fresh $k under the hood, because it's already right there in the surface language (in this case, foo):

  foo = mytask! "abc"
is already equivalent to:

  mytask "abc" (\foo -> ... )
> See for example CPS-async in Scala:

Why didn't you mention Scala earlier :D. This is the same trick as Scala for-comprehensions. Here's an example in that repo: https://github.com/rssh/dotty-cps-async/blob/36f2e3b61542b39...

> Also every higher order function would need to be there in two variants.

Literally every HOF? Nah, but you're right that there is duplication - and there needs to be, since there is a distinction between:

  convert a *list of filenames* (input) into a *list of Tasks* (output) which will read those filenames

  convert a *list of filenames* into Tasks which will read those filenames, and return their *contents* (output)
If you have List and Task modules, then "do this List of Tasks" obviously has to live in List's source code or Task's source code. It's an N*M implementations problem, but fuck it. Computers do Lists of Tasks so it's worth implementing.

Haskell does a little better non the non-duplication front. By going down the typeclass/monad route, you can put the "do this List of Monads" in List.hs, and "Task is a Monad" in Task.hs, and you're all set. But it still makes a distinction between 'map the list' and 'map the list and return the results of running the monads inside it'. It calls them map and mapM respectively, and also has mapM_ which carries out the effects but also ignores the output.

Anyway, back to my Java day job where the duplication issue in the stdlib is solved by just ... leaving out functionality. Let's see how we tackle the List of Tasks issue: https://stackoverflow.com/questions/30025428/convert-from-li...

noelwelsh on 2024-04-29

Effect systems seem to be popping up everywhere at the moment, though it's not clear the author realizes this is what they are implementing (there are no references to effect system, effect handler, or algebraic effect in the post.)

rtfeldman on 2024-04-29

Author here - I actually gave a talk about effect systems and Roc awhile back: https://youtu.be/7SidSvJcPd0?si=iwyguLUWIX3FDq9f

I thought this particular post already introduced enough terminology without opening up that particular topic. :)

C3POXTC on 2024-04-29

I think he's pretty aware of effect systems. He talks about them in his podcast, for example. Roc, as a descendant of Elm, is trying to avoid using too much jargon and explain what you can do with it, rather than scaring beginners away with technical jargon.

kwhitefoot on 2024-04-29

This is a bit off topic I suppose but I'm wondering about lines like this one:

    ls | grep "roc_nightly.*tar.gz" | xargs rm
Isn't this just an over complicated way of saying:

    rm roc_nightly*tar.gz

fifticon on 2024-04-29

If you think of the shell as a REPL and an iterative environment, the first variant has many benefits. The first two parts, might be an attempt to see 'can I write a regex to remove the parts we need to get rid of?. I had better run it first, to see it catches the ones I intended. Oops two times in a row, that one didn't catch them all, and that one would include some of those we MUST keep'.

I often do the same with SELECT/DELETE on production databases, so I can visually inspect what I'm about to destroy. Transactions don't help me, because if I haven't verified that I'm correct, I won't know that I should use ROLLBACK.

AnimalMuppet on 2024-04-29

OK, but you could still do

  ls roc_nightly*tar.gz
and then replace "ls" with "rm" after you're sure that it gets the right files. (This is very much like your SELECT/DELETE, in fact.)

williamdclt on 2024-04-29

I do that all the time in interactive shells. First because it’s easy to bring back the last command and append to it, second because it gives me more confidence about what I’m actually deleting (I could mess up my rm command if I’m writing it from scratch).

With a couple of zsh aliases that becomes quick to write in a stupid-but-simple way: “ls G roc_nightly G tar.gz X rm”

layer8 on 2024-04-29

There can be limits on the size of the command line (ARG_MAX). The pipe formulation circumvents such a limit. See https://en.wikipedia.org/wiki/Xargs#Examples.

SonOfLilit on 2024-04-29

It's a much more generic way of saying it, e.g. you can't use the other way for

    ls | grep -P "roc_(nightly|weekly).*\.tar\.gz" | xargs rm
(forgive me if I got grep -P regex syntax wrong)

pyrale on 2024-04-29

> “Eventually you have to do some effects. Otherwise you're just heating up the CPU.”

Heating up a CPU is side-effectful enough to crack encryption [1]!

I for one stick to executing my purely functional code in my mind only.

[1]: https://www.researchgate.net/publication/263314485_The_Tempe...

cryptonector on 2024-04-29

  >  What about the "all pure functions" part? By definition, pure functions don't have side effects, right? (A side effect is when a function changes some state outside the function itself, like a global variable or the file system.) So…how can these functions be pure if all this I/O is happening?
  
  > It's surprisingly simple:
  
  >     Each function returns a value describing what I/O it wants done.
  >     The compiled program has a runtime which looks at those values and actually performs the I/O they describe.
So... like the IO monad.

bradrn on 2024-04-29
cryptonector on 2024-04-29

Yes, sorry I didn't read through all the comments.

hoseja on 2024-04-30

Why the hell was I expecting an article about fashion.