Shipping pre-alpha builds!

It's a huge milestone just to be shipping cross-platform builds on a brand-new tech stack - pretty amazing that everything came together. The "last mile" of getting to this point can be a grind!

In the past 6 months, we:

• Built an entirely new native application framework - Revery 
• Forked Vim to provide a synchronous integration model - Libvim 
• Shipped a native ReasonML app

However, there's a saying: if you're not embarrassed by what you've shipped - you've shipped too late! By that metric... we certainly haven't shipped too late :) There's no question that the builds are at a pre-alpha quality level.

There are a lot of basic features missing, like:

• Font configuration
• Word wrap
• Basic error messages / dialogs (ie - "You have unsaved changes!")
• Mouse integration

And then some larger things - like, today, we don't have a language integration story (integrating VSCode extensions will be the focus of the next milestone, and Vim extensions after that...) 

This combination of gaps means that we're still not at 'daily-editor' level... yet.

However, we'll keep working through these issues in parallel with the feature work. Every day, the build should be getting more usable and more feature-rich.

Shipping

Shipping builds represents us overcoming the first major unknown in delivering a product - can we ship? Ship on a brand new tech stack? The answer is yes, and we have our pipeline set up to publish nightly builds, so that users who pre-ordered can always have access to the builds (and later - auto-update). These builds can be accessed via our Early Access Portal 

Shipping on each platform had some unique challenges, though - this one case where I missed Electron.... as having a package like electron-builder to get do the heavy-lifting of creating application packages was convenient.  

Windows was actually the easiest. The binary is statically linked and I have a lot of experience with InnoSetup (and Ryan helped a ton - setting up file associations and other improvements!). Only additional piece was hooking up code-signing which wasn't too bad. There are a few bugs we need to iron out with our installer; and even though we have code-signing hooked up in our publish pipeline, SmartScreen can be a pain - it'll take some time / installs for our builds to not trigger a SmartScreen dialog. We're tracking this in #509.

OSX had some tricky aspects, as there are several `dyllib`s we depend on (ie, FreeType2/Harfbuzz - from esy's build). The built executable has hardcoded paths to the build location's dyllibs. However, these can be remapped using `install_name_tool` to provide rpath's relative to the executable location. It turns out there is a great tool called macdyllibbundler that automates this, and we created an esy package for it (esy-macddylibbundler).

Not only that, but code-signing wasn't enough to make GateKeeper happy... we also had to hook up notarization to our pipeline. We hit several bumps in the way, and unfortunately we still have work left to unblock GateKeeper - we're tracking that in #510.

Linux was also tricky - we wanted an install format that you could download and run. AppImage seemed like a great fit for this. We also had similiar issues to OSX, in that the executable would contain hardcoded paths to libraries it needs (FreeType2 and Harfbuzz, but also lots of others like GTK...). The linuxdeploy tool took care of remapping RPATHs for us in this case, and with that, bundling the AppImage was not too bad.

We have code-signing hooked up for OSX and Windows, but nothing for Linux at the moment - does anyone have suggestions on best-practices for that? I'm also trying to figure out, when we get to the point of having auto-update within the app, what the best strategy for Linux is.

Another thing we were missing is documentation... so I've started some basic docs here. Like the build, these are also 'pre-alpha' quality - this is something we'll have to keep investing in.

Performance

Performance was the primary reason for switching to our new technology stack - we wanted something fast and native.

The initial feedback we have so far regarding performance has been great (from our Discord channel):

> "The main application shell starts up very fast on OSX!"

> "It's much faster compared to Oni 1 in my experience."

> "IT'S LEGITIMATELY FASTER THAN VIM IN A TERMINAL"

> "Tinkering around in Oni after working with the Vim Plugin in VSCode all day, this just feels like the character appears on the screen before I even press the key"

So it's awesome to hear that and see progress on that front. It's a night and day difference from Onivim 1, IMO.

That being said - there is still a lot of room for improvement. 

Specifically:

• Syntax highlighting strategy is very slow (flash-of-unstyled-content, etc)
• Rendering strategy is sub-optimal (too many draw calls!)
• Parallelize some startup tasks (ie, the data population for the file explorer blocks startup rendering)

Our idle CPU usage (when the user is not typing anything) can get a bit high in some circumstances - I have a fix for this incoming but won't make it in the pre-alpha (should be in soon though).

From there, we still have a lot of 'low-hanging' fruit - for example, OCaml has an optimizing compiler called 'flambda' - we should be using that for our release builds. We haven't used the full set of optimizing flags for gcc as well. Still lots to be done!

Quality

The last post - Update #3: A re-architecture - was really about quality (tracking down an intermittent crash, etc) - leading up to us deciding to create libvim . That migration has gone pretty well, in that most core functionality is working (as someone put it - we're basically Vi-compatible right now). However, there are still some cases where we encounter blocking input paths like:

• Paste into command window freezes program 
• Crash when pressing A or I in Visual Mode 

We consider any of these crashes / hangs as daily editor blockers. If you find one, please log it!

The good news (the difference between our situation before) is that these crashes / hangs tend to be deterministic - which is a much better place to be... deterministic bugs are like candy for developers.

Besides crashes, there are still cases where our libvim library can get 'out-of-sync' with the editor - we're working through those issues to improve reliability.

Next Steps

August marks the entry into our next milestone - VSCode extension host integration.

Specific work will be:

• Native syntax highlighting
• Extension discovery, installation, and uninstallation (Extension Management)
• Integration of basic LSP functionality
  - Buffer Updates
  - Diagnostics
  - Hover
  - Completion
  - Formatting Provider
  - Snippets
  - Codelens
• Out-of-box support for ReasonML / OCaml
• Integration of SCM (Version Control) functionality
• Integration of Debug Adapter Protocol (debugging)
• Tasks + Problem Matchers

Native syntax highlighting is top-of-mind for me right now. I'm strongly considering using TreeSitter, but I'll have more on that later as I dive deeper.

In this milestone, there will also be some non-technical issues to address.  In particular, many VSCode extensions are liberally licensed via MIT, but some aren't, and some specifically require a VSCode installation. One of the most requested plugins - the remote development plugin - has one of the most restrictive licenses. Of course, I do not want to put any Onivim users in the position of violating EULAs, so we'll be working through that to see what we can do and have clear guidance.

In parallel, I expect there will be an increase in incoming bugs since more people will have their hands on the builds. So we'll be working through daily editor blockers, too....

...it's going to be a busy few months ;)

You can help us by 'thumbs-upping' issues that are relevant to you, or logging issues that haven't been posted yet. This helps us prioritize and make sure we're focusing on the right things!

Pricing

Overcoming this first unknown - of shipping - is  a big deal! We've now raised the pre-order price to a $19 lifetime license or a $2 monthly license (via Patreon). As we continue to deliver on milestones - we'll keep bumping up the price, until we get to our final pricing model. So if Onivim 2 is interesting to you...  there is  never going to be a better time to pre-order & back than now!

I've snapshotted anyone who has contributed on Patreon up to this point to deliver the lifetime license key. Those will be available to view in the early access portal eventually (before they are needed in the editor).

Special Thanks

- @CrossR for all his help bug fixing and prepping for the release

- @Manas and @manuhornung for their work on Skia

- @tcoopman, @sysvsto , @tatchi, @r281GQ, and @SeraphRoy for their PRs

- @szbergeron, @Apos, @raguay, @mitch, @jordwalke, @manuhornung for testing the builds and helping us find / catch issues before sending the builds out more broadly.

And THANK YOU to all the patrons, backers, pre-orderers - we wouldn't have made it this far without your support. Shipping is a huge milestone and there is a lot of work ahead... but I believe we are well on our way to delivering a fast, modern modal editor that combines the best of Vim, VSCode, and Sublime Text.

...after all this release work (packing, documentation, website, etc), it'll feel great to start working and coding toward this next milestone! Being able to integrate and leverage VSCode extensions is one of the key value propositions of Onivim 2 (and selfishly, will be useful for my workflow), so I can't wait to have that in place.