dotemacs

CONTRIBUTING.md (12182B)

---

 # The SLY Hacker's Handbook
 
 ## Reporting bugs
 
 The most important thing when reporting bugs is making sure that the
 developer has a way to reproduce it. To do this, he needs to rule out
 interference from external factors like other Emacs extensions or
 other Lisp-side code. Here's a great example of a bug report
 
 ```
 $ emacs --version
 Emacs 24.3
 $ sbcl --version
 SBCL 1.2.1
 $ cd sly
 sly $ emacs -Q -L . -l sly-autoloads --eval '(setq inferior-lisp-program "sbcl")' -f sly
 
 I get the REPL but when I try to X, I get Y
 OR
 I don't get the REPL at all because frankinbogen!
 ```
 
 
 ## Coding style
 
 This section is very empty, in the meantime try to be sensible and
 emulate or improve on SLY's existing style.
 
 ### Commit messages
 
 ChangeLog files are gone! However, the syntax of ChangeLogs is very
 useful to everybody and Emacs supports it perfectly:
 
 * in Emacs, for every snippet that you've changed, type `C-x 4 a` (or
   `add-change-log-entry-other-window`)
 
 * Emacs will open up a ChangeLog buffer, but this is just a dummy
   buffer that you can ignore. However, the content inside it should be
   pasted (sans indentation) to the commit message.
 
 * As an added bonus, if you are using Emacs >= 24.4 and `vc-dir` to
   prepare your commits, Emacs does that for you automatically.
 
 The benefits of this format are great. One can still use `M-x
 vc-print-log` in a source file and browse through its ChangeLog
 without the hassle of ChangeLog conflicts.
 
 ### General philosophy
 
 I keep a sentence of the previous Coding Guide that I like very much.
 
 > Remember that to rewrite a program better is the sincerest form of
 > code appreciation. When you can see a way to rewrite a part of SLY
 > better, please do so!
 
 
 ## Lisp code file structure
 
 The code is organized into these files:
 
 * `slynk/slynk-backend.lisp`: Definition of the interface to non-portable
 features.  Stand-alone.
 
 * `slynk/backend/slynk-<cmucl|...>.lisp`: Back-end implementation
 for a specific Common Lisp system.  Uses slynk-backend.lisp.
 
 * `slynk/slynk.lisp`: The top-level server program, built from the other
 components.  Uses `slynk-backend.lisp` as an interface to the actual
 backends.
 
 * `sly.el`: The Emacs front-end that the user actually interacts
 with and that connects to the Slynk server to send expressions to, and
 retrieve information from the running Common Lisp system.
 
 * `contrib/sly-<extension>.el`: Elisp code for SLY extensions.
 
 * `contrib/slynk-<extension>.lisp`: Supporting Common Lisp related
 code for a particular extension.
 
 
 ## SLY-Slynk RPC protocol
 
 The info in this section would be something for a future "Slynk
 Programmer's Guide" to be included in the regular manual or a separate
 one.
 
 Follows a brief description of the SLY-Slynk protocol. The protocol is
 *s-exp messages* over *s-exp primitives* over *UTF-8* over *TCP*.
 Let's start top-down:
 
 ### S-exp messages
 
 Most messages in the top group look like Lisp function calls. The
 functions are known as "Slyfuns" and are defined with a `DEFSLYFUN`
 operator in the `slynk-*.lisp` side. These are the "remote procedures"
 of the RPC protocol. There must be about 100 or so of them, maybe
 more, I haven't counted. Slyfuns appear in both Slynk's core and in
 supporting contrib's Slynk code.
 
 For a future reference manual, I think there has to be a way to
 automatically harvest the `DEFSLYFUN` definitions and their
 docstrings.
 
 Another type of message contains calls to "channel methods". These are
 slightly different from Slyfuns. Their return value is ignored, but
 otherwise they also work like function calls. They're good for
 expressing a reply-free evaluation in the context of a "channel".
 
 These are defined with `sly-define-channel-method` and
 `DEFINE-CHANNEL-METHOD` and on the SLY and Slynk sides, respectively.
 
 The only use right now is in `sly-mrepl.el`,
 
 ### S-exp primitives
 
 This is a much smaller set of primitives, the most common is
 `:EMACS-REX`, "rex" is for "Remote EXecution".
 
 Informally it's saying: "here is Slyfun X's call number 3487 with
 argumentss Y, for evaluation in thread Z" ). The asynchronous reply
 `:RETURN`, if it ever arrives, will be "your call 3487 returned the
 following sexp".
 
 ```lisp
 (:emacs-rex
  (slynk:connection-info)
  nil t 1)
 (:return
  (:ok
   (:pid 16576 :style :spawn :encoding
         :lisp-implementation
         (:type "International Allegro CL Enterprise Edition" :name "allegro" :version "8.1 [Windows] (Sep 3, 2008 19:38)" :program nil)
         :package
         (:name "COMMON-LISP-USER" :prompt "CL-USER")
         :version "1.0.0-alpha"))
  1)
  ```
 
 The return value, read into Elisp sexps is what is passed to the
 callback argument to the Elisp function `sly-eval-async`. Here's the
 way to get the PID of the underlying Slynk process.
 
 ```elisp
 (sly-eval-async '(slynk:connection-info)
    (lambda (info) (plist-get info :pid)))
 ```
 
 The primitives `:CHANNEL-SEND` and `:EMACS-CHANNEL-SEND` implement
 channel methods. Channels are named by number, and normally have a
 special serving thread in the Common Lisp implementation of
 Slynk. Here is an extract showing the `:PROCESS`, `:WRITE-VALUES` and
 `:PROMPT` channel methods for the REPL.
 
 ```lisp
 (:emacs-channel-send 1
                      (:process "(list 1 2 3)"))
 (:channel-send 1
                (:write-values
                 (("(1 2 3)" 2))))
 (:channel-send 1
                (:prompt "COMMON-LISP-USER" "CL-USER" 0))
 ```
 
 There are also debugger-specific primitives, like `:DEBUG-ACTIVATE`
 and `:DEBUG-RETURN`. Then there are indentation-specific primitives
 like `:INDENTATION-UPDATE`. These could/should become
 `:EMACS-CHANNEL-SEND`s in the future (but that would probably finally
 break Swank compatibility).
 
 ### UTF-8 and TCP
 
 UTF-8 is relevant because the information in the wire are text-encoded
 sexp's that sometimes carry strings with chunks of code, for example,
 and these can have funky characters.
 
 TCP is well TCP, a host and a port and reliable transfer make SLY work
 well over any IP network.
 
 ### Common Lisp bias
 
 *Note: This section is very incomplete*
 
 SLY has is primarily a Common-Lisp IDE and the supporting Slynk have
 strong Common-lisp bias. There have been many attempts, some quite
 successful at creating Slynk backends for other languages.
 
 I believe that some of the Slyfuns will always be Common-Lisp specific
 and should be marked as such. Others can perhaps be more naturally
 adapted to other languages.
 
 It's very important that a future reference manual have this in
 consideration: remove the CL bias from the protocol's description, at
 least from some of its layers, so that things like
 [swank-js](https://github.com/swank-js/swank-js) can one day be more
 easily implemented.
 
 
 ## Architecture changes from SLIME to SLY
 
 As of time of writing (SLY 1.0, SLIME 2.9) the following list
 summarizes the main architecture differences between SLY and SLIME. If
 it's not mentioned here, it's a safe bet that some particular
 mechanism you're interested in stayed the same and any SLIME
 documentation is applicable to SLY.
 
 ### Swank is now called Slynk
 
 SLY can be loaded alongside SLIME both in the same Emacs or Lisp
 image. This interoperability meant that SLY's Lisp server had to be
 renamed to "Slynk".
 
 SLY can still speak the Swank protocol, and should be able to connect
 to any other non-Lisp backends such as Christopher Rhodes' [swankr][4]
 or have non-SLIME clients connect to it such as Robert Brown's
 [swank-client][5].
 
 This is done via a contrib called `sly-retro` and its `slynk-retro`
 counterpart. The contrib's code is loaded by `M-x sly` or `M-x
 sly-connect` *on demand*, meaning that it is possible to start the
 Slynk server without the contrib's Lisp counterpart. See the section
 called "Slynk-loading method"" for how this works in SLY.
 
 *If* it is loaded, `sly-retro` ensures that messages leaving SLY still
 look like
 
      (:emacs-rex (swank:connection-info) nil t 1)
 
 It also ensures that incoming messages are directed to the `SLYNK` and
 `SLYNK-BACKEND` packages. This particular redirection is done via
 package nicknames and a trick in `lib/lisp/slynk-rpc.lisp`. The trick
 is necessary only for the first bootstrapping messages, because on
 startup the `sly-retro` contrib hasn't kicked in and nicknames are not
 immediately setup.
 
 The nicknames pose a compatibility hazard if the user tries to load
 SLIME's Swank server into the Lisp image where Slynk is already
 setup. Therefore, users wishing to run both servers alongside in the
 same Lisp image must ensure that the `sly-retro` contrib is not in
 `sly-contribs`.
 
      (setq sly-contribs (delq 'sly-retro sly-contribs))
 
 [4]: https://github.com/gigamonkey/swankr
 [5]: https://github.com/brown/swank-client
 
 ### Slynk-loading method
 
 In SLIME, `M-x slime` immediately tells the Lisp process started by
 Emacs to use SLIME's own `swank-loader.lisp` program to compile and
 load all possibly available lisp under its directory (including
 contrib's) before the Swank server is created with
 `SWANK:CREATE-SERVER`.
 
 In SLY, the elisp variable `sly-init-function` is set to
 `sly-init-using-asdf` by default, meaning that `M-x sly` will try to
 load Slynk (the SLY equivalent to Swank) via `ASDF:LOAD-SYSTEM`. But 
 this will load only Slynk and no contribs.
 
 Slynk contribs are also represented as ASDF systems. Internally the
 function `sly-contrib--load-slynk-dependencies` will ask Slynk to put
 the contrib's path to the ASDF load path. The `SLYNK:REQUIRE-MODULE`
 abstraction will then call `ASDF:LOAD-SYSTEM`.
 
 In SLY, a contrib's associated Slynk modules is loaded on demand, not
 forced on the user's Lisp run-time.
 
 This also allows the developer to write completely independent
 third-party extensions to SLY, with both Emacs and Lisp parts. See the
 URL http://github.com/joaotavora/sly-hello-world for an example
 extension.
 
 Additionally, if SLY detects that ASDF is not available in the Lisp
 run-time, it will fallback to the old `slynk-loader.lisp` mechanism,
 which has also been revised to support the previous two use cases. Any
 of the two methods is transparent from Emacs's perspective.
 
 ### mREPL
 
 `slime-mrepl` is an experimental SLIME contrib that inspired
 `sly-mrepl`, which is a much enhanced version of it and the default
 REPL for SLY. The main difference to the popular `slime-repl` contrib
 is that `sly-mrepl` is based on Emacs's own `comint.el` so that that
 SLY does not need to worry about functionality like history navigation
 and persistent history, which are consistent with other Emacs modes
 based on `comint.el`.
 
 `sly-mrepl` allows multiple REPLs through the use of channels, which
 are abstraction pioneered in SLIME. Channels are like separate
 mailboxes in the Lisp run-time, and it's slightly different from the
 regular `:emacs-rex` RPC calls in that they directly invoke a remote
 method but expect no reply.
 
 In `slynk-mrepl.lisp`, the `mrepl` class multiple inherits from
 `slynk:channel` and `slynk:listener`. The first takes care of
 channel-based communication and the second has the REPL-specific
 context.
 
 See the section on the "RPC protocl" and switch to the `*sly-events*`
 buffer to see what's going on. 
 
 ### Display-related code
 
 SLIME's age and historical compatibility with XEmacs means it
 reinvented (and possibly invented) many buffer/window/display managing
 techniques that are available today in GNU Emacs's core. Interactive
 buttons, display-related and completion-code have all been pruned as
 much as possible and now reuse Emacs' own libraries.
 
 Hopefully this will make SLY's code focus on SLY's "business logic"
 and easier to read.
 
 ### Channels
 
 TODO
 
 ### Listeners
 
 TODO
 
 
 ## Pull requests
 
 * Read [how to properly contribute to open source projects on Github][1].
 * Use a topic branch to easily amend a pull request later, if necessary.
 * Commit messages should use the syntax of GNU ChangeLog entries.
 * Open a [pull request][2] that relates to *only* one subject with a
   clear title and description in grammatically correct, complete
   sentences.
 
 [1]: http://gun.io/blog/how-to-github-fork-branch-and-pull-request
 [2]: https://help.github.com/articles/using-pull-requests
 [3]: http://www.gnu.org/prep/standards/html_node/Style-of-Change-Logs.html#Style-of-Change-Logs