frostfs.info/content/proposals/proc/FEP-0001.md

588 lines
28 KiB
Markdown
Raw Normal View History

---
FEP: 1
Title: FEP Purpose and Guidelines
Author: Stanislav Bogatyrev <realloc@realloc.spb.ru>
Discussions-To: https://github.com/TrueCloudLab/frostfs.info/pull/3
Status: Draft
Type: Process
Date: 2023-02-01
---
### What is a FEP?
FEP stands for FrostFS Enhancement Proposal. The idea and most of the process is
derived from Python [PEP-1](https://www.python.org/dev/peps/pep-0001/). Almost
all text was simply copied and modified.
A FEP is a design document providing information to the FrostFS community, or
describing a new feature for FrostFS or its processes or environment. The FEP
should provide a concise technical specification of the feature and a rationale
for the feature.
We intend FEPs to be the primary mechanisms for proposing major new features,
for collecting community input on an issue, and for documenting the design
decisions that have gone into FrostFS. The FEP author is responsible for
building consensus within the community and documenting dissenting opinions.
Because the FEPs are maintained as text files in a versioned repository, their
revision history is the historical record of the feature proposal. This
historical record is available by the normal git commands for retrieving older
revisions, and can also be browsed on GitHub in [frostfs.info
repository](https://github.com/TrueCloudLab/frostfs.info/tree/master/content/proposals).
The current rendered version is available on [frostfs.info
site](https://frostfs.info/proposals/).
### FEP Audience
The typical primary audience for FEPs are the FrostFS Core and Service
developers, Architecture Committee, as well as developers of applications
relying on FrostFS or using it's parts.
However, other parts of the FrostFS community may also choose to use the process
(particularly for Informational FEPs) to document expected API conventions and
to manage complex design coordination problems that require collaboration across
multiple projects.
### FEP Types
There are three kinds of FEP:
1. A **Standards Track** FEP describes a new feature or implementation for
FrostFS. It may also describe an interoperability standard that will be
supported outside the standard API for current FrostFS versions before a
subsequent FEP adds support for that API in a future version.
2. An **Informational** FEP describes a FrostFS design issue, or provides
general guidelines or information to the community, but does not propose a
new feature. Informational FEPs do not necessarily represent a FrostFS
community consensus or recommendation, so users and implementers are free to
ignore Informational FEPs or follow their advice.
3. A **Process** FEP describes a process surrounding FrostFS, or proposes a
change to (or an event in) a process. Process FEPs are like Standards Track
FEPs but apply to areas other than the FrostFS itself. They may propose an
implementation, but not to FrostFS's codebase; they often require community
consensus; unlike Informational FEPs, they are more than recommendations, and
community members are typically not free to ignore them. Examples include
procedures, guidelines, changes to the decision-making process, and changes
to the tools or environment used in FrostFS development.
### FEP Workflow
#### Starting with an idea
There are several references in this FEP to the "Architecture Committee",
"Community Committee", "Committers" and "Developers". The detailed roles and
project governance description is available in FEP-X (FrostFS Project
Governance).
The FEP process begins with a new idea for the FrostFS stack. It is highly
recommended that a single FEP contain a single key proposal or new idea; the
more focused the FEP, the more successful it tends to be. Most enhancements and
bug fixes don't need a FEP and can be submitted directly to the GitHub issue
tracker for the corresponding repository. The Architecture Committee reserves
the right to reject FEP proposals if they appear too unfocused or too broad. If
in doubt, split your FEP into several well-focused ones.
Each FEP must have a champion -- someone who writes the FEP using the style and
format described below, shepherds the discussions in the appropriate GitHub
issues or chats, and attempts to build community consensus around the idea. The
FEP author should first attempt to ascertain whether the idea is feasible for
the new FEP. Creating an issue in appropriate GitHub repository with 'idea' or
'discussion' label is usually the best way to go about this, unless a more
specialized venue is appropriate.
Vetting an idea publicly before going as far as writing a FEP is meant to save
the potential author time. Asking the FrostFS community first if an idea is
original helps prevent too much time being spent on something that is guaranteed
to be rejected based on prior discussions (searching the internet does not
always do the trick). It also helps to make sure the idea is applicable to the
entire community and not just the author. Just because an idea sounds good to
the author does not mean it will work for most people in most areas where
FrostFS is used.
Once the champion has asked the community as to whether an idea has any chance
of acceptance, a draft FEP should be presented to the appropriate venue
mentioned above. This gives the author a chance to flesh out the draft FEP to
make properly formatted, of high quality, and to address initial concerns about
the proposal.
#### Submitting a FEP
Following the above initial discussion, the workflow varies based on whether any
of the FEP's co-authors are Committers. If one or more of the FEP's co-authors
are Committers, they are responsible for following the process outlined below.
Otherwise (i.e. none of the co-authors are Committers), then the FEP author(s)
will need to find a sponsor for the FEP.
Ideally, a Committer sponsor is identified, but non-Committer sponsors may also
be selected with the approval of the Architecture Committee. Members of the
GitHub "Committers" and "Architecture Committee" teams are pre-approved to be
sponsors. The sponsor's job is to provide guidance to the FEP author to help
them through the logistics of the FEP process (somewhat acting like a mentor).
Being a sponsor does **not** disqualify that person from becoming a co-author.
The sponsor of a FEP is recorded in the "Sponsor:" field of the header.
Once the sponsor or the Committer(s) co-authoring the FEP deem the document
ready for submission, the proposal should be submitted as a draft FEP via a
[GitHub pull request in frostfs.info
repository](https://github.com/TrueCloudLab/frostfs.info/pulls). The draft must
be written in FEP style as described below, else it will fail review immediately
(although minor errors may be corrected by the editors).
The standard FEP workflow is:
* You, the FEP author, fork the [frostfs.info
repository](https://github.com/TrueCloudLab/frostfs.info/), and create a file
named `content/proposals/<type>/fep-X.md` that contains your new FEP. Use "X"
as your draft FEP number.
* In the "Type:" header field, enter "Standards Track",
"Informational", or "Process" as appropriate, and for the "Status:"
field enter "Draft". For full details, see [FEP Header Preamble]({{<ref
"#fep_header_preamble">}}).
* Update `.github/CODEOWNERS` such that any co-author(s) or sponsors with write
access to the repository are listed for your new file. This ensures any future
pull requests changing the file will be assigned to them.
* Push this to your GitHub fork and submit a pull request.
* The FEP editors review your PR for structure, formatting, and other
errors. Approval criteria are:
* It sound and complete. The ideas must make technical sense. The editors do
not consider whether they seem likely to be accepted.
* The title accurately describes the content.
* The FEP's language (spelling, grammar, sentence structure, etc.) and code
style should be correct and conformant. FEPs with invalid markup will not be
approved.
Editors are generally quite lenient about this initial review, expecting that
problems will be corrected by the reviewing process.
**Note:** Approval of the FEP is no guarantee that there are no embarrassing
mistakes! Correctness is the responsibility of authors and reviewers, not the
editors.
If the FEP isn't ready for approval, an editor will send it back to the author
for revision, with specific instructions, using GitHub pull requests
mechanism.
* Once approved, Architecture Committee will assign your FEP a number.
Once the review process is complete, and the FEP editors approve it (note that
this is *not* the same as accepting your FEP!), they will squash commit your
pull request onto the `master` branch.
The FEP editors will not unreasonably deny publication of a FEP. Reasons for
denying FEP status include duplication of effort, being technically unsound, not
providing proper motivation or addressing backwards compatibility, or not in
keeping with the FrostFS philosophy. The Architecture Committee can be consulted
during the approval phase, and are the final arbiter of a draft's FEP-ability.
As updates are necessary, the FEP author can check in new versions if they (or a
collaborating developer) have write access to the FEP repository. Getting a FEP
number assigned early can be useful for ease of reference, especially when
multiple draft FEPs are being considered at the same time.
Standards Track FEPs consist of two parts, a design document and a reference
implementation or prototype with proof of concept. It is generally recommended
that at least a prototype implementation be co-developed with the FEP, as ideas
that sound good in principle sometimes turn out to be impractical when subjected
to the test of implementation.
#### Discussing a FEP
As soon as a FEP number has been assigned and the draft FEP is committed to the
FEP repository, a discussion thread for the FEP should be created to provide a
central place to discuss and review its contents, and the FEP should be updated
so that the ``Discussions-To`` header links to it. Normally it should be an
issue or Pull Request on GitHub.
If a FEP undergoes a significant re-write or other major, substantive changes to
its proposed specification, a new GitHub issue should typically be created to
solicit additional feedback. If this occurs, the ``Discussions-To`` link must be
updated and a new ``Post-History`` entry added pointing to this new thread.
FEP authors are responsible for collecting community feedback on a FEP before
submitting it for review. However, to avoid long-winded and open-ended
discussions, strategies such as soliciting private or more narrowly-tailored
feedback in the early design phase, collaborating with other community members
with expertise in the FEP's subject matter, and picking an
appropriately-specialized discussion for the FEP's topic (if applicable) should
be considered. FEP authors should use their discretion here.
Once the FEP is assigned a number and committed to the FEP repository,
substantive issues should generally be discussed in corresponding GitHub pull
request reviews or issues. This ensures everyone can follow and contribute,
avoids fragmenting the discussion, and makes sure it is fully considered as part
of the FEP review process. Comments, support, concerns and other feedback on
this designated thread are a critical part of what the Architecture Committee or
FEP-Delegate will consider when reviewing the FEP.
#### FEP Review & Resolution
Once the authors have completed a FEP, they may request a review for style and
consistency from the FEP editors. However, content review and acceptance of the
FEP is ultimately the responsibility of the Architecture Committee or Community
Committee, which is formally initiated by opening a Pull Request once the
authors determine the FEP is ready for final review and resolution.
To expedite the process in selected cases (e.g. when a change is clearly
beneficial and ready to be accepted, but the FEP hasn't been formally submitted
for review yet), the Architecture Committee or Community Committee may also
initiate a FEP review, first notifying the FEP author(s) and giving them a
chance to make revisions.
For a FEP to be accepted it must meet certain minimum criteria. It must be a
clear and complete description of the proposed enhancement. The enhancement must
represent a net improvement. The proposed implementation, if applicable, must be
solid and must not complicate the system unduly. Finally, a proposed enhancement
must follow the spirit of decentralization and reliability in order to be
accepted by the Architecture or Community Committees. (However, the spirit is an
imprecise description; it may be defined as whatever is acceptable to the
Committee. This logic is intentionally circular.)
Once a FEP has been accepted, the reference implementation must be completed.
When the reference implementation is complete and incorporated into the main
source code repository, the status will be changed to "Final".
To allow gathering of additional design and interface feedback before committing
to long term stability for data storage format, protocols or API, a FEP may also
be marked as "Provisional". This is short for "Provisionally Accepted", and
indicates that the proposal has been accepted for inclusion in the reference
implementation, but additional user feedback is needed before the full design
can be considered "Final". Unlike regular accepted FEPs, provisionally accepted
FEPs may still be Rejected or Withdrawn *even after the related changes have
been included in a FrostFS release*.
Wherever possible, it is considered preferable to reduce the scope of a proposal
to avoid the need to rely on the "Provisional" status (e.g. by deferring some
features to later FEPs), as this status can lead to version compatibility
challenges in the wider FrostFS ecosystem.
A FEP can also be assigned the status "Deferred". The FEP author or an editor
can assign the FEP this status when no progress is being made on the FEP. Once a
FEP is deferred, a FEP editor can reassign it to draft status.
A FEP can also be "Rejected". Perhaps after all is said and done it was not a
good idea. It is still important to have a record of this fact. The "Withdrawn"
status is similar - it means that the FEP author themselves has decided that the
FEP is actually a bad idea, or has accepted that a competing proposal is a
better alternative.
When a FEP is Accepted, Rejected or Withdrawn, the FEP should be updated
accordingly. In addition to updating the Status field, at the very least the
Resolution header should be added with a direct link to the relevant post making
a decision on the FEP.
FEPs can also be superseded by a different FEP, rendering the original obsolete.
This is intended for Informational FEPs, where version 2 of an API can replace
version 1.
The possible paths of the status of FEPs are as follows:
![FEP process flow diagram](process_flow.svg)
While not shown in the diagram, "Accepted" FEPs may technically move to
"Rejected" or "Withdrawn" even after acceptance. This will only occur if the
implementation process reveals fundamental flaws in the design that were not
noticed prior to acceptance of the FEP. Unlike Provisional FEPs, these
transitions are only permitted if the accepted proposal has *not* been included
in a FrostFS release - released changes must instead go through the regular
deprecation process (which may require a new FEP providing the rationale for the
deprecation).
Some Informational and Process FEPs may also have a status of "Active" if they
are never meant to be completed. E.g. [FEP-1]({{< ref "/proposals/proc/fep-0001"
>}}) (this FEP :smile:).
#### FEP Maintenance {#fep_maintenance}
In general, FEPs are no longer substantially modified after they have reached
the Accepted, Final, Rejected or Superseded state. Once resolution is reached, a
FEP is considered a historical document rather than a living specification.
Formal documentation of the expected behavior should be maintained elsewhere,
such as the API specification.
If changes based on implementation experience and user feedback are made to
Standards track FEPs while in the Provisional or (with Committee approval)
Accepted state, they should be noted in the FEP, such that the FEP accurately
describes the implementation at the point where it is marked Final.
Active (Informational and Process) FEPs may be updated over time to reflect
changes to development practices and other details. The precise process
followed in these cases will depend on the nature and purpose of the FEP
in question.
Occasionally, a Deferred or even a Withdrawn FEP may be resurrected
with major updates, but it is often better to just propose a new one.
### What belongs in a successful FEP?
Each FEP should have the following parts/sections:
1. Preamble -- RFC-2822 style headers in [Hugo page level
params](https://gohugo.io/variables/page/#page-level-params) format
containing meta-data about the FEP, including the FEP number, a short
descriptive title (limited to a maximum of 44 characters), the names, and
optionally the contact info for each author, etc.
2. Abstract -- a short (~200 word) description of the technical issue being
addressed.
3. Motivation -- The motivation is critical flor FEPs that want to change the
FrostFS protocols, data formats, API, or ecosystem. It should clearly explain
why the existing specification is inadequate to address the problem that the
FEP solves. This can include collecting documented support for the FEP from
important projects in the FrostFS ecosystem. FEP submissions without
sufficient motivation may be rejected.
4. Rationale -- The rationale fleshes out the specification by describing why
particular design decisions were made. It should describe alternate designs
that were considered and related work, e.g. how the feature is supported in
other systems.
The rationale should provide evidence of consensus within the community and
discuss important objections or concerns raised during discussion.
5. Specification -- The technical specification should describe the semantics of
any new feature. The specification should be detailed enough to allow
competing, interoperable implementations.
6. Backwards Compatibility -- All FEPs that introduce backwards
incompatibilities must include a section describing these incompatibilities
and their severity. The FEP must explain how the author proposes to deal with
these incompatibilities. FEP submissions without a sufficient backwards
compatibility treatise may be rejected outright.
7. Security Implications -- If there are security concerns in relation to the
FEP, those concerns should be explicitly written out to make sure reviewers
of the FEP are aware of them.
8. How to Teach This -- For a FEP that adds new functionality or changes system
behavior, it is helpful to include a section on how to teach users, new and
experienced, how to apply the FEP to their work.
9. Reference Implementation -- The reference implementation must be completed
before any FEP is given status "Final", but it need not be completed before
the FEP is accepted. While there is merit to the approach of reaching
consensus on the specification and rationale before writing code, the
principle of "rough consensus and running code" is still useful when it comes
to resolving many discussions of API details.
10. Rejected Ideas -- Throughout the discussion of a FEP, various ideas will be
proposed which are not accepted. Those rejected ideas should be recorded
along with the reasoning as to why they were rejected. This both helps
record the thought process behind the final version of the FEP as well as
preventing people from bringing up the same rejected idea again in
subsequent discussions.
In a way this section can be thought of as a breakout section of the
Rationale section that is focused specifically on why certain ideas were not
ultimately pursued.
11. Open Issues -- While a FEP is in draft, ideas can come up which warrant
further discussion. Those ideas should be recorded so people know that they
are being thought about but do not have a concrete resolution. This helps
make sure all issues required for the FEP to be ready for consideration are
complete and reduces people duplicating prior discussion.
12. Footnotes -- A collection of footnotes cited in the FEP, and a place to list
non-inline hyperlink targets.
13. Copyright/license -- Each new FEP must be placed under a Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 International License ([CC BY-NC-SA
4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/)).
### FEP Formats and Templates
FEPs are UTF-8 encoded text files using the Markdown format in a dialect used by
[Hugo](https://gohugo.io/) static site generator. Markdown allows for rich
markup that is still quite easy to read, but also results in good-looking and
functional HTML.
The FEP text files are automatically converted to HTML for easier online reading
at [FrostFS site](https://frostfs.info/proposals/)
### FEP Header Preamble {#fep_header_preamble}
Each FEP must begin with an RFC-2822 style preamble in [Hugo page level
params](https://gohugo.io/variables/page/#page-level-params) format. The
headers must appear in the following order. Headers marked with "*" are
optional and are described below. All other headers are required.
```YAML
FEP: <fep number>
Title: <fep title>
Author: <list of authors' real names and optionally, email addrs>
* Sponsor: <real name of sponsor>
Discussions-To: <URL of current canonical discussion thread>
Status: <Draft | Active | Accepted | Provisional | Deferred | Rejected |
Withdrawn | Final | Superseded>
Type: <Standards Track | Informational | Process>
Date: <date created on, in yyyy-mm-dd format>
* Requires: <fep numbers>
* Replaces: <fep number>
* Superseded-By: <fep number>
```
The Author header lists the names, and optionally the email addresses of GitHub
handles of all the authors/owners of the FEP. The format of the Author header
values must be:
```YAML
Random J. User <random@example.com>
```
if the email address is included, and just:
```YAML
Random J. User
```
if the address is not given.
If there are multiple authors, each should be on a separate line as defined in
YAML list format, like:
```YAML
Author:
- Random J. User <random@example.com>
- Snegurocka (@snegurochka-chan)
```
The Sponsor field records which Committer or developer (approved by the
Architecture Committees) is sponsoring the FEP. If one of the authors of the FEP
is a Committer then no sponsor is necessary and thus this field should be left
out.
The Discussions-To header provides the URL to the relevant discussion thread for
the FEP. Normally a GitHub issue or Pull Request. For email lists, this should
be a direct link to the thread in the list's archives, rather than just a
mailto: or hyperlink to the list itself.
The Type header specifies the type of FEP: Standards Track, Informational, or
Process.
The Date header records the date that the FEP was assigned a number. Dates
should be in `yyyy-mm-dd` format, e.g. `2023-02-01`.
FEPs may have a Requires header, indicating the FEP numbers that this FEP
depends on.
FEPs may also have a Superseded-By header indicating that a FEP has been
rendered obsolete by a later document; the value is the number of the FEP that
replaces the current document. The newer FEP must have a Replaces header
containing the number of the FEP that it rendered obsolete.
### Auxiliary Files
FEPs may include auxiliary files such as diagrams. All support files may be
placed in a subdirectory called ``fep-XXXX``, where "XXXX" is the FEP number.
When using a subdirectory, there are no constraints on the names used in files.
### Changing Existing FEPs
Draft FEPs are freely open for discussion and proposed modification, at the
discretion of the authors, until submitted to the corresponding Committee for
review and resolution. Substantive content changes should generally be first
proposed on the FEP's discussion thread listed in its `Discussions-To` header,
while copyedits and corrections can be submitted as a GitHub issue or Pull
Request. FEP authors with write access to the FEP repository can update the FEPs
themselves by using a GitHub PR to submit their changes. For guidance on
modifying other FEPs, consult the [FEP Maintenance]({{<ref
"#fep_maintenance">}}) section.
See the [Contributing
Guide](https://github.com/TrueCloudLab/frostfs.info/blob/master/CONTRIBUTING.md)
for additional details, and when in doubt, please check first with the FEP
author and/or a FEP editor.
### Transferring FEP Ownership
It occasionally becomes necessary to transfer ownership of FEPs to a new
champion. In general, it is preferable to retain the original author as a
co-author of the transferred FEP, but that's really up to the original author. A
good reason to transfer ownership is because the original author no longer has
the time or interest in updating it or following through with the FEP process,
or has fallen off the face of the 'net (i.e. is unreachable or not responding to
email). A bad reason to transfer ownership is because the author doesn't agree
with the direction of the FEP. One aim of the FEP process is to try to build
consensus around a FEP, but if that's not possible, an author can always submit
a competing FEP.
If you are interested in assuming ownership of a FEP, you can also do this via
pull request. Fork the FEP repository, make your ownership modification, and
submit a pull request. You should mention both the original author and
`@TrueCloudLab/architecture-committee` or `@TrueCloudLab/community-committee` in a
comment on the pull request. (If the original author's GitHub username is
unknown, use email.) If the original author doesn't respond in a timely manner,
the Committee will make a unilateral decision (it's not like such decisions
can't be reversed :smile:.
### Committee Responsibilities & Workflow
A FEP editor must be added to the `@TrueCloudLab/architecture-committee` or
`@TrueCloudLab/community-committee` group on GitHub, depending on the relevant
FEP types, and must watch the [FEP
repository](https://github.com/TrueCloudLab/frostfs.info).
Note that developers with write access to the `FEP repository` may handle the
tasks that would normally be taken care of by the FEP editors. Alternately, even
developers may request assistance from FEP editors by mentioning the relevant
group on GitHub.
For each new FEP that comes in an editor does the following:
* Make sure that the FEP is either co-authored by a Committer, has a Committer
as a sponsor, or has a sponsor specifically approved for this FEP by the
Committee.
* Read the FEP to check if it is ready: sound and complete. The ideas must make
technical sense, even if they don't seem likely to be accepted.
* The title should accurately describe the content.
* The file name extension is correct (i.e. `.md`).
* Ensure that everyone listed as a sponsor or co-author of the FEP who has write
access to the FEP repository is added to `.github/CODEOWNERS`.
* Skim the FEP for obvious defects in language (spelling, grammar, sentence
structure, etc.), and code style . Editors may correct problems themselves,
but are not required to do so.
* If a project is portrayed as benefiting from or supporting the FEP, make sure
there is some direct indication from the project included to make the support
clear. This is to avoid a FEP accidentally portraying a project as supporting
a FEP when in fact the support is based on conjecture.
If the FEP isn't ready, an editor will send it back to the author for revision,
with specific instructions.
Once the FEP is ready for the repository, a FEP editor will:
* Assign a FEP number (almost always just the next available number, but
sometimes it's a special/joke number, like 666 or 3141).
* Check that the author has correctly labeled the FEP's type
("Standards Track", "Informational", or "Process"), and marked its
status as "Draft".
* Ensure all CI build and lint checks pass without errors,
and there are no obvious issues in the rendered preview output.
* Merge the new (or updated) FEP.
* Inform the author of the next steps (open a discussion thread and
update the FEP with it, post an announcement, etc).
Updates to existing FEPs should be submitted as a GitHub pull request.
### Copyright
This work is licensed under a Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 International License.