eurobsdcon2014-mandoc-paper004075500017500001750000000000001241204214200166315ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/0004075500017500001750000000000001240211410200167635ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/0/section.roff010064400017500001750000000003511240211432300213650ustar00ischwarzeischwarze.nr H1 -1 .H 1 "Introduction, motivation, and history" .so 0/goals.roff .so 0/man_intro.roff .so 0/formats_intro.roff .so 0/roff.roff .so 0/roff_plus.roff .so 0/man.roff .so 0/mdoc.roff .so 0/formats_plus.roff .so 0/mandoc_plus.roff eurobsdcon2014-mandoc-paper/0/goals.roff010064400017500001750000000177561240416307200210560ustar00ischwarzeischwarze.H 2 "Quality of software and documentation" .H 3 "Quality of software" Obviously, software needs to be correct, robust and secure, or you wouldn't use it for fear of getting incorrect results, being out of service when you need it most, or losing or leaking your data. .P Software also needs to be well-designed for usability, or you cannot use it because learning how to use it or actually using it would take too much time and effort. .P It directly follows that documentation is a critical, integral part of any software. Without documentation, it is impossible to judge correctness of a piece of software because only the documenation can specify what the software is supposed to do. Without documentation, usability of the software is very bad; it cannot be used at all without experimentation, and even if you figure out the part of the functionality you need, there is no way of making you sure you use it correctly, you use it in a way reliably and securely producing correct results. The only way to be sure would be to read and understand the complete source code, which is impractical in almost all cases, even for a professional software developer. .H 3 "Quality of documentation" When asked for the quality of a given piece of software, most people think of the quality of the code. Is it well-structured? Is it simple? Was it designed and written with secure coding practices in mind? Is it reliable and robust? Is it efficient? .P However, documentation comes with a bunch of additional quality criteria of its own. Obviously, the documentation needs to be correct. A factual error in the documentation is often almost indistinguishable from a bug in the code: In both cases, the software does not do what you expect. Documentation needs to be complete. Features missing from the documentation are not much better than unimplemented or buggy features for the reasons mentioned above. Documentation needs to be concise. The worth of even a brilliantly designed interface is greatly dimnished if you have to toil hours on end trying to make head or tail of the documentation before you can start using the software. .P What is often overlooked is that documentation needs to be easy to find and easily accessible.\*F .FS /papers/bsdcan11-mandoc-openbsd.html .br Almost all content from that presentation is included in the present paper, at some places in updated or corrected form. It is subsequently cited in the following way: BSDCan 2011 p. 2 .FE It's a waste of time if you need a web search engine or even .Xr locate 1 on the local machine before you find it; and even if you do find it, you may wonder whether it corresponds to the particular version of the software you happen to be using. Unless there is one single standard place for all documentation, you may even miss the existence of some of it. Some documents may require a web browser, some a text processor, some an .Xr info 1 program, and if all use different formatting conventions and different controls for navigating it, distraction and misunderstandings will abound. .P Software documentation is not just plain text. It embeds and explains syntax elements used by various programming languages and by thousands of utility user interfaces. To ease understanding, such syntax elements need to be marked up, resulting both in special formatting when the manual is displayed and in enabling searches for words in specific syntactic roles. .P Finally, documentation needs to be easy to write. There aren't legions of bored technical writers around waiting for new free software being written, waiting to pick it up and document it. And even if they were, they would have a hard time doing the work. Essentially, documenting a piece of software requires reading and understanding the complete code, so the author of the software is about the only person able to adequately and efficiently do the job. Unfortunately, few software developers enjoy writing documentation, most prefer writing code. So in practice, if writing the documentation is difficult or tedious, it will end up being done poorly or not at all. . .bp .H 3 "Case study: OpenBSD" Among the OpenBSD\*F .FS / .FE developers, all the points made above are part of a general consensus. Consequently, that project can be used as an example of one way to deal with these points, and to illustrate some of the problems that arise. .P Even though the relevance of this paper is by no means limited to OpenBSD, that system will repeatedly be used for illustration purposes throughout this paper, usually in subsections entitled "Case study: OpenBSD". This approach is particularly instructive because almost all of the pioneering work related to the .Xr mandoc 1 project\*F was done in the context of the OpenBSD system, paving the way for other systems. .FS http://mdocml.bsd.lv/ .FE .P In OpenBSD, all new reference documentation is written in one single format, the .Xr mdoc 7 language,\*F .FS http://mdocml.bsd.lv/man/mdoc.7.html .FE and put into one single place, the system manual pages accessible via the .Xr man 1 viewer and the .Xr apropos 1 search tool. No user-visible code addition, change, or deletion can be committed without updating all documentation affected by the change at the same time. This is easy to handle for developers since all documentation is organized strictly in reference-manual manner. Every piece of documentations closely accompagnies the utility, function, device, or file format it documents. Consequently, all documentation is always complete and up-to-date, even in OpenBSD-current snapshots. .P There are some limitations to this system, though. The most obvious one arises when including software having lower documentation standards. The SQLite documentation\*F .FS http://sqlite.org/docs.html .FE is a recent and blatant example of such problems. While the documentation provided by SQLite is accurate, complete and concise, fulfilling some quite important requirements, it lacks versioning, ease of access, and syntax markup. It is not included in the software distribution but only made available on the web, so the version you can get access to \(em if you have Internet access, a browser, and figure out where it is \(em almost never corresponds to the version of the software you run. On top of that, it critically relies on GIF images\*F .FS http://sqlite.org/images/syntax/select-stmt.gif .FE that cannot be displayed on a terminal, but require special software. Consequently, OpenBSD contains the SQLite3 library without any documentation. This is an example of an unsolved problem. Automatic or semi-automatic conversion of the documentation to a better format might be possible, but would require very substantial work. The .Xr nginx.conf 5 manual\*F .FS /cgi-bin/man.cgi/OpenBSD-5.5/man5/nginx.conf.5 .FE is an example where a semi-automatic conversion of web content\*F .FS http://nginx.org/en/docs/dirindex.html .FE to an actual manual page was recently performed, and it did indeed require considerable effort. .P Another issue with the approach "all documentation is reference manuals" is how to answer questions related to the choice of the right tools, like "does OpenBSD include web servers, and if so, which one is adequate for my purposes?" Users cannot be expected to search for the answer of such a question in the .Xr httpd 8 manual, and even if they would, it would be the wrong place for explaining that OpenBSD-specific versions of .Xr nginx 8 and Apache 1.3 exist in ports, how these were enhanced with respect to upstream versions, how that might influence the choice of tools, and why you shouldn't use Apache 2 unless you really need it. For questions of this type, OpenBSD has a second place for documentation, the FAQ.\*F .FS /faq/ .FE There are plans to include it as a special section into the manual, but this has not been done yet. Right now, it's only available on the web, causing the usual versioning problems: It is only updated for each release and does not cover OpenBSD-current. Also, it does not show up when searching the manuals with .Xr apropos 1 . eurobsdcon2014-mandoc-paper/0/man_intro.roff010064400017500001750000000035161240416322500217240ustar00ischwarzeischwarze.H 2 "Manuals from the user's perspective" Considering manual pages, for most users, only one single tool will come to mind: .Xr man 1 , the manual viewer. Maybe surprisingly, until this summer, .Xr man 1 was not part of the mandoc toolbox, even though the related search tool, .Xr apropos 1 , which is also known as .Nm man .Fl k , was.\*F .FS /papers/bsdcan14-mandoc.pdf .br Almost all content from that presentation is included in the present paper, at some places in updated or corrected form. It is subsequently cited in the following way: BSDCan 2014 p. 2 .FE Very recently, .Xr man 1 has finally been integrated. .bp .P Now we have one single tool with a unified, simple user interface, performing the following steps in sequence: .AL .LI Find one or more manuals in the file system. The traditional tools for this purpose are .Xr man 1 for retrieval by name and .Xr apropos 1 for searches. .LI Transparently call a formatter on them, where required. The traditional formatting commands are .Xr mandoc 1 and .Xr nroff 1 . .LI Display the formatted text, typically in a pager. Again, the traditional tool for this purpose is .Xr man 1 . .LE .P During the last three years, major steps forward for the mandoc toolbox were, from the user perspective: .BL .LI Output modes -Tuft8 and -Tlocale are available since May 20, 2011. .LI Semantic searching is ready for production since April 14, 2014. .LI The new .Xr man.cgi 8 featuring semantic searches is online on www.openbsd.org since July 12, 2014. .LI A .Xr man 1 implementation providing the unified user interface is available in OpenBSD-current since August 26, 2014. However, OpenBSD still installs the traditional .Xr man 1 implementation by default. .LI The full user interface including .Xr mandoc 1 functionality is reachable via the .Xr man 1 command name since August 30, 2014. .LE eurobsdcon2014-mandoc-paper/0/roff.roff010064400017500001750000000026311240416326500206730ustar00ischwarzeischwarze.H 2 "Origins of the roff(7) language syntax" Considering the history of the roff language, we can start by celebrating an anniversary. This year, 2014, roff is looking back on exactly half a century of active development.\*F .FS BSDCan 2014 p. 3 .FE .P When preparing his thesis at the MIT, professor Jerome H. Saltzer in 1964 wrote the RUNOFF utility which became later known as roff in the UNIX world.\*F .FS http://manpages.bsd.lv/history.html .FE His original implementation used the Michigan Algorithm Decoder (MAD, 1959) programming language for the Compatible Time-Sharing System (CTSS, 1961) operating system running on the MIT's IBM 7094 mainframe computer. He took inspiration from the Memo, Modify, and Ditto tools written by Lowry, Corbato, and Steinberg the year before. .P He already used the fundamental concept of text and macro lines. Macro lines contain a period (\(oq.\(cq), a macro name, and optional arguments. They control formatting \(em and, in modern macro languages, they are also used to specify semantic markup. All other lines are text lines. .bp .P The following requests he introduced in 1964 are still in use today: .VL 5n 2n 1 .LIb ad select text adjustment mode .LIb br break the output line .LIb ce center some lines of text .LIb fi enable text filling mode .LIb in indent some lines of text .LIb ll set the output line length .LIb nf disable text filling mode .LIb sp insert vertical spacing .LE eurobsdcon2014-mandoc-paper/0/formats_intro.roff010064400017500001750000000016701240215110300226110ustar00ischwarzeischwarze.H 2 "Manuals from the author's perspective" Of course, one concern of manual authors will be which languages and tools they have to use to write their manuals. .P Nowadays, we recommend using one simple, versatile language for all software documentation: .Xr mdoc 7 . It is based on the .Xr roff 7 language, the roots of which extend well beyond the UNIX era, back to 1964. The .Xr mdoc 7 language is the successor of the .Xr man 7 language that was first used to format the AT&T Version 7 UNIX manuals in 1979. It was designed for 4.4BSD by the Berkeley Computer Systems Research Group in 1990. It is now supported by both mandoc and groff.\*F .FS http://www.gnu.org/software/groff/ .FE Groff development started in 1989, mandoc in 2008. .P During the last two years, the main step forward in the mandoc toolbox from the author's perspective was the implementation of the mdoc(7) to man(7) converter. More details on that follow later in this paper. eurobsdcon2014-mandoc-paper/0/roff_plus.roff010064400017500001750000000010251240215276500217330ustar00ischwarzeischwarze.H 2 "Advantages of the roff macro syntax" .BL .LI It can easily be hand-edited with minimal typing overhead.\*F .FS BSDCan 2014 p. 5 .FE .LI It looks unobtrusive and does not muddle the actual text. .LI It harmonizes very well with .Xr diff 1 . .LI It allows high quality output in multiple output formats, in particular for terminal output and typesetting. .LI It works with simple, fast, portable, readily available tools. .LI It does not need any heavyweight or cumbersome toolchains, in particular, it does not require XML. .LE eurobsdcon2014-mandoc-paper/0/man.roff010064400017500001750000000041431240416362100205060ustar00ischwarzeischwarze.H 2 "Origin of the basic manual structure" Ken Thompson and Dennis M. Ritchie already used roff when preparing the AT&T Version 1 UNIX manual at the Bell Labs in 1971.\*F .FS BSDCan 2014 p. 6, http://minnie.tuhs.org/cgi-bin/utree.pl?file=V1/man/manintro.txt .FE Dennis M. Ritchie and Joseph F. Ossanna wrote this version of roff for UNIX in DEC PDP-11 assembler. .P The format of the manuals was inspired by the CTSS manuals. .P The following section headers have been in use since Version 1 UNIX: NAME, SYNOPSIS, DESCRIPTION, FILES, SEE ALSO, DIAGNOSTICS, BUGS. .P Until AT&T Version 3 UNIX, the manuals were formatted in pure roff, without using any roff macro set. The only roff source file included was the file .Pa man0/aa , disabling adjustment with .Cm na , clearing the hyphenation character with .Cm hc , setting up a default indentation with .Cm in , and defining a default page footer line with .Cm fo . The latter no longer exists in modern roff and would be implemented in terms of .Cm wh (page position trap) and .Cm tl (three-part header) nowadays. The only roff requests used abundantly in the Version 3 UNIX manuals were .Cm sp (insert vertical space) to mark paragraph breaks and .Cm ti (temporary indent for the next output line) to set section headers without indentation. A considerable list of various macros still in use today occurred occasionally, for example .VL 5n 2n 1 .LIb bp break output page .LIb br break output line .LIb ce center next line .LIb fi enable fill mode .LIb in change permanent indent .LIb nf disable fill mode .LIb nx include input file .LIb ta set tab stop positions .LIb tc set tab fill character .LIb tr translate character .LIb ul underline .LE .P Precursors to .Xr man 7 and .Xr mdoc 7 macros occurred in Version 4 to Version 6 UNIX (1973-1975). For example, the files .Pa man0/naa for terminal output and .Pa man0/taa for typesetter output defined macros for the following purposes: .VL 5n 2n 1 .LIb th page title, now TH/Dt .LIb sh section header, now SH/Sh .LIb bd bold text, now B/Sy .LIb it italic text, now I/Em .LE .P The .Xr man 7 language first appeared in Version 7 AT&T UNIX (1979). eurobsdcon2014-mandoc-paper/0/mdoc.roff010064400017500001750000000021101240215472100206440ustar00ischwarzeischwarze.H 2 "Origin of semantic markup in manuals" The .Xr mdoc 7 semantic markup macro language was designed to format the manuals of the 4.4BSD release. The translation of the manuals from .Xr man 7 to .Xr mdoc 7 was performed by Cynthia Livingston of USENIX.\*F .FS BSDCan 2014 p. 7 .FE The first few of these translated manuals appeared in 4.3BSD-Reno in 1990. The formatter used for this version was Brian Kernighan's device independent troff, written in K&R C, running on BSD UNIX on DEC VAX. .P The advantages of the .Xr mdoc 7 language are: .BL .LI Considerable expressive power for semantic markup, while .Xr man 7 is a presentation level language only. .LI It works in practice as a standalone language, while .Xr man 7 regularly requires resorting to low-level roff features. .LI Consequently, .Xr mdoc 7 shows a more more uniform appearance and is easier to read and write than .Xr man 7 . .LI Portability is no longer an issue: for legacy systems still not having .Xr mdoc 7 , .Xr mandoc 1 can be used to convert to .Xr man 7 . .LI The .Xr mdoc 7 languages supports semantic searching. .LE eurobsdcon2014-mandoc-paper/0/formats_plus.roff010064400017500001750000000006611240215517400224540ustar00ischwarzeischwarze.H 2 "Classic documentation formats (summary)" The .Xr roff 7 input syntax, the .Xr mdoc 7 semantic markup, and the .Xr man 1 presentation format have proven timeless by their simplicity and efficiency.\*F .FS BSDCan 2014 p. 9 .FE Nobody has come up with a better basic concept yet, even though many have tried, and regarding the formats, there is indeed little one could wish. .P Consequently, modern tools are needed for all this. eurobsdcon2014-mandoc-paper/0/mandoc_plus.roff010064400017500001750000000015621240416345200222430ustar00ischwarzeischwarze.H 2 "Advantages of mandoc" .BL .LI Functional \(em all in one binary:\*F .FS BSDCan 2014 p. 10 .FE .BL \n(Pi 1 .LI Searching by filename, page name, word, substring, regular expression, semantic keys .LI .Xr mdoc 7 , .Xr man 7 , .Xr tbl 7 and some .Xr eqn 7 and .Xr roff 7 input .LI ASCII, UTF-8, HTML, XHTML, PostScript, PDF output .LI .Xr mdoc 7 to .Xr man 7 conversion .LI includes .Xr mandoc 1 , .Xr man 1 , .Xr apropos 1 , .Xr whatis 1 , and .Xr makewhatis 8 .LE .LI Free \(em ISC/BSD-licensed, no GPL code. .LI Lightweight \(em ANSI C, POSIX, no C++ code. .LI Portable \(em includes .Pa compat_*.c files for missing functions on older systems. .LI Small \(em source tarball (uncompressed) is 8% of groff, executable binary 50%. .LI Fast \(em for .Xr mdoc 7 , typically 5 times faster than groff, typically about a hundred times faster than an AsciiDoc/DocBook toolchain. .LE eurobsdcon2014-mandoc-paper/1004075500017500001750000000000001240344412200167755ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/1/section.roff010064400017500001750000000001461240344413000213730ustar00ischwarzeischwarze.H 1 "Using the mdoc(7) formatting language" .so 1/start.roff .so 1/structure.roff .so 1/badnest.roff eurobsdcon2014-mandoc-paper/1/structure.roff010064400017500001750000000043021240307074400217730ustar00ischwarzeischwarze.H 2 "Block nesting" The .Xr mdoc 7 language does not only provide formatting cues and semantic markup to manual pages, but it also provides a structure to the document.\*F .FS BSDCan 2011 p. 7 .FE Sections, subsections, displays, lists, list items, and quoting enclosures are blocks that can nest within each other, and that can contain text and markup elements, which in turn can contain text. So while a .Xr man 7 document merely told you "this string is to be set in an italic font", an .Xr mdoc 7 document might tell you something like "this is a function argument in a function prototype in the SYNOPSIS section" or "this is an option character marked as optional in a list item of a tagged list in the DESCRIPTION section." .P Here is an example of nested blocks: .VERBON 16 $ mandoc chgrp.1 SYNOPSIS chgrp [-fh] [-R [-H | -L | -P]] group file ... .VERBOFF .P The corresponding .Xr mdoc 7 source code reads as follows: .VERBON 16 $ less chgrp.1 [...] \&.Sh SYNOPSIS \&.Nm chgrp \&.Op Fl fh \&.Oo \&.Fl R \&.Op Fl H | L | P \&.Oc \&.Ar group \&.Ar [...] .VERBOFF .P Mandoc represents this by this syntax tree: .VERBON 16 $ mandoc -Ttree chgrp.1 # much simplified Sh (block) SYNOPSIS Nm (block) chgrp Op (block) Fl (elem) fh Oo (block) Fl (elem) R Op (block) Fl (elem) H (text) | Fl (elem) L (text) | Fl (elem) P Ar (elem) group Ar (elem) file ... .VERBOFF .P By contrast, the traditional roff design knew no block structure. On the lower level of that design, .I "roff requests" provided physical formatting, registers to store data, and macro expansion facilities. On the higher level of that design, the .I "mdoc macros" called physical formatting requests, set registers to keep state, but did not result in any kind of a syntax tree in any stage of their processing. So even though the .Xr mdoc 7 language itself always contained structural information, before the advent of .Xr mandoc 1 , that information could never actually be used for anything but was discarded in the very first step of the formatting process. eurobsdcon2014-mandoc-paper/1/badnest.roff010064400017500001750000000104451240414460400213560ustar00ischwarzeischwarze.H 2 "Badly nested blocks" Unfortunately, and in contrast to many other languages featuring nested blocks, in particular XML, the .Xr mdoc 7 language does not only support the nicely nested blocks shown in the previous section, but it also allows badly nested blocks, that is, blocks that overlap without any of them being completely contained in the other.\*F .FS BSDCan 2011 p. 8 .FE .P The simplest case of badly nested blocks can be constructed with just two blocks, if both blocks first open and then close .I "in the same order". In that case, text following the opening of the first block is only contained in the first block, text following the opening of the second block is contained in both, and text following the closing of the first block is only contained in the second one. .P Here is an example with .I explicit blocks, that is, blocks being closed with an explicit block end macro: .VERBON 16 \&.Ao ao \&.Bo bo \&.No ac Ac \&.No bc Bc .VERBOFF .P This example formats as: .VERBON 16 bc] .VERBOFF .P The same result can be constructed with .I implicit blocks, that is, blocks always extending to the end of their input line and automatically closing their scope at the end of the line. Here is the variant where an explicit block breaks an implicit one, that is, the end macro of the explicit block occurs while the scope of the later implicit block is still open: .VERBON 16 \&.Ao ao \&.Bq bq ac Ac eol .VERBOFF .P This example formats as: .VERBON 16 eol] .VERBOFF .P Conversely, an implicit block can break an explicit one, that is, the line containing the implicit block might end before the end macro of the later explicit block arrives: .VERBON 16 \&.Aq aq Bo bo eol \&.No bc Bc .VERBOFF .P This example formats as: .VERBON 16 bc] .VERBOFF Actually, this is the most important case in practice, and we shall return to it below. .P Note that implicit macros cannot break each other. If they are on different input lines, their scopes do not intersect at all. If they are on the same input line, they both end at the same point, at the end of the line, so the earlier one completely contains the later one. . .H 3 "The scope extension macro" There is one enclosure macro that, in contrast to all other enclosure macros, generates no output whatsoever, neither at the beginning nor at the end of its scope: .BR Xo / Xc . Its sole purpose is to have its scope broken by an implicit macro, effectively extending the scope of the implicit macro onto the following lines. The only case where this is used in practice is to extend the head scope of the list item macro, .It. .Pp Here is a practicle example: .VERBON 16 $ less find.1 [...] \&.It Xo \&.Ic -exec Ar utility \&.Op argument ... \&.No ; \&.Xc .VERBOFF .P This example formats as: .VERBON 16 -exec utility [argument ...] ; .VERBOFF .P For modern roff implementations including groff and mandoc, this can equivalently be written as: .VERBON 16 \&.It Ic -exec Ar utility Oo argument ... Oc No ; .VERBOFF .P The original motivation for introducing the .Cm Xo macro here was that historic roff implementations only supported a limited number of macro arguments on a macro line, and the above one-line version would have violated that limit, causing the last one or two arguments to be lost. .P When working on .Xr mandoc 1 in 2010, our first thought was: deprecate this abomination, tell manual authors to use the the one-line version, the historical argument limit is no longer relevant, so we get a clean definition of the language. However, large numbers of manuals in various operating systems and in countless portable software packages use .Cm It .Cm Xo , and there is no way to find and change them all, everywhere, or even to change the habits of people writing new manuals. .P So somewhat reluctantly at first, we implemented support for badly nested blocks in .Xr mandoc 1 in the following way, using the first example cited above for the explanation: The Ao block contains the .I whole Bo block, and the Bo block contains an Ao body-end element in the middle, indicating where Ao formatting is supposed to end. . .H 3 "Case study: OpenBSD" Related timeline: .nf 2010 Feb 23 remove .Oo .Xo .Oc .Xc mis-nesting from manuals (questionable) 2010 Feb 26 support .It Xo (good); all mdoc(7) manuals build now 2010 Jun 29 support badly nested blocks in general (even better) .fi eurobsdcon2014-mandoc-paper/1/start.roff010064400017500001750000000135201240416370400210720ustar00ischwarzeischwarze.H 2 "Getting started with mdoc" This section describes the most important things a beginner needs to know about the .Xr mdoc 7 language in order to write a new manual page from scratch. .P An .Xr mdoc 7 source file consists of .I "macro lines" starting with a dot (\(oq.\(cq) in the first column, mostly specifying document structure and providing semantic annotations, and .I "text lines" starting with any other character. Macros take any number of .I arguments , separated by blank characters (\(oq\ \(cq). If an argument contains a blank character, enclose the whole argument in double quotes (\(oq"\(cq). .P Every manual page starts with a .I prologue exclusively containing macro lines and never any text lines. After that, some .I sections will follow, each starting with an .Cm Sh macro. Sections have conventional names and follow a strict conventional order. Avoid custom section names, except when splitting the DESCRIPTION in very long manuals. .H 3 "The mdoc prologue" Always use the following macros in the following order: .BL 5n .LIb Dd Document date. Use the format .Ar "Month day, year" with a full English month name, a one- or two-digit day number, and a four-digit year. .LIb Dt Document title and section number. The first argument is the name of the manual page that will be passed to the .Xr man 1 command, but converted to ALL CAPS. The second argument is the section number as a single digit, see the .Xr man 1 or .Xr mdoc 7 manuals for lists of section numbers. .LIb Os Optional operating system name. Just leave it blank. .LIb Sh Section header. The argument of the first section header macro must be the exact string .Cm NAME . .LIb Nm Page name. The same name you provided for .Cm Dt , but now using its normal lower-case or upper-case spelling, whatever is appropriate. .LIb Nd One-line description. No quoting is needed. .LE .P As a beginner, it may help to remember the number .I six : If your prologue has six lines, it's probably complete. .P Here is a typical example: .VERBON 16 \&.Dd July 16, 2013 \&.Dt CAT 1 \&.Os \&.Sh NAME \&.Nm cat \&.Nd concatenate and print files .VERBOFF .H 3 "The SYNOPSIS section" In manuals documenting utilities (sections 1 and 8) and library functions (sections 2 and 3), the next section always starts with .B ".Sh SYNOPSIS" . This section only documents syntax, not semantics. It never contains any free text. Do not worry about formatting. Just specify the syntax, formatting will be done automatically. .P For utilities, you almost always need the following macros: .BL 5n 1 .LIb Nm Utility name. Usually the same you used in the NAME section. .LIb Op Optional syntax element. .LIb Fl Command line options (flags). .LIb Ar Command line arguments. .LE .bp .P A typical example looks like this: .VERBON 16 \&.Sh SYNOPSIS \&.Nm cat \&.Op Fl benstuv \&.Op Ar .VERBOFF .P The formatted output is: .VERBON 16 \fBSYNOPSIS\fP \fBcat\fP [\fB\-benstuv\fP] [\fIfile ...\fP] .VERBOFF .P Most macros can take other macros as arguments. In that case, the .I called macros don't have a dot, like .Cm Fl and .Cm Ar in the example above. The .Cm Op macro is an example of an .I enclosure macro, having a .I scope that can contain macros and text. For .Cm Op , the scope extends to the end of the input line. .P For library functions, you almost always need the following macros: .BL 5n 1 .LIb In Include file. .LIb Ft Function type. .LIb Fo Begin (open) function declaration. .LIb Fa Function argument. .LIb Fc End (close) function declaration. .LE .P A typical example looks like this: .VERBON 16 \&.Sh SYNOPSIS \&.In unistd.h \&.Ft ssize_t \&.Fo read \&.Fa "int d" \&.Fa "void *buf" \&.Fa "size_t nbytes" \&.Fc .VERBOFF .P The formatted output is: .VERBON 16 \fBSYNOPSIS\fP \fB#include \fP \fIssize_t\fP \fBread\fP(\fIint d\fP, \fIvoid *buf\fP, \fIsize_t nbytes\fP); .VERBOFF .P The .Cm Fo macro is a .I "block macro" starting a scope that requires .I explicit closure by the .Cm Fc companion macro. .H 3 "The description section" Next, every manual page has a section starting with .B ".Sh DESCRIPTION". Start it by explaining the purpose of the topic, followed by a concise description of the syntax and semantics of all features, except those to be described in the following sections coming after the DESCRIPTION itself: RETURN VALUES (of library functions), ENVIRONMENT, FILES, EXIT STATUS (of utilities), EXAMPLES, DIAGNOSTICS (of utilities), ERRORS (of library functions). Finally, you can add some sections containing concluding material: SEE ALSO, STANDARDS, HISTORY, AUTHORS, CAVEATS, BUGS. .H 2 Resources The most important resource to use is the .Xr mdoc 7 manual page. In particular, when wondering which macro to use for markup, first look at the MACRO OVERVIEW section for candidates, then at the description of the specific macro you consider to choose in the MACRO REFERENCE section. The .Xr mdoc 7 manual also contains some more details regarding the various manual sections. .P To see examples of good usage, look at existing manuals in the OpenBSD base tree. This is particularly helpful when wondering about customary choices of macro arguments. For example, you might find the customary form of an option list in the DESCRIPTION section in this way, which is: .VERBON 16 The options are as follows: \&.Bl -tag -width Ds \&.It Fl a .VERBOFF .P Run .B "mandoc -Tlint" on what you have written. It catches most syntax errors and provides some stylistic hints regarding syntax. .P If ambiguities remain even after studying the .Xr mdoc 7 manual, try looking at the .Xr groff_mdoc 7 manual contained in the textproc/groff port. In some cases, it may contain additional hints. .P If you feel information is missing from the .Xr mdoc 7 manual or think something could be made clearer, write to the mandoc discussion list.\*F .FS mailto:discuss@mdocml.bsd.lv .FE .P Kristaps has written a full tutorial, "Practical UNIX Manuals".\*F .FS http://manpages.bsd.lv/ .FE eurobsdcon2014-mandoc-paper/2004075500017500001750000000000001240214030500167715ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/2/section.roff010064400017500001750000000001461240214032200213670ustar00ischwarzeischwarze.H 1 "Manual pages for portable software" .so 2/choice.roff .so 2/sudo.roff .so 2/implementation.roff eurobsdcon2014-mandoc-paper/2/choice.roff010064400017500001750000000020021240414514100211540ustar00ischwarzeischwarze.H 2 "Choosing the language" Consider portable software packages like .Xr sudo 8 , OpenSSH, OpenSMTPd, and so on.\*F .FS BSDCan 2014 p. 33 .FE Which markup language should be chosen for the manual pages? At first, one might think that choosing .Xr mdoc 7 would cause issues for some legacy systems that still don't have .Xr mdoc 7 after it has been freely available for more than 20 years (hello, Solaris). However, using .Xr man 7 makes maintenance much harder and gives up semantic markup, which would be a very bad idea indeed. .P The mandoc toolbox provides a good way out of this dilemma. Write the manual pages using the .Xr mdoc 7 language. Use .Nm mandoc .Fl Tman to convert them to .Xr man 7 format \(em that converter is fully operational since November 19, 2012 and constantly maintained, so you need not fear that it might go away. Include both the .Xr mdoc 7 and .Xr man 7 versions into distribution tarballs. Let .Nm ./configure figure out what the target system supports and install the best supported version. eurobsdcon2014-mandoc-paper/2/sudo.roff010064400017500001750000000014351240232403100207000ustar00ischwarzeischwarze.H 2 "Case study: the sudo(8) manuals" .H 3 "Build system for the distribution tarball" Among the maintainer targets, the Makefile contains targets similar to the following, shown here in simplified form:\*F .FS BSDCan 2014 p. 34 .FE .VERBON 6 sudo.man: sudo.mdoc mandoc -Tman sudo.mdoc > sudo.man sudo.cat: sudo.mdoc mandoc sudo.mdoc > sudo.cat .VERBOFF .H 3 "Installation system" .BL .LI If .Nm ./configure finds .Xr mandoc 1 , it installs the .Pa *.mdoc pages. .LI If .Nm ./configure does not find .Xr nroff 1 , it installs the .Pa *.cat pages. .LI If .Nm ./configure successfully tests .Nm nroff .Fl mdoc , it installs the .Pa *.mdoc pages. .LI Otherwise, it installs the .Pa *.man pages. .LI To override this autodetection logic, it provides options .Fl -with-mdoc and .Fl -with-man . .LE eurobsdcon2014-mandoc-paper/2/implementation.roff010064400017500001750000000032601240414530400227570ustar00ischwarzeischwarze.H 2 "Implementation of the mdoc to man converter" The converter first runs the .Xr mdoc 7 parser, constructing exactly the same abstract syntax tree in memory as when running .Fl Tascii , .Fl Thtml , or .Fl Tps .\*F .FS BSDCan 2014 pp. 35 .FE .P After that, it runs a dedicated mdoc-to-man output module, structured similarly as the mdoc-to-ASCII output module, but sharing no code. That module consists of only one file, 1600 lines of very straightforward C source code. Its central component is one macro lookup table containing pre- and post-node action functions and pre- and post-node output strings for each .Xr mdoc 7 macro type. The module iterates the syntax tree and calls the appropriate action functions for each .Xr mdoc 7 node. .P An alternative, slightly more flexible approach would have been to first translate the .Xr mdoc 7 syntax tree to a .Xr man 7 syntax tree, then provide a non-translating .Xr man 7 output module. That would have allowed man-to-man code normalization as a by-product. However, the direct approach was simpler and has so far proven sufficient for all practical needs. .P The converter is a typical example of a tool that was technically quite easy to build on top of existing infrastructure, that is on top of the .Xr mandoc 3 parser library, but quite useful and powerful in practice.\*F .FS BSDCan 2014 pp. 36 .FE At first, I underestimated the importance of this tool, so development only proceeded haltingly, nearly exclusively at hackathons: .BL .LI I started development on September 17, 2011 (s2k11, Ljubljana). .LI The bulk of the work was done around July 10, 2012 (g2k12, Budapest). .LI It is ready for production since November 19, 2012 (c2k12, Coimbra). .LE eurobsdcon2014-mandoc-paper/3004075500017500001750000000000001240314556700170125ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/3/section.roff010064400017500001750000000002751240314323300214000ustar00ischwarzeischwarze.nr H1 2 .H 1 "Quality control for existing manuals" .so 3/design.roff .so 3/goals.roff .so 3/mandoc-Tlint.roff .so 3/mdoclint.roff .so 3/igor.roff .so 3/gmdiff.roff .so 3/consistency.roff eurobsdcon2014-mandoc-paper/3/design.roff010064400017500001750000000037311240414552100212100ustar00ischwarzeischwarze.H 2 "Why it is critical to get errors and warnings right" A good system of warning and error messages considerably improves the quality of a given piece of software. Without it, even with good documentation, users have a hard time to figure out .I what exactly is wrong as soon as anything goes wrong \(em and sooner or later, something .I will always go wrong. .P There are many pitfalls to avoid when implementing a message system, and all of them cause their specific class of problems for the user: .BL .LI A fatal error gets thrown: The manual doesn't format at all, which is very inconvenient. By all means, we want to report errors, but we now try hard to get going and not error out, no matter what broken manual page source code we find.\*F .FS BSDCan 2011 p. 10 .FE .LI An error message is missing or too generic: Users have a hard time to fix their errors.\*F .FS BSDCan 2011 p. 21 .FE .LI A warning message is missing: Users don't even notice their dangerous idioms. .LI A few warnings too many, or too prominent: Users get annoyed and switch off all warnings. .LI More than one or two knobs: Users don't remember and don't use them. .LI Too few and too many can happen all at once! It did with mandoc, and it had too many knobs - at first. .LE .P For .Xr mandoc 1 , improving the message system was an iterative process that took several steps until we arrived at we have today. Major cleanups were done in July 2009, May 2010, August 2010, October 2010, January 2011, March 2011, July 2014, ... .P Related timeline: .nf 2009 Jul 12: fewer knobs: remove -Wsyntax -Wcompat 2010 May 13: fewer knobs: remove -fno-ign-chars 2010 May 23: unified error and warning system by kristaps@ 2010 Aug 19: simple, consistent user interface for error handling 2010 Oct 24: do not throw fatal errors when there is no need to 2010 Oct 26: downgrade nearly 20 errors to warnings 2011 Jan 16: downgrade yet another bunch of fatal errors 2011 Jan 22: check argument count validation for all in_line() macros .fi eurobsdcon2014-mandoc-paper/3/consistency.roff010064400017500001750000000015571240414753100223070ustar00ischwarzeischwarze.H 2 "Consistency checking with makewhatis" While building databases, .Xr makewhatis 8 can watch out for the following issues: .BL .LI Mismatch of the section number given in the manual page with the directory the page is stored in.\*F .FS BSDCan 2014 p. 24 .FE .LI Mismatch of the architecture name given in the manual page with the directory the page is stored in. .LI File name does not appear as a name in the NAME section (since April 4, 2014). .LE .P The checks that were already performed by the old .Xr makewhatis 8 are preserved: .BL .LI Missing NAME section, missing name(s) and/or missing description. .LI A name in the name section does not appear as an MLINK in the file system (since April 4, 2014). .LE .P Besides, direct inspection of the database has been used to catch markup errors. .P With .Xr makewhatis 8 , more can be done later, all this is just a start. eurobsdcon2014-mandoc-paper/3/mandoc-Tlint.roff010064400017500001750000000102431240416401700222650ustar00ischwarzeischwarze.H 2 "Checking with mandoc -Tlint" The most important tool for any kind of manual page quality control is .Xr mandoc 1 itself, in particular its .Fl W and .Fl Tlint options. .H 3 "Catching fatal errors" When processing a larger amount of manual pages, the first step is to check whether any of these pages causes a fatal error. When dealing with just a handful of pages, this step can be skipped, the fatal errors will show up among the normal errors below. .P To find fatal errors, run a command similar to: .VERBON 16 mandoc -Tlint -Wfatal * .VERBOFF .P Any fatal errors that show up here must be fixed and should be dealt with first, before looking at anything else. A fatal error means that the user is unable to read the manual at all, because parsing the manual fails outright and no output whatsoever can be generated. Consequently, having a manual containing a fatal error is not much better than having no manual at all. .P Very few types of fatal errors exist. Those that may occasionally occur in practice are all related to file inclusion: .BL \n(Pi 1 .LI use of the unsafe macro .Cm Bd .Fl file .LI use of the .Cm so file inclusion request with an absolute path .LI use of .Cm so with a path containing ".." .LI .Cm so pointing to a file that doesn't exist or can't be opened .LE .P See the .Xr mandoc 1 manual in the portable mandoc distribution\*F .FS http://mdocml.bsd.lv/man/mandoc.1.html .FE for details, section "FATAL errors". . .H 3 "Catching errors" The most important step when doing quality control with .Xr mandoc 1 is to deal with errors. These are issues that can potentially cause loss of information, severe misformatting, or severe portability problems. .P To find errors, run a command similar to: .VERBON 16 mandoc -Tlint -Werror * .VERBOFF .P The .Xr mandoc 1 utility tries very hard to avoid false positives when reporting errors. So, if it reports any errors, you very probably want to fix them. If you run into anything you did on purpose and want to keep it as it is but .Xr mandoc 1 calls it an error, please do report that to the mandoc developers. .P There are more types of errors than fatal errors, but the number of error types that occur in practice is still rather small: .BL .LI unencoded non-ASCII characters in the input .LI unknown or mistyped macro or request names .LI blocking issues: .BL \n(Pi 1 .LI opening blocks that are never closed again .LI closing blocks that were never opened .LI items outside lists .LI bad nesting of blocks that don't support it .LE .LI severe issues with macro or request arguments: .BL \n(Pi 1 .LI missing essential arguments .LI invalid arguments that cannot be handled adequately .LI excessive arguments that get completely lost during formatting .LE .LE .P Again, see the .Xr mandoc 1 manual in the portable mandoc distribution for details, section "ERRORS". . .H 3 "Mandoc warnings" Everything else .Xr mandoc 1 considers problematic is classified as a warning. .P To see the warnings, run a command similar to: .VERBON 16 mandoc -Tlint * .VERBOFF .P In any tree containing many low-quality manuals, this is likely to produce a lot of output. On the other hand, the complete set of 145 OpenBSD system call (section 2) manuals currently causes only 28 warnings grand total. Of these 145 manuals, 141 have no warnings at all. .P We do try to avoid bogus warnings, but sometimes we fail. Also, some usage that is usually a bad idea may be justified in exceptional cases, in which case an occasional false positive is the price to pay to avoid frequent false negatives. .P So, fixing all warnings is usually a good idea, but don't do so blindly, there may be exceptions. .bp .P Warning types are too numerous to provide an exhaustive list here, but most are of these or similar kinds: .BL \n(Pi 1 .LI structural errors and syntax errors that only have local effects and do not cause information loss .LI low quality syntax like badly nested blocks or macro usage in contexts expecting plain text .LI macros that have no effect or are slightly misplaced .LI missing arguments or information, if the effect is only local .LI violations of usual structural or formatting conventions .LI warnings about robustness and portability .LI dubious usage of white space and comments .LE eurobsdcon2014-mandoc-paper/3/goals.roff010064400017500001750000000073241240414616700210550ustar00ischwarzeischwarze.H 2 "Goals of quality control for manuals" There are many possible motivations for taking an existing suite of manuals and doing quality checks on them: .BVL \n(Pi .LI "Making sure all manuals actually produce output." In exceptional cases, manual page files can be so broken that formatters do not produce any output at all, but just show an empty page, abort processing, or even crash, if the broken input triggers bugs in the processors. .LI "Making sure that all intended content is actually shown." Some particularly severe markup errors may cause document content to be completely lost during formatting. The most frequent example are probably mistyped macro names. Another example is inadvertently putting text on a preceding macro line instead of starting a new line, exceeding the maximum number of arguments for that particular macro, if it has such a limit. .LI "Catching severe formatting errors." Even if all intended text is shown, severe misformatting can make it hard to read. .LI "Catching typos and stylistic glitches." These can be distracting, sometimes even confusing, when users are reading manuals and trying to understand the content. .LI "Improving portability." Some particular constructions work with some formatters but fail outright with other widespread formatters, for example nested displays. Obviously, such constructions should be avoided. Of course, judgement is needed here, portability can easily be overdone. If you were to write manuals in a way compatible with very old, historic formatters, you would end up with .Xr mdoc 7 code that would be hard to read and maintain. .LI "Improving robustness." Some requests in principle work with all formatters, but are inherently fragile, in particular the .Cm so (file inclusion) request. .LI "Unifying the style of displayed manuals." If the manuals displayed to users follow common conventions with respect to structure, arrangement of content, formatting, and wording, they are easier to understand. .LI "Improving the style of the source code." If the .Xr mdoc 7 source code of the manual pages follows common conventions, even if these don't make a difference for what is displayed, editing and maintenance become simpler for authors and developers. .LI "Finding formatter bugs." While strictly speaking, this is not a matter of .I "manual page" quality control, but rather of pager and formatter .I "utility" quality control, some of the techniques that can be used are so similar that discussing the two together helps a lot. Besides, cases exist where distinguishing formatting bugs, parser bugs, and formatter bugs is non-trivial, or even merely a matter of definition. .LE .P Obviously, these goals are very diverse, and the difficulty to do the required checks manually or automatically varies greatly. Various tools are available, most having strengths in one or a few particular areas, but not doing much with respect to other areas. For some goals, there is hardly any automatic support, and manual work is required. .P In the following, tools are listed roughly in the order of importance for somebody doing bulk checks on a larger body of manuals. For just checking a single page, you are likely to get away with the simplest of these tools plus some manual checking. On the other hand, for hunting parser bugs, the more elaborate of these tools are certainly needed. .P To avoid distraction, the following sections assume that all the source files you want to work on are in a single directory, and that directory contains no other files. In practice, that will sometimes not be the case, and you may need tools like .Xr find 1 . One useful technique is to prepare a list of files you want to work on and then use commands similar to: .VERBON 16 mandoc -Tlint -Werror $(cat files.list) .VERBOFF eurobsdcon2014-mandoc-paper/3/mdoclint.roff010064400017500001750000000015131240313676700215570ustar00ischwarzeischwarze.H 2 "Checking with mdoclint" The .Xr mdoclint 1 utility by Thomas Klausner has a similar focus as .Nm mandoc .Fl Tlint . It contains some additional tests, so it's a useful complement. It also tries to avoid false positives, though not quite as strictly as .Nm mandoc .Fl Tlint . So the same caveat applies: Do not slavishly follow its findings. .P The .Xr mdoclint 1 utility is available on NetBSD\*F .FS http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/textproc/mdoclint/ .FE and OpenBSD\*F .FS http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/regress/usr.bin/mdoclint/ .FE and ought to be trivial to port to other systems providing the .Xr perl 1 programming language. .P To run it, simply give the names of the files you want to check on the command line. Options are almost exclusively needed if you want it to shut up about some of its findings. eurobsdcon2014-mandoc-paper/3/igor.roff010064400017500001750000000021621240314250000206650ustar00ischwarzeischwarze.H 2 "Checking with igor" The .Xr igor 1 utility\*F .FS http://en.wikipedia.org/wiki/Igor_%28character%29 .FE by Warren Block has a completely different focus than the two linters: It mostly cares about style and spelling and knows relatively little about syntax. .P In contrast to the two linters, .Xr igor 1 isn't afraid of false positives. Consequently, it is almost unusable for bulk verifications of large trees of high-quality manuals because any signal will likely be drowned in lots of noise. .P For finding candidates of bad style or spelling in smaller sets of manuals, it is quite useful and finds whole classes of issues the two linters are completely unaware of. .P It is available as a port in both FreeBSD\*F .FS http://svnweb.freebsd.org/ports/head/textproc/igor/ .FE and OpenBSD\*F .FS http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/textproc/igor/ .FE and should also be trivial to port to any system having .Xr perl 1 . .P Running it works almost the same way as for .Xr mdoclint 1 . Simply provide the names of the files to be checked as command line arguments. To suppress unwanted messages, use command line options. eurobsdcon2014-mandoc-paper/3/gmdiff.roff010064400017500001750000000015251240314556700212030ustar00ischwarzeischwarze.H 2 "Groff-mandoc comparisons" Comparing the output of .Xr groff 1 and .Xr mandoc 1 for the same input file is an important way of finding bugs in the parsers and formatters. In some cases, such differences can also provides hints that code is of limited portability. .P Of course, if code formats differently, that doesn't necessarily imply that the code can be improved. While it's sometimes easy to spot parser and formatter bugs in such comparisons, interpreting them to identify markup of questionable quality definitely requires wide experience with the .Xr mdoc 7 language. .P A very simple but handy shell script to run such comparisons, .Nm gmdiff , is available from the portable mandoc repository.\*F .FS http://mdocml.bsd.lv/cgi-bin/cvsweb/gmdiff?cvsroot=mdocml .FE Simply provide the names of the files to check as command line arguments. eurobsdcon2014-mandoc-paper/4004075500017500001750000000000001240343075300170055ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/4/section.roff010064400017500001750000000004641240342310000213730ustar00ischwarzeischwarze.H 1 "Searching and displaying manual pages" .\" XXX kristaps@ started makewhatis/apropos on April 2, 2011 .\" XXX rewrite for SQLite3 since June 8, 2012 .so 4/traditional.roff .so 4/semantic.roff .so 4/intermediate.roff .so 4/complex.roff .so 4/outkey.roff .so 4/man.roff .so 4/cgi.roff .so 4/database.roff eurobsdcon2014-mandoc-paper/4/traditional.roff010064400017500001750000000016651240232651000222530ustar00ischwarzeischwarze.H 2 "Backward compatibility" The mandoc toolbox preserves the existing functionality of traditional search tools:\*F .FS BSDCan 2014 p. 12 .FE .BVL 1c .LI "\fBapropos\fP \fIkeywords\fP ..." Search case-insensitively for substrings in names and descriptions (working since October 19, 2013). .LI "\fBman\fP \fB\-k\fP \fIarguments\fP" As before, an alias for: .Nm apropos .Ar arguments .br But it now also supports new-style arguments, see below. .LI "\fBapropos\fP [\fB\-C\fP \fIfile\fP] [\fB\-M\fP \fIpath\fP] \ [\fB\-m\fP \fIpath\fP] [\fB\-S\fP \fIarch\fP] [\fB\-s\fP \fIsection\fP] \ \fIkeywords\fP ..." Traditional options are all supported. .LI "\fBwhatis\fP \fIkeywords\fP ..." Search case-insensitively for complete words in page names only. .LIb "makewhatis" Rebuild all configured databases. .LI "\fBmakewhatis\fP \fB\-d\fP \fIdirectory files\fP ..." Update entries for the given .Ar files in one database (working since June 3, 2013). .LE eurobsdcon2014-mandoc-paper/4/semantic.roff010064400017500001750000000023071240232714000215360ustar00ischwarzeischwarze.H 2 "Markup-sensitive search" In addition to traditional functionality, the mandoc toolbox supports searching in specific semantic contexts identified by macro keys. The following are examples of macro keys that can be searched for, ordered by frequency:\*F .FS BSDCan 2014 p. 13 .FE .VL 8n 2n 1 .LIb Nm manual page names .LIb Nd manual page descriptions .LIb sec manual section numbers .LIb arch machine architectures .LIb Xr cross references .LIb Ar command argument names .LIb Fa function argument types and names .LIb Dv preprocessor constants .LIb Pa file system paths .LIb Cd kernel configuration directives .LIb Va variable names .LIb Ft function return types .LIb Er error constants .LIb Ev environment variables .LIb In include file names .LIb St references to standards documents .LIb An author names .LI ... and so on ... .LE .P Here is an example of a search command and its results: .VERBON $ apropos Ev=USER Mail, mail, mailx(1) \(en send and receive mail csh(1) \(en a shell (command interpreter) with Clike syntax login(1) \(en log into the computer logname(1) \(en display user's login name slogin, ssh(1) \(en OpenSSH SSH client (remote login program) su(1) \(en substitute user identity [...] .VERBOFF eurobsdcon2014-mandoc-paper/4/intermediate.roff010064400017500001750000000033771240232765400224270ustar00ischwarzeischwarze.H 2 "Markup-sensitive search features" Search keys can be OR'ed by simply joining them with commas, using a common search string for all of them:\*F .FS BSDCan 2014 p. 14 .FE .VERBON 6 $ apropos \fBFa,Ft,Va,Vt=\fPtimespec EV_SET, kevent, kqueue(2) \(en kernel event notification mechanism clock_getres, clock_gettime, clock_settime(2) \(en get/set/calibrate date and time futimens, futimes, utimensat, utimes(2) \(en set file access and modification times nanosleep(2) \(en high resolution sleep parse_time(3) \(en parse and unparse time intervals poll, ppoll(2) \(en synchronous I/O multiplexing pselect, \fBselect(2)\fP, FD_CLR, FD_ISSET, FD_SET, \ \fBFD_ZERO(3)\fP \(en synchronous I/O multiplexing sem_timedwait, sem_trywait, sem_wait(3) \(en decrement (lock) a semaphore tstohz, tvtohz(9) \(en translate time period to timeout delay [...] .VERBOFF Searching across all macro keys is possible with the special .Cm any keyword: .VERBON 6 $ apropos \fBany=\fPulimit ksh, rksh(1) \(en public domain Korn shell sh(1) \(en public domain Bourne shell getrlimit, setrlimit(2) \(en control maximum system resource consumption .VERBOFF Regular expressions are supported by using the \(oq~\(cq operator instead of \(oq=\(cq since October 19, 2013: .VERBON 6 $ apropos "Nm~^[gs]et.*gid" endgrent, getgrent, getgrgid, getgrgid_r, getgrnam, getgrnam_r, setgrent, setgrfile, setgroupent(3) \(en group database operations getegid, getgid(2) \(en get group process identification getpgid, getpgrp(2) \(en get process group getresgid, getresuid, setresgid, setresuid(2) \(en get or set real, effective and saved user or group ID setegid, seteuid, setgid, setuid(2) \(en set user and group ID setpgid, setpgrp(2) \(en set process group setregid(2) \(en set real and effective group IDs [...] .VERBOFF eurobsdcon2014-mandoc-paper/4/complex.roff010064400017500001750000000022711240416412600214070ustar00ischwarzeischwarze.H 2 "Complex search queries" By default, multiple search terms are joined with OR, but the .Fl s and .Fl S options attach to the rest of the search expression with AND:\*F .FS BSDCan 2014 p. 15 .FE .VERBON 6 $ apropos \-s 1 tbl Nm=eqn deroff(1) \(en remove nroff/troff, eqn, pic and tbl constructs eqn(1) \(en format equations for troff or MathML eqn2graph(1) \(en convert an EQN equation into a cropped image neqn(1) \(en format equations for ascii output tbl(1) \(en format tables for troff .VERBOFF .bp Explicit logical AND and OR are supported: .VERBON 6 12 $ apropos Nd=gigabit \fB\-a\fP Cd=sbus gem(4) \(en GEM 10/100/Gigabit Ethernet device ti(4) \(en Alteon Networks Tigon I and II Gigabit Ethernet device .VERBOFF Precedence can be changed with parantheses: .VERBON 6 $ apropos \-s 1 terminal\ \fB\-a \e(\fP At~[1\-6] \fB\-o\fP Bx~^[12] \fB\e)\fP clear, tput(1) \(en terminal capability interface lock(1) \(en reserve a terminal reset, tset(1) \(en terminal initialization script(1) \(en make typescript of terminal session stty(1) \(en set the options for a terminal device interface tty(1) \(en return user's terminal name .VERBOFF .P Complex search queries are working since January 4, 2014. eurobsdcon2014-mandoc-paper/4/outkey.roff010064400017500001750000000014201240233045200212460ustar00ischwarzeischwarze.H 2 "Flexible output format" The names, section numbers, and architectures of the search results are always shown because these are needed to access the results with .Xr man 1 .\*F .FS BSDCan 2014 p. 16 .FE By default, .Xr apropos 1 also shows the one-line descriptions. .P With the .Fl O option, any other macro key can be shown instead: .VERBON 6 $ apropos \-O Cd wireless acx(4) \(en acx* at pci? # acx* at cardbus? an(4) \(en an* at isapnp? # an* at pcmcia? # an* at pci? ath(4) \(en ath* at pci? dev ? function ? # ath* at cardbus? dev ? function ? # gpio* at ath? athn(4) \(en athn* at cardbus? # athn* at uhub? port ? # athn* at pci? atu(4) \(en atu* at uhub? port ? atw(4) \(en atw* at pci? # atw* at cardbus? .VERBOFF The .Fl O option is available since December 31, 2013. eurobsdcon2014-mandoc-paper/4/cgi.roff010064400017500001750000000031751240366571500205170ustar00ischwarzeischwarze.H 2 "Web interface for manual search and display" The .Xr man.cgi 8 program contained in the mandoc toolbox provides the same user interface as on the command line. It has a .Xr man 1 and an .Xr apropos 1 mode, the latter using the same query syntax. .P The main additional feature of the web interface are hyperlinks generated for cross-page .RB ( Xr ) and in-page .RB ( Sx ) cross-references. The code has additional potential because it preserves semantic markup, but that's not used yet for anything except (simple) CSS formatting. .P The .Xr man.cgi 8 program uses the same directory structure and database format as the command line tools, which makes setup rather easy. Configuration instructions are provided in the .Xr man.cgi 8 manual. .P Of course, setting up a .Xr man.cgi 8 server only makes sense for providers of operating systems and of major software packages having many manuals: Do not run your own server with a copy of the manuals of your favourite system, that would merely add a risk to get outdated and confuse people. .P Special thanks goes to .I "S\('ebastien Marie" for doing an extensive security audit of the .Xr man.cgi 8 code and reporting a considerable number of security-relevant bugs that have all been fixed by now. .P Related timeline: .nf 2011 Nov 09: kristaps@ starts development of man.cgi 2012 Mar 23: first mandoc release containing man.cgi (1.12.1) 2012 Jun 08: kristaps@ starts moving the database backend to SQLite 2014 Jul 09: schwarze@ switches over man.cgi to SQLite 2014 Jul 12: new man.cgi running on the openbsd.org website 2014 Aug 10: first mandoc release containing the SQLite-based man.cgi (1.13.1) .fi eurobsdcon2014-mandoc-paper/4/database.roff010064400017500001750000000136471240416440000215100ustar00ischwarzeischwarze.H 2 "Database implementation" The old .Pa whatis.db was a plain text file.\*F .FS BSDCan 2014 p. 17 .FE Now we need a structured database, but a client-server model would be overkill and merely a hindrance. So SQLite\*F .FS http://www.sqlite.org/ .FE was the logical choice. .P The database contains four tables. They contain, respectively, one record ... .VL 10n 2n 1 .LI \fBmpages\fP \&... per physical page, containing the description .LI \fBmlinks\fP \&... per file system entry, containing section, architecture, filename .LI \fBnames\fP \&... per page name; for all manual page names, not just file names .LI \fBkeys\fP \&... per key=value pair .LE .P The mlinks table has full support for: .BL \n(Pi 1 .LI hard links since December 27, 2013 .LI symbolic links since April 18, 2014 .LI redirections using the roff .so request since March 19, 2014 (used by X.org) .LE . .H 2 "Search algorithm" Each search requires either two or three queries:\*F .FS BSDCan 2014 p. 18 .FE .AL .LI \f(CWSELECT FROM mpages\fP .br to find the pages to be displayed. .P Conveniently, searching for descriptions is fastest \(em in the first step, access one single table only, finding the .Ar pageid . Searching for names is the second in speed \(em it requires only a simple JOIN to a small table. .LI \f(CWSELECT FROM names\fP .br to find the page names to be displayed. .P This is very fast because it is just a simple SELECT in a small table using the indexed .Ar pageid . .LI \f(CWSELECT FROM keys\fP .br to find the values to be displayed. .P Only needed when .Fl O is given. Otherwise, we already have the description from search step 1. Very fast, too, just another simple SELECT indexed by .Ar pageid . .LE . .H 2 "Optimization" As usual, optimization is not a well-defined task in the mathematical sense.\*F .FS BSDCan 2014 p. 19 .FE We want high speed and small size: .BL \n(Pi 1 .LI a small database .LI a short database build time .LI low apropos memory consumption .LI short search times .LE .P Of course, these optimization goals conflict. .P The gprof(1) profiler was used a lot. . .bp .H 3 "Search speed optimization" .BL .LI Moving the .B descriptions from the keys table directly into the mpages table gained roughly a .B "factor 4" in speed for searches by description \(em at no cost, the database shrank, too, due to reduced pageid overhead (April 9, 2014).\*F .FS BSDCan 2014 p. 20 .FE .LI The dedicated .B names table gained roughly a .B "factor 4" in speed for searches by name \(em at almost no cost (April 9, 2014). .LI Adding an index to the .B mlinks table sped up the second step in the algorithm, name retrieval, by about a factor of 20, which resulted in an overall .B "30% economy" for simple searches, and more for searches returning many results. The cost was a 10% growth of the database (April 16, 2014). .LI Adding an index to the .B keys table sped up the third step in the algorithm, .Fl O value retrieval, which is dominant in .Fl O searches, resulting roughly in a .B "factor 4" speedup of such searches. The cost was another 10% grow of the database. .LI By providing an SQLITE_CONFIG_PAGECACHE with .Xr mmap 3 MAP_ANON, execution time decreased by 20\-25% for simple (Nd and/or Nm) queries, 10\-20% for non-NAME queries, and even .Xr apropos 1 resident memory size decreased by 20% for simple and by 60% for non-NAME queries. Cache size is a compromize to provide nearly optimal speed gain for all queries while limiting additional memory consumption to about 15% (April 11, 2014). .LE . .H 3 "Database build time optimization" This is relevant because .Xr mandoc.db 5 is built during regular base system and Xenocara snapshots builds on all architectures, and we don't want to slow down developers during development and testing.\*F .FS BSDCan 2014 p. 21 .FE .BL .LI Quick mode: Abort parsing after the NAME section: factor 2 in speed and factor 4 in size (January 5, 2014). In the following, all speedups refer to quick mode. .LI Do not sync to disk after each individual manual page, only sync to disk one single time when all data is ready: 87% (January 6, 2014). .LI In quick mode, do not clear user-defined macros clashing with .Xr mdoc 7 or .Xr man 7 standard macros when parsing .Dd or .TH: 25% (January 6, 2014). .LI In quick mode, do not validate and normalize the date format: 18% (January 6, 2014). .LI Do not copy predefined strings into the dynamic string table: 10% (January 6, 2014). .LI Cache .Xr uname 3 result: 3% (January 7, 2014). .LI Do not index the keys in the keys table: 12% (and 42% size reduction) (January 18, 2014). .LI No primary keys in the mlinks and keys table: 15% (and 3% size) (January 18, 2014). .LI Properly handling .so redirections reduced Xenocara database build time by 40% and database size by nearly 50% (March 19, 2014). .LE . .bp .H 3 "Database size optimizations" All size reductions refer to quick mode.\*F .FS BSDCan 2014 p. 22 .FE .BL .LI Do not store the descriptions twice, once for searching and once for display, at the expense of somewhat more complicated, but not slower search code: 9% (January 5, 2014). .LI Remove the redundant "file" column from the mlinks table: 9% (January 5, 2014). .LI Sort macro keys by frequency: 11% (January 18, 2014). .LI Always store the arch in lower-case only: 1.5% (January 19, 2014). .LE . .H 3 "Search and database performance summary" With the old, plain text .Xr apropos 1 , a simple search took about 10 milliseconds on my notebook.\*F .FS BSDCan 2014 p. 23 .FE With the new, SQLite .Xr apropos 1 , it is unavoidably slower due to the SQL overhead and because the names are now separated from the descriptions. It now takes about 40 milliseconds. However, the difference is of no practical relevance even on moderately old hardware. .P Base system database size grows from 250 kB to 900 kB (quick mode) or about 3800 kB (fully featured mode). That is not a practical problem for any of our architectures. During system builds, database build times are reduced by roughly a factor 3 with respect to the old Perl .Xr makewhatis 8 . eurobsdcon2014-mandoc-paper/4/man.roff010064400017500001750000000156651240416422200205230ustar00ischwarzeischwarze.H 2 "Unified user interface including man(1)" The very latest development is that the mandoc toolbox now provides an implementation of .Xr man 1 and a unified interface for .Xr mandoc 1 , .Xr man 1 , .Xr whatis 1 , and .Xr apropos 1 . That is, all the command line options of all these utilities are available in all of them and have the same meaning everywhere, and almost all functionality is available from all command names, even though the default behaviour of the different command names is still different. .H 3 "How it works" The new unified main program always uses a five-step process: .AL .LI Decide how to interpret the command line arguments. .LI Build a list of manual pages, usually from a database search. .LI Decide which kind of output to provide. .LI Optionally spawn a pager. .LI Loop around the list of manual pages, producing some output for each. .LE .bp .P Some .B "input options" are available to specify the meaning of the command line arguments: .VL 6n 2n 1 .LIb \-l Interpret each command line argument as an (absolute or relative) filename. No database search is done. This is the default when called as .Nm mandoc . .LI * The default when called as .Nm man without an input option is to interpret each command line argument as a name and require exact matches (as opposed to word, substring, or regular expression matches). The default output mode in this case is to show exactly one manual considered the best match. Currently, there is not (yet?) any option to force this behaviour, so it's only available when called as .Nm man for now. .LIb \-f Interpret each command line argument as a name and match against complete words (as opposed to substring or regular expression matches). This is the default when called as .Nm whatis . .LIb \-k Support the full .Xr apropos 1 search syntax. This is the default when called as .Nm apropos . The default output mode in these two cases is to display a list of names, section numbers, and description lines of matching manuals. .LE .P Some .B "database selection" options only matter when .Fl l is not active: .VL 6n 2n 1 .LIb \-C Use the specified .Ar file instead of the default configuration file. .LIb \-M Use the specified .Ar path instead of the default one. Do not use any configuration file. .LIb \-m Use the specified .Ar path in addition to the default one. .LIb \-S Restrict the search to the specified .Ar architecture . .LIb \-s Restrict the search to the specified .Ar section . .LE .P Some .B "output options" are available to specify which kind of output to provide: .VL 6n 2n 1 .LIb \-a Display all matching manual pages, one after the other. This is the default for .Fl l input mode. .LIb \-h Display only the SYNOPSIS lines of the matching pages. Implies .Fl a . .LIb \-O When showing a list in .Fl f or .Fl k mode, display the specified macro key instead of the .Cm Nd one-line descriptions. .LIb \-w Display only the pathnames of the matching manual pages. .LE .P Some .B "parser and formatter options" only take effect when a parser is actually run: .VL 6n 2n 1 .LIb \-I Override the default operating system name for the .Xr mdoc 7 .Cm Os macro. .LIb \-m Specify the input format. Defaults to .Fl mandoc , requesting autodetection. .LIb \-O Comma-separated formatter-specific output options. .LIb \-T Select the output format. Defaults to .Fl Tascii . .LIb \-W Specify the minimum message .Ar level to be reported on the standard error output and to affect the exit status. Defaults to .Fl Wfatal . .LE .P Finally, the .Fl c option can be used to suppress the pager and just copy the formatted manuals to standard output. .H 3 "How this came about" Four months back at BSDCan 2014 in Ottawa, i presented a slide "possible future directions". This project was .I not listed. I didn't expect myself that i would do this. .P But then, on August 9 this year (less than two months ago now) Paul Onyschuk of Alpine Linux (which is the first Linux distro that integrated mandoc, in July 2010) asked me: \(lqAre there any plans for providing a .Xr man 1 command also? This would make mdocml a possible, standalone replacement for the groff and man-db combination (typical in Linux distributions).\(rq .P Almost, i returned my standard negative answer, but then i stopped short and realized that almost all the needed code was already there and it cheaply allows doing fancy things without complexity. I had to do the mandoc 1.13.1 and 1.12.4 releases first, so it took two weeks from the idea to the first working implementation... .H 3 "What a \(oqname\(cq is" This question is critical because the .Xr man 1 command is supposed to display manual pages whose names exactly match the command line arguments. The traditional .Xr man 1 utility only uses filenames as names in this sense. .P When .Xr makewhatis 8 runs, it adds names to the .Va names table in the .Xr mandoc.db 5 databases, noting in the .Va names.bits column of that table where the name came from .RB ( Dt / TH header line, .Cm Nm in the NAME section, .Cm Nm in the SYNOPSIS section, file name). It also saves the file names where stuff was found into the .Va mlinks table and links both tables together (and to the .Va keys table with all the search terms) via the .Va mpages table. .P For .Xr man 1 mode, right now, all types of names are used. That can easily be tuned, and probably it should. .BL \n(Pi 1 .LI The name from the .BR Dt / TH is probably not all that useful for this purpose because it lacks the case information. .LI The .Cm Nm macros from the NAME section should almost certainly be used. That way, all hard links, symbolic links, and .so link files become obsolete. Consequently, the number of files in a typical operating system installation can be reduced by more than three thousands. .LI The .Cm Nm macros from the SYNOPSIS section should probably not be used for this purpose. .LI The file names should probably still be used, just in case someone installs manuals the old way and screws up the name sections. .LE .H 3 "Which issues remain" .BL .LI The .Fl i (interactive) option hasn't been integrated yet, that code is still kept separately in the .Xr manpage 1 utility which was never used in any kind of system integration. .LI Most .Xr man.conf 5 features are not supported. The following types of lines are completely ignored: .Cm _build , .Cm _default , .Cm _subdir , .Cm _suffix , and .Ar section lines. The search order 1, 8, 6, 2, 3, 5, 7, 4, 9 is hardcoded. Instead of .Cm _default , the .Cm _whatdb lines are used. All subdirectories are hardcoded to \f(CW{man,cat}N\fP. .LI When finding a formatted and an unformatted manual of the same name in the same section, the old .Xr man 1 shows the one that was less recently changed. The mandoc .Xr man 1 currently always prefers the unformatted version, even if it's older than the formatted version. .LI The mandoc .Xr man 1 may show some additional results, and some of those may be bogus, if NAME sections contain bogus .Cm Nm macros. .LI The mandoc .Xr man 1 ignores the MACHINE environment variable, and i'm not planning to add support for it. .LE eurobsdcon2014-mandoc-paper/5004075500017500001750000000000001240344412200170015ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/5/section.roff010064400017500001750000000004041240344520000213730ustar00ischwarzeischwarze.H 1 "Integrating mandoc as a base system documentation formatter" .so 5/start.roff .so 5/fatal.roff .nr Cl 3 .so 5/errors.roff .so 5/pod2man.roff .so 5/tbl.roff .so 5/roff.roff .nr Cl 2 .so 5/compare.roff .so 5/local.roff .so 5/switch.roff .so 5/lessons.roff eurobsdcon2014-mandoc-paper/5/start.roff010064400017500001750000000104501240416446300211000ustar00ischwarzeischwarze.H 2 Preparations .H 3 "Preliminary note" More than three years ago, when summarizing the status of the groff to mandoc switch at BSDCan 2011, i already said this:\*F .FS BSDCan 2011 p. 20 .FE .P In OpenBSD, we now reached the stage: It just works. To get there, we .BL .LI re-implemented a small family of languages. .LI turned the whole paradigm of these languages upside down. .LI remained compatible where it matters. .LI flouted compatibility that would just hinder. .LE .P Other systems can now do the same, if they want: .BL .LI without much risk. .LI without having to fear major surprises. .LI getting support from us in case of need \(em just mail us! .LE .P Meanwhile, NetBSD and illumos have switched from groff to mandoc, too, in 2012 and 2014, respectively. So you certainly don't have to fear the switch, nowadays. Just follow the plan laid out below. . .H 3 "Getting started" The first step is to import the mandoc codebase into the base repository of the target operating system, collect some initial experience using it, and enable it to be built and installed by default. .P The most important aspect of this phase is to establish the contact to everybody in the operating system's developer community who is interested and might wish to be involved. Get familiar with specific requirements of the system and any specific topics the developers are concerned about. Do not neglect this aspect or it will bite you later. .P This phase also provides a great opportunity to find bugs in mandoc, to report them, and to get them fixed, without any of the stress that might arise from already being in production, or even from having made specific plans when to go into production. . .H 3 "Case study: OpenBSD" In OpenBSD, this phase lasted from June to December 2009.\*F .FS BSDCan 2011 p. 5 .FE Because mandoc has matured since that time, all this ought to be much easier now. In any case, this phase can be shorter. Start the next phase early, it will help you to find bugs and to focus on the most relevant ones.\*F .FS BSDCan 2011 p. 10 .FE .P In OpenBSD, we also used this phase to start a regression suite.\*F .FS BSDCan 2011 p. 22 .FE At first, naturally, we encountered lots of new bugs and few regressions. Consequently, automatic tests were not a priority. However, we started collecting anyway. As the code matured, the number of tests slowly increased, and the suite slowly became more important. Maintenance was done at intervals to keep the suite neat and tidy as it grew. .P Already in 2011, the suite regularly caught issues when doing major merges. It took less than two years for the continuous effort to start to pay off. By now, the suite is an indispensable tool used whenever applying any changes to the mandoc codebase. .bp .P Having that suite available on your system would definitely be helpful. Unfortunately, a portable version of the regression suite does not yet exist. Setting it up would be a worthwhile project. Ideally, the tests should not be copied; they would quickly get outdated and require constant maintenance. Rather, one should try to provide Makefile fragments (or something similar) to help the OpenBSD suite to run on other systems, at least on those having a BSD make(1). .P .nf Related OpenBSD timeline: 2008 Nov 22: first commit to mdocml.bsd.lv by kristaps@ 2009 Mar 27: first direct commit by schwarze@ to OpenBSD (not mandoc) 2009 Apr 06: mandoc imported into OpenBSD by kristaps@ 2009 Apr 15: first help from another OpenBSD developer (miod@) 2009 May 23: schwarze@ first talks to jmc@ about mandoc 2009 May 31: at c2k9 in Edmonton, schwarze@ talks to deraadt@ 2009 Jun 09: kristaps@ agrees to work closely together 2009 Jun 14: merge to OpenBSD started by schwarze@ 2009 Jun 15: first patches merged back from OpenBSD to bsd.lv 2009 Jun 21: mandoc usable in OpenBSD and in sync with bsd.lv 2009 Jun 23: bugfixing in OpenBSD started by schwarze@ 2009 Jul 05: OpenBSD 4.6 release rolled without mandoc 2009 Jul 12: joerg@ sends his first patch from NetBSD 2009 Jul 18: uqs@ sends his first patch from FreeBSD 2009 Oct 27: start src/regress/usr.bin/mandoc 2010 Jan 02: start of systematic integration 2010 Jun 30: major update of the mandoc test suite 2010 Jul 01: enable mandoc regression tests; ok phessler@ 2010 Dec 04: major additions to the regression suite 2011 Feb 05: commit many regression tests found in my trees .fi eurobsdcon2014-mandoc-paper/5/fatal.roff010064400017500001750000000024341240073176300210340ustar00ischwarzeischwarze.H 2 "Identify fatal issues" Try to build all manuals of the target operating system with mandoc. Ignore all non-fatal issues for now. Report all fatal errors to the mandoc developers. Have them fixed upstream or devise workarounds. We are glad to provide help! Only as a last resort, change the affected manuals in your tree.\*F .FS BSDCan 2011 p. 10 .FE .H 3 "Case study: OpenBSD" In OpenBSD, this phase took place in January and February 2010.\*F .FS BSDCan 2011 p. 6 .FE Since mandoc hardly throws any fatal errors any longer, this phase ought to be much easier nowadays. None of the fatal errors that held us back in OpenBSD in 2010 and that i listed in my 2011 BSDCan talk is still fatal today. The mandoc parser is now able to recover from all of them and keep going, so they could all be downgraded to non-fatal errors and some even to mere warnings. .P .nf Related timeline: 2010 Jan 02: first patches to mdoc(7) manuals to fix the build with mandoc "Fine. Even if mandoc goes nowhere, it has found some bugs. ;)" jmc@ 2010 Feb 17: first patch to a man(7) manual in order to fix the build 2010 Feb 20: found first manual bug caused by DocBook 2010 Feb 24: first non-fatal manual fix found by -Tlint 2010 Feb 25: tree now builds with mandoc 2010 Mar 18: OpenBSD 4.7 release rolled without mandoc .fi eurobsdcon2014-mandoc-paper/5/pod2man.roff010064400017500001750000000042011240415200600212660ustar00ischwarzeischwarze.H 3 "Supporting the pod2man(1) preample" The .Xr pod2man 1 utility is one example of a documentation format translation program.\*F .FS BSDCan 2011 p. 9 .FE It takes documentation in the .Xr perlpod 1 format, which is an acronym for "plain old documentation", and translates it to .Xr man 7 code. Every manual page written by .Xr pod2man 1 starts with a document preamble of low-level .Xr roff 7 code. The preamble varies from one Perl version to another, but is the same for every manual written by the same Perl version. .P Here is some example code from a .Xr pod2man 1 preamble: .VERBON $ less perl.1 [...] \&.de Sp \" Vertical space (when we can't use .PP) \&.if t .sp .5v \&.if n .sp \&.. \&.de Vb \" Begin verbatim text \&.ft CW \&.nf \&.ne \\$1 \&.. \&.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .VERBOFF .P This code uses various low-level roff requests: strings definitions (.ds), macro definitions (.de), conditional requests (.if) and so on. .P When integrating .Xr mandoc 1 into OpenBSD in 2010, there was no chance to get all that implemented quickly. So we went for a temporary, quick and dirty solution: We did not implement the low-level roff requests being used, but instead added explicit support for the strings and macros this low-level code defines. The strings were hardcoded, the macros implemented as if they were .Xr man 7 macros, and the rest of the low-level instructions were merely parsed an ignored. That was sufficient because the preamble is always the same. .P That way, it was possible to get all .Xr man 7 manuals to work with .Xr mandoc 1 , without changing any of the manuals in any way. .P About half a year later, support for string and macro definitions was implemented and the temporary workarounds could be removed. So today, you won't encounter the same issues again, but of course, you may run into other .Xr roff 7 features unimplemented in .Xr mandoc 1 that other .Xr man 7 code generators happen to use. . .H 4 "Case study: OpenBSD" Related timeline: .nf 2010 Mar 01 implement pod2man(1) pseudo-macros 2010 Sep 20 mandoc can now handle the standard pod2man preamble 2010 Nov 28 remove the pod2man(1) pseudo-macros .fi eurobsdcon2014-mandoc-paper/5/errors.roff010064400017500001750000000030571240416447500212670ustar00ischwarzeischwarze.bp .H 2 "Investigate non-fatal errors" Run .Nm mandoc .Fl Tlint .Fl Werror on all manuals in your system. Prioritize fallout that ruins content or formatting. Distinguish mandoc bugs from markup bugs. Report mandoc bugs upstream. .P The most serious issues showing up in this phase typically do so in .Xr man 7 manuals, not in .Xr mdoc 7 manuals. This is not a coincidence. While the .Xr mdoc 7 language is powerful enough to express all formatting that might reasonably be wished for in a manual page, the same is not true for the .Xr man 7 language. Even in a well-written .Xr man 7 manual, it is unavoidable to resort to the use of some low-level .Xr roff 7 formatting instructions. .P Many lower-quality .Xr man 7 manuals, in particular those generated from other input formats by automatic translation tools, use elaborate low-level roff constructions, some of which .Xr mandoc 1 may not yet support. Most of these will show up as "unknown macros" in .Nm mandoc .Fl Tlint output, and output formatting may or may not be degraded. Please report those "unknown macro" errors you find in real-world manuals to the .Xr mandoc 1 developers \(em unless they are mere typos in request or macro names, of course. .P The remaining subsections of this section discuss particular classes of "unknown macro" errors encountered when integrating .Xr mandoc 1 into OpenBSD. The problems discussed here were solved in 2010. So if you are only interested in integrating into a new platform, and not that much in .Xr mandoc 1 architecture and development, you can safely skip these subsections. eurobsdcon2014-mandoc-paper/5/compare.roff010064400017500001750000000034551240233634500213760ustar00ischwarzeischwarze.H 2 "Check mandoc output" For the manuals in your tree, systematically compare .Xr groff 1 and .Xr mandoc 1 output. The .Nm gmdiff tool in the mdocml.bsd.lv repository can help with that. To reduce noise to manageable levels, look at the patches to textproc/groff in OpenBSD ports. For example, they disable adjustment and hyphenation, make page header lines agree with mandoc, and remove the special handling of the .Pa macro in the FILES section. The comparison will not be quite easy because there are still some trivial differences, most of them regarding whitespace. That is, mandoc and groff output will not be completely identical, but try to make sure no content gets lost and no formatting is completely garbled. .P If you find any serious issues, report them, in particular if .Xr mandoc 1 fails to flag them as ERRORs. Patching manuals is usually not the right approach in this phase. .P Quickly move on to the next phase, that is, when you are convinced there are no show-stopper issues, not when you feel everything is perfect.\*F .FS BSDCan 2011 p. 10 .FE Your system will mature best when it's enabled by default and when you get and use real-world feedback. . .H 3 "Case study: OpenBSD" In OpenBSD, such comparisons were done as an iterative process in several cycles:\*F .FS BSDCan 2011 p. 16 .FE Comparisons were first run on a small part of the tree. Some parsing and formatting issues were identified. They were prioritized by a rough, inexact estimate of frequency and severity. Some of them were then fixed in .Xr mandoc 1 : Sufficiently few that we didn't lose too much time; sufficiently many to significantly reduce the noise. Then, we started over with a larger part of the tree. .P Related timeline: .nf 2010 Aug 15 systematic bug hunting in /bin and /sbin 2011 Jan 23 systematic bug hunting in /usr/bin .fi eurobsdcon2014-mandoc-paper/5/switch.roff010064400017500001750000000052471240233663500212540ustar00ischwarzeischwarze.H 2 "Switch over to build with mandoc" Once .Xr mandoc 1 is ready to build all of your tree and no show-stopper issues are left, you can do the switch to use it instead of groff for building your manuals. Choose the timing wisely. Make sure there is plenty of time until the next release. Rushing this change in shortly before a release would be a bad idea. Formatting quality of manual pages is not among the things people focus on during release testing, so chances are issues you overlooked won't be found before release, and once the release is out of the door, you end up in a rather awkward situation that's hard to fix. .P After switching the build, watch out for bug reports from users of the -current code. Report and fix bugs as quickly as possible. .P The switch itself can be done in one or two steps. OpenBSD did it in two steps: The formatter was switched from groff to mandoc in April 2010. Source manuals are installed instead of preformatted manuals since October 2010. .P NetBSD did it in one step in February 2012, switching both the build system to install unformatted instead of formatted and changing the manual formatter from groff to mandoc in .Xr man.conf 5 at the same time. If you feel confident you can manage the one-step process, i recommend you follow the example of NetBSD. Nowadays, .Xr mandoc 1 is mature enough to rely on it as a run-time formatter right away, assuming you have done the necessary testing for your particular system as described in the last few sections. . .H 3 "Case study: OpenBSD" Mandoc was ready for production right before the OpenBSD 4.7 release was tagged.\*F .FS BSDCan 2011 p. 11 .FE The build was switched over right after the tree unlocked after the release, so there was as much time as possible to fix fallout. Bug reports from real-world users started coming in at once. Priority was given to these bug reports. All serious fallout fixed within a few days. Lots of time was left for polishing before the next release. .P Related timeline: .nf 2010 Mar 01: mandoc ready to build the tree 2010 Mar 18: OpenBSD 4.7 release rolled without mandoc 2010 Mar 20: Xenocara can now build with mandoc as well 2010 Apr 02: link mandoc to the OpenBSD build 2010 Apr 03: switch base build to mandoc, excepting tbl pages 2010 Apr 03: fix all fatal issues where mandoc kills ports builds 2010 Apr 04: first port maintainer explicitely switches to mandoc 2010 Apr 04: fix first mandoc bug that was found by ports usage 2010 Apr 05: espie@ implements USE_GROFF framework and groff-1.15 port 2010 Apr 07: first major merge from bsd.lv after the switch 2010 Apr 08: first mandoc bugfix found in ports propagates upstream 2010 Aug 12: OpenBSD 4.8 release rolled with mandoc .fi eurobsdcon2014-mandoc-paper/5/local.roff010064400017500001750000000025441240233651000210320ustar00ischwarzeischwarze.H 2 "Watch out for local features!" The major BSD trees forked from each other about two decades ago, and one copy of groff has dwelt in each of them for that long time. Chances are it has been locally patched at least in some way, and the local manuals rely on such patches. The manuals in a BSD tree might also use internal features of whatever groff version may be around. .P When trying to switch from groff to mandoc, you will probably bump into such features because mandoc is not likely to support whatever localisms have developed in your tree. .P If you can easily patch such features away in your manuals, you should probably do that because it will make your manuals more portable, not just to mandoc, but even to other breeds of groff. However, that may not be possible. In that case, please talk to the mandoc developers, and we will see what we can do to rescue your local quirks. . .H 3 "Case study: OpenBSD" Such home-grown non-standard features exist even in OpenBSD.\*F .FS BSDCan 2011 p. 15 .FE The worst one is related to SYNOPSIS formatting: An internal roff register that is specific to the pre-1.17 groff implementation is used to switch on and off SYNOPSIS mode in kernel manuals having the synopsis split into several parts. We had no reasonable choice but to support this in .Xr mandoc 1 , even though what these manuals do is an incredibly dirty hack. eurobsdcon2014-mandoc-paper/5/tbl.roff010064400017500001750000000041211240367072500205240ustar00ischwarzeischwarze.H 3 "The mandoc(1) tbl(7) implementation" Another area where issues might surface is in manuals using the .Xr tbl 7 language to format tables. This language is used in a relatively small number of pages, so it never received as much attention as the .Xr mdoc 7 and .Xr man 7 languages. All the same, most tables now format just fine, so you may get away without issues in this area. If you do run into unimplemented features or formatting bugs in your tree, please report to the mandoc developers. . .H 4 "Case study: OpenBSD" In the OpenBSD base system, only a dozen pages use .Xr tbl 7 .\*F .FS BSDCan 2011 p. 18 .FE When switching the build of all other pages to mandoc, we at first continued to format this dozen with groff because mandoc did not have .Xr tbl 7 support at the time. .P We long avoided the task of dealing with this handful of pages, even though Kristaps had written a stand-alone implementation of .Xr tbl 7 months before. This concerned only a very small number of manual pages, and it was unclear how to best integrate the .Xr tbl 7 parser and formatter into mandoc. Finally, I deliberately chose the minimal route: I hooked the code directly into the high-level parsers. That way, I got a working integration within a single weekend and immediately relied on it for production. .P Kristaps implemented a better way later: Parse .Xr tbl 7 block macros on the roff level, then call the .Xr tbl 7 parser from the main program just like in the case of .Xr mdoc 7 and .Xr man 7 . The same approach was later used for rudimentary .Xr eqn 7 support. .P Related timeline: .nf 2010 May: stand-alone implementation of tbl started by kristaps@ 2010 Aug 12: OpenBSD 4.8 release rolled with mandoc 2010 Oct 15: import tbl parser and renderer written by kristaps@ 2010 Oct 17: build tbl(1) pages with mandoc(1), not groff 2010 Oct 18: disconnect groff from the base build 2010 Oct 18: "I absolutely don't intend to merge tbl into mandoc" kristaps@ 2011 Jan 04: clean tbl integration by kristaps, remove mine 2011 Mar 20: rudimentary eqn support by kristaps@ 2011 Jul 24: complete basic support for equation blocks .fi eurobsdcon2014-mandoc-paper/5/lessons.roff010064400017500001750000000053171240416462400214360ustar00ischwarzeischwarze.bp .H 2 "Lessons learnt from the replacement project" .H 3 "Bad patches triggering good ones" During the replacement project in OpenBSD, we put preliminary code into production on multiple occasions and later on ripped it out again.\*F .FS BSDCan 2011 p. 23 .FE That may seem inefficient at first, but actually it's a perfectly sane approach: The first implementation explores the feature. The second implementation gets it right. Just don't let the first one sprawl until it can't be ripped out any more. .P This approach got used in at least five cases: .nf 2010 Mar 01 - May 14: end of sentence detection 2010 Apr 25 - May 19: roff conditionals 2010 Mar 01 - Sep 20: pod2man preamble 2010 Jun 16 - Jun 27: roff registers 2010 Oct 15 - Jan 04: tbl integration .fi . .H 3 "Clean design works even for dirty languages" For the reasons explained in the introductory section of this paper, the .Xr mdoc 7 language is the best tool available for writing documentation, and many attempts to design something better have failed. So, .Xr mdoc 7 is an excellent tool for its job, but that doesn't imply much about the quality of the language from a software engineering perspective. .P Actually, the design of languages like .Xr mdoc 7 , .Xr man 7 , .Xr roff 7 , .Xr tbl 7 , .Xr eqn 7 isn't the latest and greatest in software engineering. After all, the youngest of these languages is now about 25 years old, and the concepts have evolved (and required some compatibility) for another 25 years before that. .P All the same, the mandoc project has shown that it is possible to design and implement a relatively clean compiler even for a set of somewhat dirty languages. .P We started coding with the nice high-level stuff.\*F .FS BSDCan 2011 p. 24 .FE That gave us a very clean overall design. We edged in low-level ugliness later, only where it is required. It has proven possible to edge in low-level features even while the tool was already in production, but only because we kept all parts small and simple. In a large and complex system, changing the basic design in an afterthought would no doubt break the system. .P Two reasons to shun complexity are well known: It is the enemy of correctness and security. However, here we have seen that keeping down complexity is also critical for flexibility, which is a third reason that i guess fewer people are aware of. . .H 3 "Move fast!" A replacement project has the best chances to succeed if you quickly put your work to real-world use.\*F .FS BSDCan 2011 p. 25 .FE Do not let it rot in a corner of the OS. Keep moving fast, do not fear change. .P Only make sure you don't trap your users with incompatible changes. You won't find users when you break behaviour. And you won't find bugs when you don't have users. eurobsdcon2014-mandoc-paper/5/roff.roff010064400017500001750000000110001240416455000206640ustar00ischwarzeischwarze.H 3 "The design of mandoc(1) beyond mdoc(7)" The .Xr mandoc 1 program is not only intended as a formatter for the .Xr mdoc 7 language, but aims to format all contemporary .Xr man 7 manuals, too, and even historical ones back to AT&T Version 7 UNIX. .P Kristaps' original design intended to forget about the complete bottom layer and only implement the high-level .Xr mdoc 7 and .Xr man 7 macros.\*F .FS BSDCan 2011 p. 12 .FE However, since all .Xr man 7 manuals use at least some low-level .Xr roff 7 features, and many use more than they should, it turned out that does not work. There is no way around implementing some low-level roff requests. .P Realizing that was a slow, painful process that dragged out over many months. Finally, we reluctantly edged in some low-level roff support. Complexity was kept to the bare minimum. .P Both steps have proven to be just right: Starting from the high level gave a clean design. Our reluctance to support low-level .Xr roff 7 prevented us from getting off track. After we had put in some low-level roff support as a late addition, we were surprised how little change was required to the overall design. .bp .P The original mandoc main program looked like this:\*F .FS BSDCan 2011 p. 13 .FE .BL \n(Pi 1 .LI main loop to read input files .LI push line after line into the parser backends .LI parsers look for high-level macros, e.g. mdoc(7) .LI call the formatting frontend on the resulting syntax tree .LE .P When edging in low-level roff support, this design was changed as follows: .BL \n(Pi 1 .LI main loop to read input files .LI .B "call the roff preprocessor on each line" .LI push line after line into the parser backends .LI parsers look for high-level macros, e.g. mdoc(7) .LI call the formatting frontend on the resulting syntax tree .LE .P So there is a stunning paradigmatic switch. Classical roff implementations first expand all high-level macros into low-level requests, then pass the low-level requests into the formatters. All structural information is lost long before the main parser. By contrast, mandoc first handles the low-level requests in a preprocessor, then passes the remaining high-level code to the parsers. Structural information is kept even if it is originally intermixed with low-level roff code. .P Here is a list of .Xr roff 7 requests and their status:\*F .FS BSDCan 2011 p. 14 .FE .BL .LI mostly complete implementations in 2011: .BL \n(Pi 1 .LI string definitions: .Cm ds .LI macro definitions: .Cm de .Cm rm .LI source file inclusion: .Cm so .LE .LI additional mostly complete implementations today: .BL \n(Pi 1 .LI string definitions: .Cm as .Cm tr .LI macro definitions: .Cm am .Cm ami .Cm dei .LI conditionals: .Cm if .Cm ie .Cm el .Cm ig .LI register definitions: .Cm nr .Cm rr .LI input line traps: .Cm it .LI changing the control character: .Cm cc .LE .LI ignored requests: .BL \n(Pi 1 .LI adjustment: .Cm ad .Cm ce .Cm ta .LI spacing: .Cm ne .Cm ns .Cm pl .LI hyphenation: .Cm hy .Cm nh .Cm hw .LI character formatting: .Cm ps .Cm fam .LE .LE .P The many roff requests not mentioned here are still unimplemented, but rarely occur in real-world manuals. .P Related timeline: .nf 2010 Apr 25: implement roff conditional request for man(7) only 2010 May 15: mandoc roff library started by kristaps@ 2010 May 19: the roff library replaces my preliminary support for conditionals 2010 Jul 03: rudimentary implementation of user-defined strings 2010 Sep 22: interesting commit message: no hope for .de 2010 Nov 25: implement the .de request (define macro) 2010 Dec 01: implement the .so request (include source file) 2011 Jan 16: implement the .rm request (remove macro) 2011 Jul 28: implement the .tr request (translate characters) 2012 May 31: implement the \ez escape sequence (zero advance) 2012 Jun 12: implement the .cc request (change control character) 2013 Mar 21: implement number registers (Christos Zoulas, NetBSD) 2013 Apr 03: support simple numerical conditions (Christos Zoulas, NetBSD) 2013 Jul 13: implement the .it request (input line trap) 2013 Dec 15: implement increment and decrement in .nr 2014 Jan 22: implement the \e: escape sequence (optional line break) 2014 Feb 14: implement the .as request (append to string) 2014 Mar 08: implement string comparison in conditionals 2014 Mar 30: implement the .ll request (output line length) 2014 Apr 05: implement the .rr request (remove register) 2014 Apr 07: implement indirect references in string expansion 2014 Apr 07: almost complete implementation of numerical expressions 2014 Jul 07: implement .dei and .ami (indirect macro definitions) .fi eurobsdcon2014-mandoc-paper/6004075500017500001750000000000001240234324400170045ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/6/section.roff010064400017500001750000000002621240234323100213760ustar00ischwarzeischwarze.H 1 "Integrating mandoc as a ports documentation formatter" .so 6/problem.roff .so 6/solution.roff .so 6/evolution.roff .so 6/makewhatis.roff .so 6/move.roff .so 6/summary.roff eurobsdcon2014-mandoc-paper/6/move.roff010064400017500001750000000057501240416503500207150ustar00ischwarzeischwarze.H 2 "Moving groff to ports" This move requires two prerequisites: .AL .LI All manual pages in the base system need to format properly with mandoc, and .Xr man.conf 5 must have been switched over to use mandoc to format manuals by default. .LI The USE_GROFF infrastructure must be available in the ports framework, and all ports that have not been checked to work with mandoc must have the USE_GROFF Makefile variable set.\*F .FS BSDCan 2011 p. 19 .FE .LE . .H 3 "Case study: OpenBSD" In OpenBSD, the move of groff from base to ports was done right before the 2010 ports hackathon, so there was plenty of time to deal with fallout. And indeed, it turned out that was needed. .AL .LI With respect to item 1 above, we were late to realize that the X11 manuals use the roff .Cm so (file inclusion) request. It was implemented in a hurry after the move was already done, so X11 manuals were briefly broken. .LI With respect to item 2 above, the ports tree and the groff port had already been prepared by Marc Espie. Several porters immediately started moving over ports to mandoc on a case by case basis. Consequently, several bugs were reported in mandoc. Once again, we gave priority to fixing such real-world issues. .LE .bp .P Related timeline: .nf 2010 Aug 12: OpenBSD 4.8 release rolled with mandoc 2010 Oct 18: disconnect groff from the base build 2010 Oct 19: switch default /etc/man.conf to mandoc 2010 Oct 23: schwarze@ at p2k10 hackathon in Budapest 2010 Oct 26: support .so (low-level roff "switch source file") 2010 Oct 27: OpenBSD ports FAQ section about mandoc and groff 2010 Oct 29: landry@ performs the first major USE_GROFF removal 2010 Oct 29: millert@ removes colcrt(1), checknr(1), soelim(1) 2011 Feb 07: use mandoc in Xenocara Imake builds 2011 Mar 02: OpenBSD 4.9 release rolled without groff 2011 Mar 12: cvs rm groff 2011 Mar 19: update ports groff from 1.15 to 1.21 2011 Apr 24: tweak mandoc to conform to newest groff habits 2011 Apr 26: fixed groff-1.21 invocation for Imake ports .fi .H 3 "Benefits of the move" While moving groff from the base system to ports certainly brings benefits for the base system, in particular a reduction of GPL and C++ code, it also has important advantages for groff lovers: .BL .LI In the base system, you are stuck with the last groff version released under GPL v2, which is groff-1.19.2 released in 2005, but in the ports tree, GPL v3 doesn't hurt as much. .LI Even the license issue aside, it is much easier to keep software developed externally up-to-date in the ports tree than in the base system. .LE .P Newer groff has many new features, for example: .BL .LI groff-1.20 (Jan. 2009) added .Xr preconv 1 for Unicode/wide character support, several localization packages, XHTML support, a MathML output device for .Xr eqn 1 , the .Xr chem 1 preprocessor for .Xr pic 1 , a new hdtbl package, and many smaller features. .LI groff-1.21 (Dec. 2010) added support for japanese manual pages. .LI groff-1.22 (Dec. 2012) added .Xr gropdf 1 and .Xr pdfmom 1 to directly generate PDF output. .LE eurobsdcon2014-mandoc-paper/6/solution.roff010064400017500001750000000027221240234064300216150ustar00ischwarzeischwarze.H 2 "The OpenBSD solution for manual pages in ports" The following strategy has been designed and implemented by Marc Espie, and it has proven very sturdy and very easy to use.\*F .FS BSDCan 2014 p. 31 .FE .P Make sure that no port tries to preformat manuals during the build target. Let every port install manual page sources during the fake install target. .P For the majority of ports, .Xr mandoc 1 can handle all manuals: That's it, you are done with respect to these. .P For the remaining minority of ports: Set a special boolean .Xr make 1 variable in the port Makefile, in OpenBSD called USE_GROFF. That variable implies a build dependency on the groff port. When building the package, the ports framework runs groff on the fly and packages the preformatted pages instead of the source pages.\*F .FS BSDCan 2011 p. 19 .FE .P After installing the packages, this will work just fine at run time: The preformatted pages will be diplayed directly by .Xr man 1 , and .Xr man 1 will format the source pages with .Xr mandoc 1 , with no dedicated configuration. .P For example, in OpenBSD in the spring of 2014 there were 7952 ports, 1217 of which still USE_GROFF (15%). Some of these probably don't really need it, but there is no hurry. Removing USE_GROFF needs a manual check \(em which was already done for about 3000 ports during the last three years. Instructions are available for checking ports in this respect.\*F .FS /faq/ports/specialtopics.html#Mandoc .FE eurobsdcon2014-mandoc-paper/6/problem.roff010064400017500001750000000021431240233773600214070ustar00ischwarzeischwarze.H 2 "Handling manual pages in ports" The .I "Law of Feature Creep" is quite well-known in general: If a software offers some feature, sooner or later somebody will use it.\*F .FS BSDCan 2014 p. 30 .FE .P It allows an easy corollary, applying it to the documentation of portable software: For every feature of the roff language (and for every groff extension), no matter how arcane and how obviously irrelevant for manual pages, sooner or later somebody will want to port a third-party software abusing that feature to format its manual pages. .P That is unfortunate because .Xr mandoc 1 is not a complete nroff implementation and it is not clear that it will ever be. .P In the base system of an operating system, this is not a problem: Given a finite set of manuals, all the required features can be implemented in .Xr mandoc 1 , or alternatively, it is possible to patch away the worst abuse in the handful of manuals affected. .P But in ports, \(lqmandoc or nothing\(rq is not a viable strategy: That would inevitably leave you with some seriously misformatted manuals, and in some cases with no usable manuals at all. eurobsdcon2014-mandoc-paper/6/makewhatis.roff010064400017500001750000000011561240234320100220670ustar00ischwarzeischwarze.H 2 "makewhatis(8) in ports" Base and X manual databases are essentially static.\*F .FS BSDCan 2014 p. 32 .FE But packages get installed and deinstalled. You could wait for the periodic .Xr weekly 8 .Xr makewhatis 1 rebuild. A better idea works as follows: .BL .LI During pkg_add, run .Nm makewhatis .Fl d .Pa /usr/local/man .Ar "files ..." .LI During pkg_delete, run .Nm makewhatis .Fl u .Pa /usr/local/man .Ar "files ..." .LE .P This is done routinely on OpenBSD, and it works seamlessly with the new makewhatis. So, right after .Xr pkg_add 1 , you call .Xr apropos 1 , and it finds the freshly installed manual pages. eurobsdcon2014-mandoc-paper/6/evolution.roff010064400017500001750000000023511240416501400217620ustar00ischwarzeischwarze.H 2 "Features that help, in particular for ports" Obviously, the more manuals .Xr mandoc 1 can handle, the easier will all this become. As explained above, usage of low-level .Xr roff 7 features causes most of the problems. Consequently, the ongoing efforts to improve low-level .Xr roff 7 support help ports in particular. .P The following features of this kind have been added to .Xr mandoc 1 during the last two years:\*F .FS BSDCan 2014 p. 37 .FE .BL .LI Indirect references in .Xr roff 7 expansion since April 7, 2014. .LI Expansion of .Xr roff 7 number registers since March 21, 2013 (Christos Zoulas). .LI Almost complete support for .Xr roff 7 numerical expressions since April 7, 2014. .LI Numeric comparison in .Xr roff 7 conditionals since April 3, 2013 (Christos Zoulas). .LI String comparison in .Xr roff 7 conditionals since March 8, 2014. .LI Newly supported .Xr roff 7 requests: .BL 5n 1 .LIb as append to string .LIb cc change control character .LIb it set input line trap .LIb ll change output line length .LIb rr remove register .LIb tr translate characters .LE .LI newly supported .Fl man-ext macros: .BL 8n 1 .LIb EX/EE example display .LIb OP optional element .LIb PD paragraph distance .LIb UR/UE uniform resource identifier .LE .LE eurobsdcon2014-mandoc-paper/6/summary.roff010064400017500001750000000015721240234421600214400ustar00ischwarzeischwarze.H 2 "Summary: step by step instructions" .AL .LI Build and commit a textproc/groff port (version 1.22.2), even if you have groff in base.\*F .FS BSDCan 2011 p. 19 .FE .LI Introduce the USE_GROFF port Makefile variable. Trigger a groff build dependency for USE_GROFF. Implement format-on-the-fly for USE_GROFF and install formatted. Without USE_GROFF, install source manuals. Pay attention to get packing lists right. .LI Safe way: Turn USE_GROFF on for all ports having manuals. Shortcut: Skip those that don't have it in OpenBSD. .LI Remove groff from base. .LI Start removing USE_GROFF on a case by case basis. To be extra safe: Only do it for ports having no mandoc ERRORs and identical output with groff and mandoc. .LI This is another critical phase: Stay tuned for bug reports from users and work with upstream to get them resolved. .LE .P In OpenBSD, this happened in October 2010. eurobsdcon2014-mandoc-paper/7004075500017500001750000000000001240213325400170035ustar00ischwarzeischwarzeeurobsdcon2014-mandoc-paper/7/section.roff010064400017500001750000000002031240213356300213770ustar00ischwarzeischwarze.H 1 "Status and next steps" .so 7/openbsd_status.roff .so 7/netbsd_status.roff .so 7/freebsd_status.roff .so 7/nonbsd_status.roff eurobsdcon2014-mandoc-paper/7/openbsd_status.roff010064400017500001750000000041011240366430100227710ustar00ischwarzeischwarze.H 2 OpenBSD .H 3 Status .BL \n(Pi 1 .LI Kristaps@ developed mandoc(1) since November 22, 2008.\*F .FS BSDCan 2014 p. 25 .FE .LI Source code in the base repo since April 6, 2009. .LI Schwarze@ maintaining it since June 14, 2009. .LI Actively maintained regression suite since October 27, 2009. .LI .Xr mandoc 1 installed with OpenBSD-current since April 2, 2010. .LI Base system manuals built with .Xr mandoc 1 since April 3, 2010. .LI USE_GROFF framework for ports by espie@ since April 5, 2010. .LI Releases fully rely on .Xr mandoc 1 since OpenBSD 4.8, November 1, 2010. .LI Groff disconnected from base build since October 18, 2010: .br .ft B .Xr mandoc 1 is the only documentation formatter in base for almost four years. .ft R .LI Groff removed from the source tree since March 12, 2011. .LI Groff 1.21/1.22 available from the ports tree since March 19, 2011. .LI No stable releases contain groff since OpenBSD 4.9, May 1, 2011. .LI Install manual sources, not preformatted manuals since June 23, 2011. .LI SQLite-based code in the source tree since December 30, 2013. .LI makewhatis(8)/apropos(1) using mandoc since April 18, 2014. .LI New .Xr man.cgi 8 running on openbsd.org since July 12, 2014. .LI All will be released with OpenBSD 5.6 on November 1, 2014. .LE . .H 3 "Possible future directions" .BL .LI Replace the traditional BSD .Xr man 1 implementation with the one from the mandoc toolbox. .LI Switch the default output mode from .Fl Tascii to .Fl Tlocale . That does no harm when using the POSIX .Xr locale 1 , and it helps people using UTF-8 locales. .LI Integrate .Xr preconv 1 into .Xr mandoc 1 for better UTF-8 handling.\*F .FS BSDCan 2014 p. 39 .FE .LI Improve .Xr pod2mdoc 1 to better support .Xr perlpod 1 to .Xr mdoc 7 transitions, in particular for the LibreSSL manuals. .LI Support automatic semantic enrichment of Perl manuals with .Xr pod2mdoc 1 . I'm not yet sure this is practicable, it's just an idea so far. .LI Support transitions from .Xr man 7 to .Xr mdoc 7 with .Xr doclifter 1 and .Xr docbook2mdoc 1 . .LI Unify parsers, allowing more low-level .Xr roff 7 improvements. .LE eurobsdcon2014-mandoc-paper/7/netbsd_status.roff010064400017500001750000000026321240416510700226260ustar00ischwarzeischwarze.bp .H 2 NetBSD .H 3 Status .BL \n(Pi 1 .LI A pkgsrc mdocml port by J\(:org Sonnenberger exists since March 1, 2009.\*F .FS BSDCan 2014 p. 26 .FE .br This implies support for \fImany\fP additional platforms. .br The current version is 1.13.1 since August 10, 2014 (Thomas Klausner). .LI The first code patch was sent upstream by J\(:org Sonnenberger on June 11, 2009. .LI Source code in the base repo and installed by default in NetBSD-current since October 21, 2009. .LI Big changes on February 7, 2012: .BL \n(Pi 1 .LI .\" src/share/mk/bsd.own.mk#rev1.695 Install source manuals, no longer install preformatted manuals. .LI .ft B Use .Xr mandoc 1 as the default run-time manual formatter instead of groff. .ft R .LI Use .Xr makemandb 8 by Abhinav Upadhyay instead of .Xr makewhatis 8 together with versions of .Xr apropos 1 and .Xr whatis 1 based on it, featuring full text search, but not semantic search. .LE .LI All this was first released with NetBSD 6.0 on October 17, 2012. .LE .P Semantic searching is not yet supported, not even as an option. . .H 3 "Recommended next steps" .BL \n(Pi 1 .LI Upgrade base system version to 1.13.1. .LI Integrating semantic search support is a serious problem because it is completely incompatible with the existing .Xr makemandb 8 which also uses SQLite3 but doesn't use the strengths of .Xr mdoc 7 in any way. Unfortunately, i must admit i have no idea how this dilemma might be solved. .LE eurobsdcon2014-mandoc-paper/7/freebsd_status.roff010064400017500001750000000021761240365761600227760ustar00ischwarzeischwarze.H 2 FreeBSD .H 3 Status .BL \n(Pi 1 .LI An mdocml port by Ulrich Sp\(:orlein exists since March 9, 2009.\*F .FS BSDCan 2014 p. 27 .FE .LI First code patch sent in by Ulrich Sp\(:orlein on July 18, 2009. .LI .ft B Source code in the base repo and installed by default since October 19, 2012. .ft R .LI First released with FreeBSD 10.0 on January 20, 2014. .LI .Xr mandoc 1 is installed, but not used. .LI Semantic searching is not yet supported, not even as an option. .LE .H 3 "Recommended next steps" .BL \n(Pi 1 .LI Upgrade base system version to mandoc-1.13.1. .LI Switch base system to use mandoc by default. .LE . .H 2 "DragonFly BSD" .H 3 Status .BL \n(Pi 1 .LI .ft B Source code in the base repo and installed by default since October 27, 2009 (Sascha Wildner) .ft R .LI First released with DragonFly BSD 3.6.0 on March 28, 2010. .LI First code patch sent in by Franco Fichtner on November 25, 2013. .LI .Xr mandoc 1 is installed, but not used. .LI Semantic searching is not yet supported, not even as an option. .LE .H 3 "Recommended next steps" .BL \n(Pi 1 .LI Upgrade to mandoc-1.13.1. .LI Switch base system to use mandoc by default. .LE eurobsdcon2014-mandoc-paper/7/nonbsd_status.roff010064400017500001750000000024161240366405400226360ustar00ischwarzeischwarze.H 2 "Status in non-BSD systems" .BVL 5n .LI illumos Mandoc is contained in the base system and used by default for formatting manuals since July 21, 2014 (Garrett D'Amore). Upgraded to 1.12.3 on August 2, 2014 (Garrett D'Amore). .LI "Minix 3" The mandoc source code is in the base repository since June 26, 2010 (Ben Gras).\*F .FS BSDCan 2014 p. 28 .FE However, the project appears to be somewhat apathetic. It is still using a version that is more than four years old. .LI "Alpine Linux" An aport exists since July 6, 2010 (Natanael Copa). It moved from testing to main on June 12, 2011 (Natanael Copa). It is continuously maintained. .LI "Arch Linux" An mdocml package exists since October 3, 2010 (Markus M. May). It was updated to 1.12.3 on April 17, 2014 (new maintainer Jesse Adams) and to 1.13.1 on August 17, 2014 (Jesse Adams). .LI "Slackware Linux" An mdocml package exists since January 7, 2014 (D\('aniel L\('evai). .LI "CentOS, Debian, Fedora, RedHat, SuSE, Ubuntu Linux" Unofficial mdocml packages exists since April 19, 2014 (Jesse Adams). .LI "MacOS X" An mdocml package exists since September 5, 2010, but it seems abandoned. .LI Cygwin An mdocml package exists since December 12, 2012 (Yaakov Selkowitz). It was updated to 1.12.2 on November 11, 2013 (Yaakov Selkowitz). .LE eurobsdcon2014-mandoc-paper/paper.roff010064400017500001750000000041621241204171600207030ustar00ischwarzeischwarze.\" .\" The following Copyright notice, license and disclaimer applies to all .\" files of this paper and is not repeated in each individual file. .\" .\" Copyright (c) 2014 Ingo Schwarze .\" .\" Permission to use, copy, modify, and distribute this paper for any .\" purpose with or without fee is hereby granted, provided that the above .\" copyright notice and this permission notice appear in all copies. .\" .\" THE PRESENTATION IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL .\" WARRANTIES WITH REGARD TO THIS PRESENTATION INCLUDING ALL IMPLIED .\" WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE .\" AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL .\" DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA .\" OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER .\" TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR .\" PERFORMANCE OF THIS PRESENTATION. .\" .ll 16.5c .nr Ej 1 .nr Fs 0 .nr Hb 4 .nr Hs 4 .ds HF "B B I I I" .ds HP "+2 0 0 0 0" .de HX .if \\$1=2 .sp .. .so macros.roff .PH "'Ingo Schwarze: Let\(aqs make manuals more useful!'- % -'\ EuroBSDCon 2014, September 26, Sofia'" .COVER .AF OpenBSD .TL Let's make manuals more useful! .AU "Ingo Schwarze" .AS This paper summarizes the lessons learnt from the first six years of mandoc(1) development (2008-2014): what to keep in mind when reading and writing documentation and setting up documentation systems for operating systems and portable software. It covers the content of the tutorial of the same name given on September 26, 2014 during EuroBSDCon in Sofia, in a form more suitable to self-instruction than a set of presentation slides. It includes some additional material extending the scope that wouldn't fit into the three hours of the tutorial, in particular all relevant content of my two BSDCan presentations given in Ottawa in May 2011 and May 2014. .AE .ND "Version: September 10, 2014" .COVEND .so 0/section.roff .so 1/section.roff .so 2/section.roff .so 3/section.roff .so 4/section.roff .so 5/section.roff .so 6/section.roff .so 7/section.roff .so exercises.roff .TC 1 1 3 eurobsdcon2014-mandoc-paper/macros.roff010064400017500001750000000003131240344165000210530ustar00ischwarzeischwarze.de Ar .I "\\$1" \\$2 .. .de Cm .B "\\$1" \\$2 .. .de Fl .B "\-\\$1" \\$2 .. .de Nm .B "\\$1" \\$2 .. .de Pa .I "\\$1" \\$2 .. .de Xr \\$1(\\$2)\\$3 .. .de Va .I "\\$1" \\$2 .. .de LIb .LI \fB\\$*\fP .. eurobsdcon2014-mandoc-paper/exercises.roff010064400017500001750000000267521240416521100215750ustar00ischwarzeischwarze.H 1 Exercises The following pages propose some exercises for the hands-on working phases of the tutorial. .P Before starting to work, consider the following points: .AL .LI The amount of exercises proposed is intentionally .I much bigger than what anybody can possibly handle in the 40 minutes available for working on them in this tutorial, such that you have a .I choice both with respect of topic area and difficulty. .LI Do not waste time studying all the instructions of all the exercises. Look at the table of contents for topic areas you are interested in, then skim the instruction texts of exercises for that area. Only study those instructions in detail you actually consider working on. .LI Feel free to work alone or with one or two partners at your choice. .LI Feel free to ask any questions you might have to me or any other participant during the working phases. Do not fear interrupting people. Everybody can work in a concentrated manner for hours on end at home. What you can have here and what's harder to get at home is .I cooperation . .LI In the first, longer (30 minutes) phase, feel free to spend all the time on one complex task or explore several - probably not more than three - smaller tasks. In the second, shorter (10 minutes) phase, you should probably focus on one task, or at most two. Try to avoid time-consuming repetitive work during the tutorial. For example, instead of writing down a long of list options, just write down the first two or three, then move on to the next section. .LI If, during your work, you discover anything you consider worth presenting to the other participants of the tutorial, talk to me during the working phase, and i may be able to reserve a slot of one to three minutes for you. Nobody will be forced to present their results, though. .LE .AE . .INITR exercises .ds Qrf Exercise \\*[Qrfh], page \\*[Qrfp]. .nr Cl 3 . .nr H2 -1 .H 2 "Recommended exercises" The following exercises are recommended for the following target audiences. Of course, all participants are free to choose any exercise they are interested in. See the table of contents at the very end of this document. .P The exercises marked "second phase" are easier to handle in the second working phase because the subject will only be explained .I after the first working phase. If you have a strong interest in some exercise marked "second phase" and feel able to handle it, there is nothing wrong with attempting it in the (longer) first working phase, though. .BVL 5n .LI "Software or documentation developers..." .BVL 5n .LI "... working on software lacking one or more manual pages:" first phase: .GETR mdoc_write .LI "... working on software having non-mdoc documentation:" first phase: .GETR mdoc_migrate .LI "... working on software having mdoc documentation:" first phase: .GETR check_basic .LI "... maintaining a portable software package:" first phase: .GETR makefile_tman .GETR configure_write .LE .LI "Porters and port maintainers:" first phase: .GETR check_basic .GETR check_bulk .br second phase: .GETR makewhatis_test .GETR integration_test .LI "Operating system developers and documentation maintainers:" first phase: .GETR check_bulk .br second phase: .GETR makewhatis_test .GETR integration_test .LI "System administrators:" second phase: .GETR integration_test .LI "End users:" second phase: .GETR makewhatis_test .LE . .H 2 "Using the mdoc(7) formatting language" .H 3 "Writing a manual from scratch" .SETR mdoc_write If there is a software, ideally not too complex and with a limited number of features, that you maintain or just care about and that doesn't have any manual page yet, start writing a manual page for it using the .Xr mdoc 7 language. .P Hints: .BL .LI Be careful to not get lost in details. .LI Start by setting up the preamble and .I all required section titles. .LI In each section, take rough notes which material will need to be covered in that section. Only after that, start filling in individual sections. .LI Pay particular attention to get the beginning of the DESCRIPTION concise and clear. It should clearly state the purpose of the software and not be wordy. .LI When drafting lists, first figure out which list entries will be needed, only then start filling them in. .LI If there is more material to cover than you can finish during the tutorial, don't fear leaving blanks in the middle. Try to work on as many different parts as possible and take advantage of the chance to ask questions that arise. .LI Consider submitting the result to the software maintainer for inclusion, possibly after finishing it at home if time is insufficient during the tutorial. .LE . .H 3 "Translating a manual to mdoc" .SETR mdoc_migrate If there is a software, ideally not too complex and with a limited number of features, that you maintain or just care about and that has a manual in a language different from .Xr mdoc 7 , start translating the manual to .Xr mdoc 7 . Working on this exercise is particularly useful if you have reason to believe that the maintainer might be willing to switch the format. .P Hints: .BL .LI Be sure to keep a copy of the original. .LI If there is more material to cover than you can finish during the tutorial, don't fear leaving blanks in the middle. Try to work on as many different parts as possible and take advantage of the chance to ask questions that arise. .LI If the original was a .Xr man 7 page, compare the output of .Xr mandoc 1 from your version with the output .Cm mandoc .Fl Omdoc produces from the original version. .LI Consider submitting the result to the software maintainer for inclusion, possibly after finishing it at home if time is insufficient during the tutorial. .LE . .bp .H 2 "Manual pages for portable software" .H 3 "Package autogenerated man and cat" .SETR makefile_tman Take a portable software package you maintain or care about that already has .Xr mdoc 7 manuals or where you consider converting the manuals to .Xr mdoc 7 . Design and implement .Xr make 1 targets in the distribution tarball build system to generate and package .Xr man 7 and cat (preformatted) versions of the existing .Xr mdoc 7 manuals. .P Caveat: .P The difficulty of this exercise, and the amount of work required, can vary greatly depending on the size and complexity of the software package and the build system used. For a large or complex package, familiarity with the build system is probably required. .P Hints: .BL .LI You must have a checked out copy of the source repository. This exercise cannot be solved starting from a mere release tarball. .LI If the .Xr mdoc 7 versions of the pages have not yet been written, don't waste your time on that, just write two or three ten-line stubs to get started. .LI Pay attention to not edit autogenerated files, like .Pa Makefile if there is also .Pa Makefile.in . .LI Try to use .Xr make 1 inference (suffix) transformation rules. Avoid writing one rule for each manual page if possible. .LI To test your changes, build and inspect a release tarball. .LE . .H 3 "Write configuration tests" .SETR configure_write Take a portable software package you maintain or care about that already has .Xr mdoc 7 manuals or where you consider converting the manuals to .Xr mdoc 7 . Design and implement configuration tests to decide whether to install mdoc, man, or cat manuals on the target system. .P Caveat: .P The difficulty of this exercise, and the amount of work required, can vary greatly depending on which build system is used and how it is used. You better have a at least a rough idea what you are doing and some experience with the build system in question, or at least with build systems in general, before attempting this exercise. .P Hints: .BL .LI If possible, check out a copy of the source repository of the software to work in. Depending on the build system used, merely having a release tarball may or may not be sufficient to do meaningful work on this exercise. .LI Pay attention to not edit autogenerated files, like .Pa ./configure in software using GNU .Xr autoconf 1 . .LI An easy test of your changes can probably be done in the checkout area by running the tool to regenerate autogenerated files, then running the configuration script and inspect what it detects. .LI The ultimate test would be to build a release tarball, than test from that. .LE . .H 2 "Quality control for existing manuals" .H 3 "Checking one or a few specific pages" .SETR check_basic Choose a small number of manual pages you maintain or care about. Run those check tools \(em .Xr mandoc 1 .Fl Tlint , .Xr mdoclint 1 , .Xr igor 1 , and maybe gmdiff \(em you have installed or can easily install on them. Prioritize your findings and take notes about those you consider relevant. If desired, prepare patches to improve the manuals. .P Ask somebody who is experienced with .Xr mdoc 7 to have a look at your patches, then send them upstream for inclusion. . .H 3 "Running bulk quality checks" .SETR check_bulk In a larger tree of manuals you maintain or care about, run .Nm mandoc .Fl Tlint .Fl Wfatal , solve any issues showing up, run .Nm mandoc .Fl Tlint .Fl Werror , and start working on the issues found. .P Hints: .BL .LI Don't be shy to use .Xr find 1 or scripting as required to quickly and efficiently get at the files you need. On the other hand, don't waste your time devising elaborate frameworks right now; this tutorial is about manuals, not about build systems! .LI Refrain from large-scale patching until you are really sure what you see are large-scale errors in your tree and cannot reasonably be solved in other ways, for example by improving .Xr mandoc 1 . For this exercise, communication is of particular importance, there .I are people around you can talk to! .LI If you arrive in a state where the are no more errors, you can proceed to look at warnings. .LE . .H 2 "Searching and displaying manual pages" .H 3 "Testing makewhatis and apropos" .SETR makewhatis_test Choose a manual tree you are maintaining or care about. Run .Xr makewhatis 8 on it. Try out various .Xr apropos 1 features and try to discover issues. Report any you find to me. .P Hints: .BL .LI For this exercise, having an up-to-date mandoc suite is more important than for most. Consider checking out from anoncvs@mdocml.bsd.lv:/cvs and building and installing from that before starting. On OpenBSD, it is better to build from .Pa /usr/src/usr.bin/mandoc instead. .LI To run .Xr makewhatis 8 on one specific tree only, you will need the .Ar dir argument. .LI If everythings appears to be working fine, try to refine your methods. For example, try the .Fl p , .Fl D , and .Fl DD options of .Xr makewhatis 8 and try to understand the output. .LI Probably, most of the output will be false positives, but you may also find some real bugs in the manuals in this way that can be worth patching. .LI If you are comfortable with SQL, run the .Xr sqlite3 1 command line client directly on the database generated by .Xr makewhatis 8 and look for anything that seems questionable. .LE . .H 3 "Checking manual locations and formats" .SETR integration_test On your favourite operating system, figure out all locations where manuals are installed and identify those that .Xr man 1 searches by default. Figure out all formats of installed manual pages (formatted/unformatted, source language, compression, whatever). Check that .Xr makewhatis 8 and .Xr apropos 1 work for all locations and all formats. Report any issues you find to me. .P Hints: .BL .LI For this exercise, having an up-to-date mandoc suite is more important than for most. Consider checking out from anoncvs@mdocml.bsd.lv:/cvs and building and installing from that before starting. On OpenBSD, it is better to build from .Pa /usr/src/usr.bin/mandoc instead. .LE ¡®Yes, sir. I felt sure you understood that. She said she had told you.¡¯ "Why, eh,--I--I don't know that my movements need have anything to do with his. Yours, of course,--" "Ah, but if it saved your life!" "No, I'm not," grumbled the Doctor, "I've had enough of this wild-goose chase. And besides, it's nearly dinner time." "I am coming to that," Lawrence said, lighting a fresh cigarette. "As soon as Bruce was in trouble and the plot began to reel off I saw that it was mine. Of course there were large varyings in the details, but the scheme was mine. It was even laid on the same spot as my skeleton story. When I grasped that, I knew quite well that somebody must have stolen my plot." Judy In a coach-house, through which we passed on our way to see the prince's favourite horses with the state carriages¡ªquite commonplace and comfortable, and made at Palitana¡ªwas a chigram,[Pg 68] off which its silk cover was lifted; it was painted bright red and spangled with twinkling copper nails. This carriage, which is hermetically closed when the Ranee goes out in it, was lined with cloth-of-gold patterned with Gohel Sheri's initials within a horseshoe: a little hand-glass on one of the cushions, two boxes of chased silver, the curtains and hangings redolent of otto of roses. "Are you certain of it? You have seen so very little of him, and you may be mistaken." "And your wife?" "I drawed on my man's bundle o' wood," said Gid, "and then dropped a little, so's to git him where he was biggest and make sure o' him." HoME²¨¶àÒ°½áÒÂ×óÏßÊÓƵ ENTER NUMBET 0016khizeu.com.cn
www.fqjxjd.com.cn
ho84.org.cn
www.eeniu.com.cn
haztcm.org.cn
www.skwallet.com.cn
www.nlzj.com.cn
www.nmxdqm.com.cn
www.plolpr.com.cn
www.qipeipu.com.cn
欧美西方美女艺术图片 美女掰b图片 外国操逼成人片 肏屄蓝魔mp5官网 骚穴骚影网址 藤冲有关的电视剧 偷拍包厢内少男少女激情狂欢 大中华色露脸对话 黄片俄罗斯大学校园 久久炮图 类似狠狠鲁的网站 台湾大佬自拍偷拍网 人于兽性爱小说 美女上厕所刚出炉图 来插妹妹的小穴 美红换交 丝袜女女舔足 性爱换妻乱伦故事 女同性恋热吻优酷视频 苍井空裸阴毛 日本美女百濑图片 成人毛片快播高清影视 宫藤新一的性福生活 成人小妹 少女金频双艳 人与shuo 武汉教室做爱吉吉 人体艺术黑木耳西西 射极品空姐超碰 骚女自慰照片 江西公安局政委用枪逼儿媳妇通奸 浅川花里子 粉嫩逼贴图 一进大门观四方 诱奷表妹 和大姨子肏屄之续 日本 大胆人体艺术图片 松岛枫 中国女裸名单 美女露体没有马赛克 国产强奸乱伦加电影yingyinxianfeng 熟女人妻15p 诱奸 少妇 小说 天使的眼泪黄色群 伦理 偷拍 皮皮 骚女浪妇乱伦爱爱 幼幼性交比赛 魔王av亚洲无码bt 就要在线撸电影站 快播欧美荡妇聚会 影音先锋韩国女主 亚洲 性交图 看黄片不用要播放器 猪猪影院百度影音艺术片 人性交电影 妹妹看黄色片 浙江真实乱伦迅雷 一个最真实 的我 一个 的我 的我 歌曲 成人爱爱在线观看 色妹妹天天撸 插妹妹成人电影院在线观看 风间由美熟女人妻黑丝商务交会种子 男性大鸡巴被操的故事 人之初性本善 致橡树原文 神经系统体格检查 狠狠撸操干妈图片大全 中国成人电影小说 日姥姥性爱淫乱三从四德群交30p 爱爱jj发综合网 黑木香一经典番号 黄蓉与刘老汉 大黑鸡大吧操妈妈 印尼大胆人体艺术 草别人媳妇 人体艺体肉欲 互联网jiqingwuyue 白虎穴尿尿艺术图片 操操淫乱穴 寡妇自拍 三级色逼一片 林心如两腿分开 人与动物三客优 金伟哥vga说明书 日本av女优馒头逼图片 十九摸 儿子和母亲激情性交 有快播可以上的色网址 幼幼色色 ribenchengrenxiaoshuo 外国性爱网站 欧美美女人穴 性爱小说撸啊撸 呕美操逼图 女人的屄照片 人体艺体阴部插图片 百度云照烧铁板 我的八年性情史 淫香淫色插逼图 有黄的的qq号 欧美色亲片 zuixinmuziluanlunxiaoshuohetupian 杜比影院最佳座位 白雪公主摸乳干 美女露屄毛的图片 绫川まどか在线先锋 影音先锋操大奶妹 手机动漫成人电影网 强奸女护士 亚洲陈丽佳人体艺术 欧美经典七夕电影 WWWANGOULECOM 我和村嫂乱伦 母子通奸网 亚州色图就射 www777影音先锋 邓紫棋图片亚洲色图偷拍自拍 白嫩美女做爱爽图36p 瑶瑶的淫叫声 大树梨纱 人体私处大胆艺术图 母子插逼视频 母子淫水 狠狠干五色天 换妻17p 性之图吧偷拍自拍亚洲色图 激情性爱动漫欧美 WWW53AVCOM 启东操逼 艳母峰臀 欧美激情裸体艺术图片 激情明星狠狠碰 欧美强奸少女图 孟十朵 妻子玲儿 黑大炮群交内射白虎视频 西西人体艺术大胆人体艺术亚洲 白领穿丝被干视频 亚洲电影第10 归乡义母种子 骚丝袜老师 性爱亚洲色图大鸡巴 长谷川美红母 2014uuucom 神马影院未来电影在线 强奸少妇图片乱伦小说 我与妈妈做爱抽插操逼 ss5678 美丽的丝袜老师妈妈4 樱井知香裸体照 摸女人大胸图片 丁香社操逼 操美女av 久久热无码在线视频 去去去电影 女性尿道口真人版图片无衣 liaozaiyantan成人免费电影网站 超碰视频网友自拍第一页 色波波影院 迅雷下载 诱色天使香香裤高腰款 熟妇张柏芝丝袜 2015最新操 酒鬼电影院会员 秋露伦理Av 咪咪色色老师小说 i嘿片网 AV换妻电影 风间由美club 奶子大骚Pmsewuyecom 自拍爱色综合社区 手机看片操女优 av丝袜教师手机天堂 16668伪C0m 花俏小姨子mp4 久久撸久久肏开心五月 搜索sexinsexnet 厨房干同学妈妈的屁眼wwwwanren8netmwanren8net 日本小金井 美国十次啦老熟女 www色色网络com 天堂网妞赶 插蝴蝶逼 青青3p luanlun爱 小姐的的性感生活作爱片 激情短篇小学 牛牛超碰免费公开在线视频 日夜色先锋资源站 在线网站AV黑人wwwgzyunhecom 手机av受美国法律保护 性感老师校园春色 天天啪夜夜操www9eyycom 洪爷小说幼女交换 美女洞毛电影 雪白的古典武侠 三级片三级片的坏的视频播放 小说专区乱伦小说目录99 色第四色wwwav565com色就是色 舔岳母的小穴 肥佬能用的三级片网站 淫荡的肉弹美女教师 熟女香蕉无a在线视频 美女操逼黄色大片 www1111r80s 欧美色图迷人的骚女15P 哥哥干av成人社区男人天堂 武藤蓝av 搜索炼狱岛 精液降头 偷拍女厕所自慰图片 xxoo成人影院熟女 freefronvideos人母 卡通图片另类变态 春药潮吹痉挛视频 2017最新www超碰com 最新kkbokk 欧美露大乳图 激情电影乱伦小说 邻家丰满少妇 绑架调教性感丝袜性奴女 风流情哥哥网站 激情小说加多撸 欧美性虐哥哥射 变态淫荡 seyuyue 人体穴鲍图 伊人在线视频变身6 中国十大禁书黄小说 国产自拍秀mp4 性交口交天天撸 裸体摄影优果网 少妇人妻自拍图区 久草在线新时代3wwwczyzxcnzxzy8com 最大胆一千美女 大几把骚逼 全国最大三级图文 干了校长的骚穴 与妻侄女 户田惠梨香av片 家庭乱伦之人妻 淫贱孝姨 干美女狗趴 变态孕妇母乳片 sss480 偷拍穴毛 yazhouxingjiaosetu 明日花绮罗京都不伦妻 se牛牛视频网址 婶婶的原味内内 偷拍自拍做爱操逼 天天瑟瑟天天撸 蕾丝兔宝宝图片 华为网盘饭岛爱 天天鲁鲁天天在线 岳母与女婿性爱故事 2016年的小萝莉网站 狐狸色成人AV网站 18岁女主播直播自慰 qinglouscom 农夫激情基地 免费小萝莉自慰在线直播 wwwmmtt11me 性交透明内衣美女乳头无遮挡 亚州av潮吹视频 高跟熟女性奴 淫秽黄色啪啪手机视频 幼同志 怡红院更新前的主页 青青草肛交灌肠视频在线播放 搜索100avcom 老熟女下一篇7p 国语对白AV在线观看wwwyaob111com 成人动漫在线免费 11ppdd页面访问升级 189df、com wwwsesoucom 舞会电影台湾 www550ai30in 伊人情人网综合wwwggsao58info 都市淫秽 朋友妻子穿着丝袜让我舔 激情网址五月天 奶大妞女 兽皇英国 老熟女喷水小说 淫淫撸 偷拍影音先锋电影网 网上聊操逼 www路ppp7cnm 色色成人Cy视频 性吧春暖花开性吧有你旧版 丝袜制服人妻交换 鬼父第十七集番号 映像av加勒比先锋影音 wwwtutu10cnM 欧美乱伦18p 大香蕉超 国产看片点我 日本女穴 阿v手机天堂 稻田淫 wwwyouijzzsmcnm 我的办公室老婆三邦年 郑重声明我们立足于52avav xingnuchuanqi 黄色电影露阴毛 黄色录像番金莲 AV91 影音先锋强奸影片 亚洲在线a手机 曰本女优在线www5sdlcom 帮帮鲁色老汉 在线成人小视频下载 黄色网站都有那些 大胆人体露阴艺术 浅田美姬 草榴免费视频 捷克??机 丝袜美女被性侵 日日性 丝袜人体艺术偷拍 凹凸视频在线av 情欲超市小说 西西里大胆艺 黄片免费网址 东方a∨在线亚州色图狠狠撸 苍井空超短裙丝袜诱惑图片 奇米成人影视色和尚、色尼姑 ckck爱情电影 恋爱记录短片分享 局长成长史686 AV成人播放器免费的 纯洁的玛利亚邪恶漫画 五月激情综合狠狠色 wwwbbb552cm 性插网页 美女丝袜撸撸五月天 色色泡泡影院 一部女生被插jj的完整黄片 wwwadcrrr222con 秋霞伦理片在线播放 教室调教老师 亚洲色图日本AV 很很艹 日骚妇内射在线视频观看 ya亚洲麒麟色影影院 qqb66666 亚洲色图欧美色图美腿丝袜 曰一日 农村少妇电影magnet 移动成人你射精 20158韩国女演员激情视频合集 酒店小姐裸体艺术照 足浴小姐做爱过程 av女教师自慰动态图 AV影音先锋影院 123红色播放 女友自拍偷拍刺激 wwwseebimei 123CTCTCOM 婷婷xx youjuzz小说专区 一级黄色wangzhan 男人体摄影 裸体两性 欧美精品超碰 强奸小村花千骨 幼少女肏屄视频18 五月婷婷婷婷五月丁香 色魔在线 国产父女乱伦小说 KTKX089 www454HUcom 在线手机播放器 偷拍自拍32 成人激情图片，电影mmmnn7777 ww777rvvom WWW9itKcom 美国名模啪啪啪 用力的操狠狠的干 小浪穴妹妹亚洲色图 525zzzcom 乱伦妈妈15p 唐伯虎点秋香不是三级片 最大色牛牛 欧美色kuaibo 手机在线tokyo 欧美幼女网mp4 760yycom 少妇干净迷人鲍优优 绳艺magnet 身穿民族服饰的中国少数民族漂亮美女大胆人体艺术7国内 www47escomxz34 Www2222magnet 色偷怕自拍视频 丁香五月天拍拍播放` 青苹果影院噜噜妈妈 公然妄想露出在线 图片区偷窥自拍亚洲色图欧美色图动漫图片美腿丝袜清纯唯美乱伦图区电 黄色三级片77天天撸 美国女孩成人网站 东京热亚洲色色 超碰av大帝在线视频 西瓜成人资源网 一级片城年5 孝姨大阴唇 pp529com 青青色草在线 504hu迅雷下载 殴州1314 母子乱论视频 微信自拍成人视频在线观看 www44cim 东方av官方 297Pmp4 骚av老师 小明看看成人永久免费视频在钱imgcctuocom 越南人体艺术露鲍 两个女人用道具做爱 影音先锋制服丝袜偷拍 爱搞搞爱撸撸爱色堂 闹洞房就去干 男根的诱惑系列 樱井亚莉偷拍自拍 色dogcom 里中结衣在线观看 特大鸡巴碰上大波霸 躶体狂插相片 熟女撒尿视频 国产成人在线视频网站 武汉18中教师门 刘亦菲阴道毛多吗 操b网址大全 欧美性生活色图 母子尾交 图 操山村老大妈 淫母之穴 00后人体图片少女无毛掰开图片 6655人体艺术果果人体艺术波谷人体艺术 刘嘉大胆人体艺术 欧美色图 成人动漫第一页 淋浴做爱av 骚逼yaoyao 9115视频在线资源sss 黑屌做爱爽片 女优性交免费电影 欧美熟肥女图片 韩国女主播朴妮唛的黄色小说 撸哇哽播 在线自拍干幼女 高清晰自拍偷拍图色色网 去哦v大 欧美父女性爱 淫秽网站肥女视频 666亚洲无码 最新日韩乱伦小说网站 操少妇游戏 日本少妇11p图片 什么片好黄 多人合集9部 河合优衣ed2k 狠狠撸美女手掰穴图片 女儿交换乱轮 欧美妈妈和她的大屌儿子 富婆和年轻帅男性交 夫妻居家性爱自拍 色色偶性爱自拍 强奸模特小说 少年与熟妇爱图 体操美女之性生活 色骚逼在线高清播放 www1314xxx 韩国三级片一对男女在大学教师xxoo后来女的怀孕了男女结婚后女的跑了男的和 夫妻坐爱一级片 成人男女做爱视频 人体艺术图片绘狗网 入江辉美在线电影 鲁av影院 动漫同志片 风间由美爱爱网站 乱伦3p生活 最美妙的骚逼 王梦溪迅雷种子下载 我姐尻屁片 亚洲色图人妻p 日本人体艺术波之轩 偷拍黄色照片 汤加l丽裸照 伦理片日本家庭教师 操逼撸撸撸吧影院 WWW_BB152_COM 保险知识 左边杨丞琳 女人淫乱图 穿挺屄裤子的图片 色色公公与好儿媳 苍井空完全服从 色姐姐乱依 人妻乱伦星野绫香 熟女的角色扮演性爱快播 快播奸少女阴道电影 都市激情撸情 女人小穴很很日 狠狠肏老婆 亚欧人体摄影 中华医药艾叶作用在线视频 美女粉乳头10p 俩根大吊插一个美眉 品色堂俺去也 l少女luluse 模逼图片 山下智久柚木提娜种子ed2k 桃色播播激情五月天 沙绪里与狗 橹二哥影院影视先锋 小说快播综合网 12306影院第一页 巨乳无码xfplay 萩原舞电影 学生逼逼电影 欧美群交欧美色图 操少妇的逼短片 韩国主播吉吉影音 影音先锋厕所偷拍片 欧美熟女系列 鹿鼎淫记 3p性感尤物内射她的小骚穴 国产强奸幼幼逼 屌配屄毛 美女做热爱性交口交 女主人的人体厕所 顶级黄色图片可看到阴道口 无码少妇在线色 美女吹萧爽吗 黄色网站是多少翱 妹妹视 人体模特毛婷人体艺术 佐藤爱理番号 开心宝贝色播网 大屌巨乳系列 重温陈冠希图片做爱 美女豪乳50p 黑网袜性爱 操熟女老师 日韩av逍遥社区 安装香港恋夜场秀 美女制服诱惑男女 亚洲人妻岛国线播放 色小说色图色电影 日韩美女映画网 成人动漫转帖区 www狠狠射c 露脸绝对领域 妻子地铁失身 淫老婆电影第一页 中文字幕都市激情家庭乱伦亚洲色图 日本美女特大胆裸体露逼 口工教师av 色色999偷拍自拍日韩美女 www968ddcnmagnet jjjhhh1com日本 老王社区lw78cc ed2kyounv 美女口交舔逼小说 类似古丽阁网站 帅男同的鸡鸡ed2k 少妇的小骚玩 av黄鳝自慰小说 经点乱系列 百度久久做爱视频 a狼电影网成人 干小嫩b10Pwwwneihan8com 原国产母子做爱乱伦 美国十四拉人体艺术 老公我要插深点快点啊 涩涩乱伦小说 波霸暴露 994yyccm 美国女人和动物zzzwww 后入式微拍 中国国模03150p 色色色乱伦熟女图片 欧美大胆嫩肉穴爽大片 这个经典的给你吧rr123win 老婆乱伦片 另类少妇AV 巨乳俏女医漫画 WWWSWWWMITAOCOM 糖糖激情操逼 男屁眼被曰小说 yinsezonghewang 爆操人妻熟女15P 福利脱衣麻将 久久热偷偷撸黑丝袜免费经典视频 电影三级mp4黄色电影 超级黑人巨屌操白妇 人妻偷拍自拍强奸 男男激情小说肉文 韩日女优大奶视频 杀神有声小说 有声性小说mp3mp3 天春色图片 春色龙 樱井莉亚口才 日本成人 求h网导航 h网游游戏 有没有免费的黄网 设置加www 开心网 五月天 的五月天 qvod东京热图片 东京热n0383.rmvb 皮皮看黄片 沙发看黄片 黄色小说短篇 大色鱼情迷 风月阁论坛 日韩色姐姐 色兔子成人 师傅搞综合 18人体艺术 amod在线 大M成人综合 我爱弟弟影院 给力QVOD色 最纯洁少女做爱 無双帝國谁有E谁有G adultbig影院 日日顺 久久爱 歪歪小说网 九型人格分析 分分操导航 2017 一本道va手机在线 youjizzⅹⅹx 婷影院 一楼一凤影院欧美首页 猫蛋蛋很黄 shenyifuli ssni 049bt下载 另类图片-色爱 汤姆影院atom55 五月丁香深爱基地 皮特成人影院 gay pornhub video chitu x77223 色色ev 客车偷拍图片大全 动漫av ftp 青青草免费线观综合网 兄嫁はいじっぱり认证补丁 人妻少妇视频系列 小林瞳电车痴汉代码 秋霞短片福利 青娱乐视频盛会 日韩成人午夜视屏 影音先锋 熟女系列 在线观看皇片你的懂的 原千岁全集协和影视 日本一级性交视频 日本人性交视频 又黄又色的影剧院 大型AV 好屌色狼 早乙女由依 最佳专辑 尹人影院大香蕉禄现 吉泽明步私处流出资源大 伦理战场 情欲影院云播 bestfornmc0m 搭讪 mgnet 玩农村小姑娘裂缝电子书 日本AⅤ无码在线观看 伦理聚合高请无码在线播放 b里香视频在线2白色爽 有个妮可舍宾怎么找她视频 妇女磁力下载 凹凸視頻免賛在線 caosaobi在线观看免费 东方影库300 wwwxyc123con ipz862在线观看 武则天一级全黄视频 迅雷下载 亚洲在线一区 丰满女人多毛 思思热在线色视频 噜死你们资源站 mum骑兵在线播放 欧美色图狠狠插 邪恶插阴口动态图 风月影院黄视频 黄片VX 厕所偷窥视频 ay电影院 wwwkp99 木村都那厕所无码 good电影神马小丢 黑人和中国人群交视频 色色网址在线 九月婷婷在线 久草在线首页老司机 500dh com福利 杀戮都市里番在线 电影天堂在线福利 淫色视频网 苍木玛娜教师链接 97人妻C○m piczz漫画 橘梨纱作品Bd播放 激情按摩胸部无码 yy6080最稳定的资源 shtv123 日本成人影片 magnet 里番姐弟的关系3 超级碰av公开在线 thzvip 草帽国产综合网 日本色木木 dv1456 老色哥第四色 亚洲av欧美av电影av视频 日日夜夜插天天插 桃花影院今日新鲜事 A片网主名字 大屁股啪啪无码高清视频 东方阿v视频在线最新 xxoo淫交视频 电击女神asia fox在线 东北可爱小骚妻又一次3p娇小身材力战大屌-9 扒开双腿拳交 给我来个娘们操逼的黄片黄片 日本黄色枧频 j鸡巴小視频 叛客与雷鬼高清 迅雷 下载 美女自慰流淫水 在线 人妻小悠福利在线 巨乳王瑞儿在线视频 啪啪肉捧往哪叉?视频 劲暴欧美 一本道 东京热 自拍 国产 无码 黑丝自拍做爱 538prom久久日逼 www骚逼 XXXWWWUUU 求个一本道的资源 杨幂醉酒视完正版视频在线观看 日本女用胸为客人性按摩影院 仲村里绪 影音先锋女主播视频网站 vtt944 国产自拍福利社视频 18v韩国主播 在线翘臀福利 校园多女一男类番号 微福利吃精 香坂里奈 热舞福利120 538porm在线插放视频 wwwtaosetvcom 日本三级天堂网无码 韩国歪歪漫画官网进入 欧美图片亚洲色理论电影 欧美AVmm625 爱色影分类 56popo体验区 www奔驰宝马成人网站 射丝袜足 在线影院 kanmitao1视频 巨乳女教师の诱惑电影 欧洲老女人肏屄 ygyg66怼粉逼 热带夜中文字幕mp4 洁泽明步 m3u8人妻 蛇精脸网红主播小兔兔现场啪啪大秀 mmavtv少妇 萝莉小妹妹av 免费做爱视频网站免费 AISS模特索菲 在线福利视频 多毛龟 66ffff视频 佐伯雪菜在线av先锋 51zhiyuan 播播影院免费A片 av5685 真人下体抽插下阴喷水视频 色喜 王丹 美女动态图张又黄又色 www,O3ⅩXX,Cm 国内在线自拍人人澡人人看 BT 冲田杏梨巨乳女教师の诱惑 神马影院午夜伦dy888mmm 大香蕉亚洲人妻小说 丝袜自慰视频在线观看 涩播音频 popo福利网盘2018 性污秽小视频 想上你日你视频 享悦国产在线 校园禁断介护 校花在浴池被强视频 香港十大禁片黄e 小苹果性交影院 老湿影院未成年人 Xxxxx161116骆驼祥子 熟女papa视频 亚州影院午夜-一 97jibayingyuan 色欲直播 处女中出视频 爱爱福利区 鸡巴操逼的黄片 500夜趣福利免费 亚州香焦视频 porn在线播放制服丝袜 91c仔内裤哥在线观看 wwwz9k6com 偷拍美女浴室伦理电影 4438x3全国最大情人网站 速看网在线观看 新妈妈 ftp 亚洲国际成人综合 4455qu类似的网站 水菜丽无码粪便av连接 wwwabc300wom 唐朝TV360 国产AV,亚洲AV 采桑洗浴中心 波多结野衣与无码观看 比比琼斯作品合集磁力 山谷两日 thunder 日本高清凌辱免费三级片 magnet 欧美虐性 av无码 中文字幕 迅雷 樱井步禁断介护在线 四房色播av 织田真子113在线观看 罗马狼窝影院 插进去拔出来综合网 台湾R级在线 SOD时间禁止器 s跳蛋调教视频 WANQUEYINGYUAN 超级碰夜色猫视频 成人日日夜拍拍 RCTD mp4 v ip eeusssvv 求手机能看哪种视频的网站 欧美激情第一页在线观看 俺去也看免费视频 视频小姐中出 三七影视成人福利播放器 伦理无料 日韩SM高清 赫敏被强奸视频 泄欲哥导航为什么看不了了 1啪啪啪视频app 色无极亚洲影院东京热 黄色福利1000 要要橹福利 乱伦1视频 岛田阳子拍了多少AV片 正在播放肛交视频 萝莉无码小视频 本田英里子MP4迅雷种子 厕所偷拍无修。 北京彪哥真皮大床激战学院派 苍井空在线毛钱 波波兔 磁力 操奶子视频 波多野结衣与狗激情直接视频 久久干视频 ftp 橹先生成人影片 色妻视频观看 暴力插美女屁眼视频 48号缚师绑美女 草榴影院女同 空姐不愿意拍视频被男友强干到高潮的视频 sskanzyz新资源 小岛南无码链接magnet 小草草大黄瓜在线观看 renrencaome 虐阴漫画 国产第一页天天拍 wwwsesw 优酷 解放军在巴黎 18禁自拍偷拍 韩国肛交视频播放 美女做爱磁力连接 捆绑天海翼 5xsq四虎 美国十次啦福利视频 射精丝袜的视频 美女被人舔阴帝视频 乱伦香蕉色视频 色狐免费无码电影 美国成人性爱电影 六月丁香手机在线观看 教练和学生作爱视频 加勒比在线视频网 强奸乱伦1 骑士美女AV视频 去火涩 成人av视 ed2k 婷婷基地色色网 五月婷综合官网 撸必撸 足交鸣人漫画 gvg567 黑丝FJ 菲菲去网站 如果电话亭 avi rhj-073 黄片日屄视频 shiwaizhipaizhao 偷偷撸电影院 手机看片永久免费在线观看国产频道 国产另类自拍亚洲 天堂国产手机a 自拍视频在线观看 teen萝莉 不打码木叶性处理医院 风月海棠空姐 3d hy工房 在线观看 femmina在线观看 陈冠希艳照1300张阿娇 黄色网站在线视频 黄色综合网站 好看的中文字幕色拍拍噜 哦哦弟弟 黄色A片直播 后人动态图 qplayer在线播放网址 www第四色 野战情线国产视频在线观看 偷拍自拍在线赌博 护士无码视频 鬼佬　在线　云播 桃乃木香奈在线三上悠亚在线 ipx247在线观看 天降艳福不是福绝版在线看 一本道0588视频 日本无码光谍区 尻屄AV 老太太h类在线视频 国产3P自拍偷拍 伦理视频黄片大全 91小青蛙红杏出墙3p 内藤幸恵 你懂的 国产在线观啪啪啪网站 国产自拍、欧美 呦呦禁处 福利国产成人强奸少女50集 玉狼影院 国产偷拍自拍中文对白在线 阴道内窥镜凌辱女友 嘿嘿嘿影院永久 可知子 无码 成人在线黄色电影 快播八戒网 我要干色和尚 大胸美女一级黄色毛片 日本绣惑电影 yingyinxianfeng ziyuang 露脸操白妹国语 日本女性爱视频在线观看 成人狂欢福利网 91偷拍视频在线观看 操同事的小女友爱剪辑 粗暴轮奸视频 国产自自拍永久免费 天天干b天天插 国产25P 凹凸视频线观看免费 不雅视频磁力链接下载 香蕉影院超频在线视频 新加坡人美国艳星口交视频 肛塞自慰视频 8338磁力链接 成人色色网美国av幼女 把鸡巴插入妈妈的阴道 24美图野战 我和老师做爱漫画 淫乱肛胶美女 日撸神 清纯漂亮的嫩妹女孩与男友在家激情做爱流出高清视频mp4 阿姨熟女丝袜 黑丝袜电影院 色图网大全 丝袜阿姨的淫荡生活 日本乱伦变态 熟妇性交视频女人色网站 1234qec0m 六年级学生屄 波谷桐原 爆操巨乳妈妈 阿v天堂2012关于苍井空的视频 淫秽的我 官场艳情纪莜竹 少妇性交图25p