HardenedBSD/hardenedbsd 1e73724bin/pkill pkill.c, share/man/man9 sbuf.9

Merge remote-tracking branch 'origin/hardened/current/master' into 

HardenedBSD/hardenedbsd 5eacfd5bin/pkill pkill.c, share/man/man9 sbuf.9

Merge branch 'freebsd/current/master' into hardened/current/master

HardenedBSD/hardenedbsd 0923281share/man/man9 sbuf.9, sys/kern subr_sbuf.c

Implement simple record boundary tracking in sbuf(9) to avoid record splitting
during drain operations. When an sbuf is configured to use this feature by way
of the SBUF_DRAINTOEOR sbuf_new() flag, top-level sections started with
sbuf_start_section() create a record boundary marker that is used to avoid
flushing partial records.

Reviewed by:    cem,imp,wblock
MFC after:      2 weeks
Sponsored by:   Netflix, Inc.
Differential Revision:  https://reviews.freebsd.org/D8536

HardenedBSD/hardenedbsd cefde59bin/pkill pkill.c

The r322210 change to pgrep's PID delimiting behaviour causes pgrep's default
output to not include a trailing new line, which is a potential POLA violation
for existing consumers. Change pgrep to always emit a trailing new line on
completion of its output, regardless of the delimeter in use (which technically
is also a potential POLA violation for existing consumers that rely on the
pre-r322210 buggy behaviour, but a line has to be drawn somewhere).

PR:     221534
Submitted by:   kdrakehp zoho com
Reported by:    kdrakehp zoho com
MFC after:      1 week
X-MFC-with:     r322210
+2-01 files

HardenedBSD/hardenedbsd 328ea10sys/dev/hyperv/storvsc hv_vstorage.h, sys/dev/hyperv/utilities vmbus_timesync.c hv_kvp.c

MFC 322488
    hyperv: Update copyright for the files changed in 2017

    Sponsored by:   Microsoft
    Differential Revision:  https://reviews.freebsd.org/D11982

HardenedBSD/hardenedbsd b7cee9bsys/x86/x86 intr_machdep.c

Fix unused varable warning in !SMP case

Fallout from r322588.  I'm not sure why !SMP is a knob we have, but, we have

Reported by:    Michael Butler <imb AT protected-networks.net>
Sponsored by:   Dell EMC Isilon

HardenedBSD/hardenedbsd f72c265contrib/netbsd-tests/usr.bin/grep t_grep.sh, usr.bin/grep util.c grep.h

MFC r318574: bsdgrep: Correct per-line line metadata printing

Metadata printing with -b, -H, or -n flags suffered from a few flaws:

1) -b/offset printing was broken when used in conjunction with -o

2) With -o, bsdgrep did not print metadata for every match/line, just
   the first match of a line

3) There were no tests for this

Address these issues by outputting this data per-match if the -o flag is
specified, and prior to outputting any matches if -o but not --color,
since --color alone will not generate a new line of output for every
iteration over the matches.

To correct -b output, fudge the line offset as we're printing matches.

While here, make sure we're using grep_printline in -A context.  Context
printing should *never* look at the parsing context, just the line.

The tests included do not pass with gnugrep in base due to it exhibiting
similar quirky behavior that bsdgrep previously exhibited.

Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd 49a5aa9contrib/netbsd-tests/usr.bin/grep t_grep.sh, usr.bin/grep util.c

MFC r318571: bsdgrep: emit more than MAX_LINE_MATCHES per line

We should not set an arbitrary cap on the number of matches on a line,
and in any case MAX_LINE_MATCHES of 32 is much too low.  Instead, if we
match more than MAX_LINE_MATCHES, keep processing and matching from the
last match until all are found.

For the regression test, we produce 4096 matches (larger than we expect
we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096
lines of output with the -o flag.

We'll also make sure that every distinct line is getting its own line
number to detect line metadata not being printed as appropriate along
the way.

PR:            218811
Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd 66ff5b0contrib/netbsd-tests/usr.bin/grep t_grep.sh, usr.bin/grep file.c

bsdgrep: fix segfault with --mmap and add relevant test

MFC r318565: bsdgrep: fix segfault with --mmap

r313948 partially fixed --mmap behavior but was incomplete.  This commit
generally reverts it and does it the more correct way- by just consuming
the rest of the buffer and moving on.

MFC r318908: bsdgrep: add --mmap tests

Basic sanity tests as well as coverage for the bug fixed in r318565.

PR:            219402
Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd e4d5001contrib/netbsd-tests/usr.bin/grep t_grep.sh, usr.bin/grep grep.c util.c

bsdgrep: Don't allow negative context flags, add more tests

MFC r318302: bsdgrep: don't allow negative -A / -B / -C

Previously, when given a negative -A/-B/-C argument bsdgrep would
overflow the respective context flag(s) and exhibited surprising

Fix this by removing unsignedness of Aflag/Bflag and erroring out if
we're given a value < 0.  Also adjust the type used to track 'tail'
context in procfile() so that it accurately reflects the Aflag value
rather than overflowing and losing trailing context.

This also fixes an inconsistency previously existing between -n and
-C "n" behavior.  They are now both limited to LLONG_MAX, to be

Add some test cases to make sure grep errors out properly for both
negative context values as well as non-numeric context values rather
than giving bogus matches.

MFC r318317: bsdgrep: add more tests for different binary flags

The existing 'binary' test in netbsd-tests/ does a basic check of the
default treatment for binary behavior, but not much more than that.

    [5 lines not shown]

HardenedBSD/hardenedbsd 464673bcontrib/netbsd-tests/usr.bin/grep t_grep.sh

MFC r318004 (ngie): Remove expected failure that no longer fails with gnu
grep in base

Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd 9aa5397sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys abd.h, sys/crypto/aesni aesni_ghash.c aesni_wrap.c

Merge remote-tracking branch 'origin/hardened/current/master' into 

HardenedBSD/hardenedbsd 15d7dadsys/dev/hyperv/netvsc if_hn.c if_hnvar.h

MFC 322483,322485-322487

    hyperv/hn: Update VF's ibytes properly under transparent VF mode.

    While, I'm here add comment about why updating VF's imcast stat is
    not necessary.

    Sponsored by:   Microsoft
    Differential Revision:  https://reviews.freebsd.org/D11948

    hyperv/hn: Fix/enhance receiving path when VF is activated.

    - Update hn(4)'s stats properly for non-transparent mode VF.
    - Allow BPF tapping to hn(4) for non-transparent mode VF.
    - Don't setup mbuf hash, if 'options RSS' is set.
      In Azure, when VF is activated, TCP SYN and SYN|ACK go through hn(4)
      while the rest of segments and ACKs belonging to the same TCP 4-tuple
      go through the VF.  So don't setup mbuf hash, if a VF is activated
      and 'options RSS' is not enabled.  hn(4) and the VF may use neither
      the same RSS hash key nor the same RSS hash function, so the hash
      value for packets belonging to the same flow could be different!
    - Disable LRO.
      hn(4) will only receive broadcast packets, multicast packets, TCP

    [23 lines not shown]

HardenedBSD/hardenedbsd caec435sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys abd.h, sys/crypto/aesni aesni_ghash.c aesni_wrap.c

Merge branch 'freebsd/current/master' into hardened/current/master

HardenedBSD/hardenedbsd 04dad8eshare/mk src.opts.mk

MFC r322356: Mark PROFILE option as broken when targetting mips64

The assembly in sys/mips/include/profile.h will only work for o32 ABI.

Submitted by:   Alexander Richardson

HardenedBSD/hardenedbsd b63e9bfsys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys abd.h

Mark ZFS ABD inline functions static.

When built with -fno-inline-functions zfs.ko contains undefined references
to these functions if they are only marked inline.

Reviewed by:    avg (earlier version)
MFC after:      1 week
Sponsored by:   Chelsio Communications

HardenedBSD/hardenedbsd 520a2a0sys/crypto/aesni aesni_ghash.c aesni_wrap.c

aesni: quiet -Wcast-qual

Reviewed by:    delphij
Approved by:    markj (mentor)
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D12021

HardenedBSD/hardenedbsd 9f5428ausr.bin/grep grep.c

MFC r317705: bsdgrep: avoid use of magic number for REG_NOSPEC

Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd 6c1bf05lib/libc/i386 Makefile.inc, sys/boot/efi/include efi.h

Merge remote-tracking branch 'origin/hardened/current/master' into 

HardenedBSD/hardenedbsd 54409fdlib/libc/i386 Makefile.inc, sys/boot/efi/include efi.h

Merge branch 'freebsd/current/master' into hardened/current/master

HardenedBSD/hardenedbsd 4956db0sys/sys taskqueue.h kernel.h

Add SI_SUB_TASKQ after SI_SUB_INTR and move taskqueue initialization there for 

This fixes a regression accidentally introduced in r322588, due to an
interaction with EARLY_AP_STARTUP.

Reviewed by:    bdrewery@, jhb@
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D12053

HardenedBSD/hardenedbsd c6b1dcbcontrib/netbsd-tests/usr.bin/grep t_grep.sh d_context_e.in, usr.bin/grep util.c grep.c

Merge remote-tracking branch 'origin/hardened/11-stable/master' into 

HardenedBSD/hardenedbsd 0a59953contrib/netbsd-tests/usr.bin/grep t_grep.sh d_context_e.in, usr.bin/grep util.c grep.c

Merge remote-tracking branch 'origin/hardened/11-stable/master' into 

HardenedBSD/hardenedbsd ff7d06ccontrib/netbsd-tests/usr.bin/grep t_grep.sh d_context_e.in, usr.bin/grep util.c grep.c

Merge branch 'freebsd/11-stable/master' into hardened/11-stable/master

HardenedBSD/hardenedbsd 090fbd7sys/boot/efi/include efi.h

Define proposed GUID for FreeBSD boot loader variables.

HardenedBSD/hardenedbsd 357044esys/boot/efi/include efi.h

Remove unused defines.

HardenedBSD/hardenedbsd f9ac1eesys/netpfil/pf pf_lb.c

MFC r322280:
pf_get_sport(): Prevent possible endless loop when searching for an unused nat port

This is an import of Alexander Bluhm's OpenBSD commit r1.60,
the first chunk had to be modified because on OpenBSD the
'cut' declaration is located elsewhere.

Upstream report by Jingmin Zhou:

OpenBSD commit message:
 Use a 32 bit variable to detect integer overflow when searching for
 an unused nat port.  Prevents a possible endless loop if high port
 is 65535 or low port is 0.
 report and analysis Jingmin Zhou; OK sashan@ visa@
Quoted from: https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf_lb.c

PR:            221201
Submitted by:   Fabian Keil <fk at fabiankeil.de>
Obtained from:  OpenBSD via ElectroBSD

HardenedBSD/hardenedbsd 6ab3759sys/net bpf.c

bpf: Fix incorrect cleanup

Cleaning up a bpf_if is a two stage process. We first move it to the
bpf_freelist (in bpfdetach()) and only later do we actually free it (in

We cannot set the ifp->if_bpf to NULL from bpf_ifdetach() because it's
possible that the ifnet has already gone away, or that it has been assigned
a new bpf_if.
This can lead to a struct ifnet which is up, but has if_bpf set to NULL,
which will panic when we try to send the next packet.

Keep track of the pointer to the bpf_if (because it's not always
ifp->if_bpf), and NULL it immediately in bpfdetach().

PR:            213896
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D11782
+3-71 files

HardenedBSD/hardenedbsd 811a0d8lib/libc/i386 Makefile.inc

force use of ld.bfd for linking i386 libc, even when using lld

lld can successfully link most of a working i386 userland and kernel,
but produces a broken libc. For now if we're otherwise using lld, and
ld.bfd is available, explicitly use it for libc.

Sponsored by:   The FreeBSD Foundation

HardenedBSD/hardenedbsd 5a79b72sys/amd64/include intr_machdep.h, sys/i386/include intr_machdep.h

x86: Add dynamic interrupt rebalancing

Add an option to dynamically rebalance interrupts across cores
(hw.intrbalance); off by default.

The goal is to minimize preemption. By placing interrupt sources on distinct
CPUs, ithreads get preferentially scheduled on distinct CPUs.  Overall
preemption is reduced and latency is reduced. In our workflow it reduced
"fighting" between two high-frequency interrupt sources.  Reduced latency
was proven by, e.g., SPEC2008.

Submitted by:   jeff@ (earlier version)
Reviewed by:    kib@
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D10435

HardenedBSD/hardenedbsd 288fba5contrib/netbsd-tests/usr.bin/grep t_grep.sh d_context_e.in, usr.bin/grep util.c grep.c

bsdgrep: fix -w flag matching with an empty pattern

MFC r317703: bsdgrep: fix -w flag matching with an empty pattern

-w flag matching with an empty pattern was generally 'broken', allowing
matches to occur on any line whether or not it actually matches -w

This fix required a good amount of refactoring to address.  procline()
is altered to *only* process the line and return whether it was a match
or not, necessary to be able to short-circuit the whole function in case
of this matchall flag. -m flag handling is moved out as well because it
suffers from the same fate as context handling if we bypass any actual
pattern matching.

The matching context (matches, mostly) didn't previously exist outside
of procline(), so we go ahead and create context object for file
processing bits to pass around.  grep_printline() was created due to
this, for the scenarios where the matches don't actually matter and we
just want to print a line or two, a la flushing the context queue and
no -o or --color specified.

Damage from this broken behavior would have been mitigated by the fact
that it is unlikely users would invoke grep -w with an empty pattern.

    [18 lines not shown]

HardenedBSD/hardenedbsd 92a8bccusr.bin/grep/regex tre-fastmatch.c

bsdgrep: fix escape map building when using TRE (BSD_GREP_FASTMATCH)

MFC r317700: bsdgrep: use calloc where appropriate in grep's tre-fastmatch

Also apply style(9) to a related NULL check.

MFC r317701: bsdgrep: correct test sense from r317700

Kyle's change in review D10098 was correct. I introduced the error when
extracting a portion of that change.

MFC r317704: bsdgrep: fix escape map building for multibyte strings

In BSD grep, fix escape map building in the regex parser. It was
previously using memory not explicitly initialized, and the MBS escape
map was being built based on a version of the pattern with escapes
already parsed out.

This is Kyle's change, but I restored the broken style that already
exists in this file.

Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd df9d78c. Makefile, share/mk bsd.crunchgen.mk

Quote ${MAKE} when passing in env in case it contains spaces.

Downstream we are wrapping MAKE with a limits(1) call which
interferes with these non-quoted cases.

Sponsored by:   Dell EMC Isilon

HardenedBSD/hardenedbsd a9c64fbusr.bin/grep grep.c Makefile, usr.bin/grep/nls ru_RU.KOI8-R.msg pt_BR.ISO8859-1.msg

MFC r317678: bsdgrep: Add GNU compatible version string indicator

As reported in r218614 it's useful to have an indication of whether or not
BSD grep was built with GNU_GREP_COMPAT.

Approved by:    emaste (mentor, blanket MFC)

HardenedBSD/hardenedbsd b2428cacontrib/netbsd-tests/usr.bin/grep t_grep.sh, usr.bin/grep util.c

MFC r317665: bsdgrep: fix -w -v matching improperly with certain patterns

-w and -v flag matching was mostly functional but had some minor

1. -w flag processing only allowed one iteration through pattern
   matching on a line. This was problematic if one pattern could match
   more than once, or if there were multiple patterns and the earliest/
   longest match was not the most ideal, and

2. Previous work "fixed" things to not further process a line if the
   first iteration through patterns produced no matches. This is clearly
   wrong if we're dealing with the more restrictive -w matching.

#2 breakage could have also occurred before recent broad rewrites, but
it would be more arbitrary based on input patterns as to whether or not
it actually affected things.

Fix both of these by forcing a retry of the patterns after advancing
just past the start of the first match if we're doing more restrictive
-w matching and we didn't get any hits to start with. Also move -v flag
processing outside of the loop so that we have a greater change to match
in the more restrictive cases. This wasn't strictly wrong, but it could
be a little more error prone.

    [6 lines not shown]

HardenedBSD/hardenedbsd db6ebb6share/man/man5 src.conf.5, tools/build/options WITHOUT_BSD_GREP_FASTMATCH

MFC r317254: bsdgrep: add BSD_GREP_FASTMATCH knob for built-in fastmatch

Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review

In comparison with the fastmatch implementation, Kyle Evans found that:

- regex(3)'s performance with literal expressions offers a speed
  improvement over fastmatch

- regex(3)'s performance, both with simple BREs and EREs, seems to be

The regex implementation was imported in r226035, and the commit message

    This is a temporary solution until the whole regex library is
    not replaced so that BSD grep development can continue and the
    backported code gets some review and testing. This change only
    improves scalability slightly, there is no big performance boost
    yet but several minor bugs have been found and fixed.

Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.

    [5 lines not shown]

HardenedBSD/hardenedbsd dd0c1c9tools/build/mk OptionalObsoleteFiles.inc

remove debug files in delete-old* when WITHOUT_DEBUG_FILES

Reported by:    brd
Reviewed by:    bdrewery, brd
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D12044

HardenedBSD/hardenedbsd 03be283sys/kern subr_intr.c

Fix compile error with option DEBUG.  This is fallout from some long-ago
INTRNG refactoring that didn't get caught at the time because code in a
debugf() statement isn't compiled unless DEBUG is defined.

PR:            221557

HardenedBSD/hardenedbsd 8f152e9sys/amd64/include sgxreg.h cpufunc.h, sys/amd64/sgx sgx.c sgx_linux.c

Merge remote-tracking branch 'origin/hardened/current/master' into 

HardenedBSD/hardenedbsd 2dfd80csys/amd64/sgx sgx.c, usr.sbin/vidcontrol vidcontrol.c

Merge branch 'freebsd/current/master' into hardened/current/master

HardenedBSD/hardenedbsd 07265a6sys/dev/ahci ahci_pci.c, sys/dev/ata ata-pci.h

Merge remote-tracking branch 'origin/hardened/11-stable/master' into 

HardenedBSD/hardenedbsd c5fc68csys/dev/ahci ahci_pci.c, sys/dev/ata ata-pci.h

Merge remote-tracking branch 'origin/hardened/11-stable/master' into 

HardenedBSD/hardenedbsd 51f69afsys/dev/ahci ahci_pci.c, sys/dev/ata ata-pci.h

Merge branch 'freebsd/11-stable/master' into hardened/11-stable/master

HardenedBSD/hardenedbsd 754d087usr.sbin/vidcontrol vidcontrol.c

Fix setting of the border color.  Teken doesn't support syscons' escape
sequence "ESC [ %d A" for this although that was used here.  I will fix
teken later, but use the more portable ioctl KDSBORDER here.

The ioctl is also much easier to use if you check that it works.  For
-b, check it and complain and exit if it failed, so that it is more
obvious that that vt doesn't support border colors.  Don't check it
when restoring the border color in revert(), since revert() is used
on vt for handling other errors.

Fix nearby error handling and style.  For the error of an invalid
color, revert() and print a specific error message using err() instead
of not revert()ing and printing spam using usage().

HardenedBSD/hardenedbsd 4b4a278sys/amd64/sgx sgx.c

Rename macro DEBUG to SGX_DEBUG.

This fixes LINT kernel build.

Reported by:    lwhsu
Sponsored by:   DARPA, AFRL

HardenedBSD/hardenedbsd 3c5634dusr.bin/grep util.c

MFC r303444 (ed): Call basename() in a portable way.

Pull a copy of the filename string before calling basename(). Change the
loop to not return on its own, so we can put a free() statement at the

Approved by:    emaste (mentor, blanket MFC)
+10-51 files

HardenedBSD/hardenedbsd f189e68sys/amd64/include sgxreg.h cpufunc.h, sys/amd64/sgx sgx.c sgx_linux.c

Merge remote-tracking branch 'upstream/master' into hardened/current/master

HardenedBSD/hardenedbsd d3f829dsys/netipsec key.c

MFC r322328:
  Make user supplied data checks a bit stricter.

  key_msg2sp() is used for parsing data from setsockopt(IP[V6]_IPSEC_POLICY)
  call. This socket option is usually used to configure IPsec bypass for
  socket. Only privileged user can set this socket option.
  The message syntax is described here

  and our libipsec is usually used to create the correct request.
  Add additional checks:
  * that sadb_x_ipsecrequest_len is not out of bounds of user supplied buffer
  * that src/dst's sa_len is the same
  * that 2*sa_len is not out of bounds of user supplied buffer
  * that 2*sa_len fits into bounds of sadb_x_ipsecrequest

  Reported by:  Ilja van Sprundel
+21-51 files

HardenedBSD/hardenedbsd 84d82b3sys/dev/syscons syscons.c, sys/sys consio.h

Undeprecate the CONS_CURSORTYPE ioctl.  It was "deprecated" in 2001,
but it was actually extended then and it is still used (just once) in
/usr/src by its primary user (vidcontrol), while its replacement is
still not used in /usr/src.

yokota became inactive soon after deprecating CONS_CURSORTYPE (this
was part of a large change to make cursor attributes per-vty).

vidcontrol has incomplete support even for the old ioctl.  I will
update it soon.  Then there are many broken escape sequences to fix.
This is just to prepare for setting cursor colors using vidcontrol.

HardenedBSD/hardenedbsd c51ad5fsys/amd64/include sgxreg.h cpufunc.h, sys/amd64/sgx sgx.c sgx_linux.c

Add support for Intel Software Guard Extensions (Intel SGX).

Intel SGX allows to manage isolated compartments "Enclaves" in user VA
space. Enclaves memory is part of processor reserved memory (PRM) and
always encrypted. This allows to protect user application code and data
from upper privilege levels including OS kernel.

This includes SGX driver and optional linux ioctl compatibility layer.
Intel SGX SDK for FreeBSD is also available.

Note this requires support from hardware (available since late Intel
Skylake CPUs).

Many thanks to Robert Watson for support and Konstantin Belousov
for code review.

Project wiki: https://wiki.freebsd.org/Intel_SGX.

Reviewed by:    kib
Relnotes:       yes
Sponsored by:   DARPA, AFRL
Differential Revision:  https://reviews.freebsd.org/D11113