-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGES
25195 lines (15861 loc) · 880 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
3.3.0-dev.543 | 2020-11-16 11:50:49 -0800
* GH-352: Improve HTTP::match_sql_injection_uri regex (Jon Siwek, Corelight)
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex. This change reduces some false positives.
3.3.0-dev.541 | 2020-11-16 11:22:00 -0800
* Simplify ssh/main.zeek by using "ssh_server_host_key" for fingerprinting (Jon Siwek, Corelight)
* Deprecate "ssh1_server_host_key" parameters *e* and *p* (Jon Siwek, Corelight)
They are named such that *e* is actually the modulus, not the exponent.
The replacement parameters are named *exponent* and *modulus* for
clarity.
* GH-1264: Implement "ssh_server_host_key" event (Jon Siwek, Corelight)
This event provides host key fingerprints for both SSH1 and SSH2.
3.3.0-dev.537 | 2020-11-16 11:03:05 +0000
* [SSH] Handle SSH version 1.99
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.
1.99 : 1.99 => 2.0
1.99 : 1.x => 1.x
1.99 : 2.0 => 2.O
(see "Compatibility With Old SSH Versions" in RFC 4253) (Brevet Vivien)
3.3.0-dev.534 | 2020-11-12 14:31:10 -0800
* Move UnknownProtocol options to init-bare.zeek (Jon Siwek, Corelight)
Otherwise the `unknown_protocol` event cannot be used independently
from `policy/mic/unknown-protocols.zeek`.
* Coverity 1436183: Initialize packet_analysis::Manager fields (Jon Siwek, Corelight)
* GH-1273: Change SizeExpr to yield "any" type when operating on "any" (Jon Siwek, Corelight)
* Add enum_names() BIF to return names of an enum type's values (Jon Siwek, Corelight)
* Add type_aliases() BIF for introspecting type-names of types/values (Jon Siwek, Corelight)
* Change Type::type_aliases map to store IntrusivePtr (Jon Siwek, Corelight)
And deprecate Type::GetAliases() and Type::AddAlias() since they
took raw pointers. Now replaced with Type::Aliases() and
Type::RegisterAlias().
* Fix lookup_ID() BIF to return enum values (Jon Siwek, Corelight)
Looking up an enum value from a string equal to its name previously
returned "<no ID value>".
3.3.0-dev.524 | 2020-11-12 12:16:00 -0700
* Move 'using namespace' declaration after other includes (Tim Wojtulewicz, Corelight)
* Use binpac.h as if it was an external dependency (Tim Wojtulewicz, Corelight)
* Fix includes of bif.h and _pac.h files to use full paths inside build directory (Tim Wojtulewicz, Corelight)
* Remove unnecessary include of NetVar.h from packet analysis plugins (Tim Wojtulewicz, Corelight)
* GH-1079: Use full paths starting with zeek/ when including files (Tim Wojtulewicz, Corelight)
* Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
The logging manager's Manager::TraverseRecord(), called when adding a
log filter to a stream, skipped any fields intoduced by a filter's
$ext_func when such fields weren't mentioned in a $include restriction
or mentioned in an $exclude restriction. This was inconsistent with
Manager::RecordToFilterVals, used when actually writing log entries,
which does include those values.
The result was that the record indices descent in Manager::RecordToFilterVals
expects to find only record values, when in fact only the record
provided by ext_func is present. This leads to type mismatches and
hard Zeek exits like this one:
1300475173.475401 fatal error in zeek/share/zeek//base/init-bare.zeek, line 4810: Val::CONVERTER (string/record) (zeek)
The fix makes ext_func's field additions decisive, meaning the
filter's include/exclude lists don't apply to it. If a user really
wants to override this, they can reset the filter's ext_func back to
our no-op default.
The included btest produces the above error when the fix is not present. (Christian Kreibich, Corelight)
3.3.0-dev.516 | 2020-11-11 12:53:02 -0700
* Support for additional DNS RR Type: LOC[29], SSHFP[44], NSEC3PARAM[51], custom BIND9 signaling[65534] (Fatema BW)
3.3.0-dev.514 | 2020-11-10 14:05:51 -0700
* GH-1269: Fix LogAscii::enable_leftover_log_rotation crash in bad dirs
Running with that option enabled inside a bad directory (e.g. lack of
permissions) crashed due to not checking for failure of opendir(). (Jon Siwek, Corelight)
3.3.0-dev.512 | 2020-11-10 13:55:33 -0700
* Change Debian 10 CI config to use GCC (Jon Siwek, Corelight)
* Update external testing repo hashes for canonification updates (Tim Wojtulewicz, Corelight)
3.3.0-dev.509 | 2020-11-10 08:41:54 -0700
* GH-1221: Add unknown_protocols.log for logging packet analyzer lookup failures (Tim Wojtulewicz, Corelight)
* Remove default_analyzer for Ethernet packet analzyer (Tim Wojtulewicz, Corelight)
3.3.0-dev.506 | 2020-11-09 16:27:04 -0800
* Add CI support for Debian 10, Fedora 31 & 33, FreeBSD 11, Ubuntu 20.04 (Christian Kreibich, Corelight)
This also adds two environment variables to the Dockerfiles for the
Debian/Ubuntu flavors to avoid stalling on user input when configuring
tzdata.
3.3.0-dev.504 | 2020-11-09 10:56:09 -0700
* Remove now-unused Packet::l2_valid field (Tim Wojtulewicz, Corelight)
* GH-1186: Remove Packet::hdr_size and uses of it.
This change also removes Packet::IP(), since Packet now contains an ip_hdr member
that points at the IP header if it exists. (Tim Wojtulewicz, Corelight)
3.3.0-dev.501 | 2020-11-04 10:41:56 -0800
* GH-1252: rename files with colons for Windows compatibility (Jon Siwek, Corelight)
A short-term measure so that the Zeek source tree can simply exist on a
Windows filesystem. For true support, the logic that decides/generates
the filename format will need to change.
* Use std::function instead of a function pointer in packet_analysis::Component (Tim Wojtulewicz, Corelight)
3.3.0-dev.495 | 2020-11-02 12:03:52 -0700
* GH-1215: Remove dispatch_map from packet analysis, replace with BIF methods for registering dispatches (Tim Wojtulewicz, Corelight)
3.3.0-dev.493 | 2020-11-02 09:51:03 -0800
* GH-1256: Write out strerror when writing errno during safe_write (Tim Wojtulewicz, Corelight)
3.3.0-dev.490 | 2020-10-27 13:41:27 -0700
* updated to mention notice_alarm.log by name. That has taken the place of the
file formerly named alarm.log (Duffy O'Craven)
3.3.0-dev.488 | 2020-10-26 11:03:51 -0700
* Improve how Zeekygen generated record/enum redefinition docs
It now provides a summary of the new fields/enums added by any given
redefinition along with associated commentary. (Jon Siwek, Corelight)
3.3.0-dev.486 | 2020-10-26 10:41:48 -0700
* GH-1245: require TLD of hostname literals to start with a letter (Jon Siwek, Corelight)
3.3.0-dev.484 | 2020-10-23 14:47:44 -0700
* Add coveralls badge to README.md (Tim Wojtulewicz)
* Remove the entire auxil directory from the results (Tim Wojtulewicz)
* Prevent non-master builds, add cirrus information to upload (Tim Wojtulewicz)
* Add support for sending data to coveralls with lcov_html.sh via Cirrus (Tim Wojtulewicz, Corelight)
3.3.0-dev.479 | 2020-10-23 12:25:15 -0700
* GH-251 (revert): remove coercion-to-signed-integer for |x| expressions (Jon Siwek, Corelight)
For `|x|`, where `x` is an expression with an integral result, an
implicit coercion of that result into signed `int` type no longer takes
place.
This was actually the behavior before Zeek 3.0 as well, but the attempt
to prevent mistakes that easily result from integer literals in Zeek
being unsigned like `|5 - 9|` causing an overflow/wraparound and
yielding a very large number is not generally consistent since overflows
are still generally able to happen in other ways and also in other
contexts besides just absolute-values. So the preference was to revert
to a behavior that favors consistency. For reference, see
https://github.com/zeek/zeek/pull/251#issuecomment-713956976
3.3.0-dev.476 | 2020-10-22 15:59:56 -0400
* Add an option to ignore packets sourced from particular subnets.
It's implemented with a new set[subnet] option named ignore_checksums_nets.
If you populate this set with subnets, any packet with a src address within
that set of subnets will not have it's checksum validated. (Seth Hall, Corelight)
* Update submodule(s) [nomail] (Jon Siwek, Corelight)
3.3.0-dev.467 | 2020-10-21 11:06:18 -0700
* Fix a couple of Coverity findings (1433618, 1433619) (Tim Wojtulewicz, Corelight)
3.3.0-dev.465 | 2020-10-20 13:34:20 +0000
* Bump BTest to 0.64.
3.3.0-dev.464 | 2020-10-20 08:43:40 +0000
* Update BTest to 0.63. New test baselines will now be store
canonified. (Robin Sommer, Corelight)
3.3.0-dev.463 | 2020-10-19 18:57:00 -0700
* Add length checks for IP header values before parsing TCP/UDP (Tim Wojtulewicz, Corelight)
3.3.0-dev.461 | 2020-10-19 11:54:23 -0700
* Fix deprecation warning in POP3 fuzzer (Jon Siwek, Corelight)
* Add header length check to GRE packet analyzer (Tim Wojtulewicz, Corelight)
3.3.0-dev.456 | 2020-10-16 12:41:58 -0700
* Avoid passing null pointers to memcmp in ZeekString comparison methods (Jon Siwek, Corelight)
3.3.0-dev.454 | 2020-10-16 10:34:53 -0700
* Change ICMP Neighbor Discovery option length storage to a uint16 (Vlad Grigorescu)
This fixes an overflow in the calculation of option lengths in
ICMP Neighbor Discovery messages.
3.3.0-dev.451 | 2020-10-16 07:09:43 +0000
* Make event ordering deterministic
NetControl::init and filter_change_tracking could basically be raised in
random order. (Johanna Amann, Corelight)
* Introduce generate_all_events bif and add option to misc/dump-events
generate_all_events causes all events to be raised internally; this
makes it possible for dump_events to really capture all events (and not
just those that were handled).
Addresses GH-169 (Johanna Amann, Corelight)
3.3.0-dev.444 | 2020-10-15 13:25:12 -0700
* Rework Sessions::Weird (Tim Wojtulewicz, Corelight)
* Prevent String::Set from crashing if passed a nullptr (Tim Wojtulewicz, Corelight)
* Store packet's ip header as unique_ptr (Tim Wojtulewicz, Corelight)
* Remove some unused includes from Packet.h (Tim Wojtulewicz, Corelight)
* Use shared_ptr for encapsulation data instead of raw pointer (Tim Wojtulewicz, Corelight)
* Review cleanup (Tim Wojtulewicz, Corelight)
* Update external testing repo hashes (Tim Wojtulewicz, Corelight)
* Reorder Packet member variables slightly for better packing (Tim Wojtulewicz, Corelight)
* Store the ip header in the packet after processing, reuse other places (Tim Wojtulewicz, Corelight)
* Change to store data in packet directly instead of keystore (Tim Wojtulewicz, Corelight)
* Don't always insert data into keystore for tunnels (Tim Wojtulewicz, Corelight)
* Add comment about packet header size and session analysis (Tim Wojtulewicz, Corelight)
* Set data to ip header's payload instead of advancing the pointer (Tim Wojtulewicz, Corelight)
* Move packet dumping to packet_mgr (Tim Wojtulewicz, Corelight)
* Make Sessions::NextPacket call packet_mgr, fix fuzzer code to do the same (Tim Wojtulewicz, Corelight)
* Move IP and IP tunnel code from Sessions into packet analyzers (Tim Wojtulewicz, Corelight)
* Prep work for IP changes
- Move all of the time handling code out of PktSrc into RunState
- Call packet_mgr->ProcessPacket() from various places to setup layer 2 data in packets (Tim Wojtulewicz, Corelight)
3.3.0-dev.426 | 2020-10-14 10:47:37 -0700
* GH-1211: Improve error message for already-defined functions (Jon Siwek, Corelight)
3.3.0-dev.422 | 2020-10-13 16:26:24 -0700
* GH-1208: Use Dictionary validity assertions only during CI (Jon Siwek, Corelight)
3.3.0-dev.420 | 2020-10-13 15:34:02 -0700
* Change "Cluster::Node$p" field to use "&default=0/unknown" (Seth Hall, Corelight)
This effectively makes defining the listening port optional in the cluster
node layout config, with "0/unknown" meaning the node is not pre-configured
to listen for incoming connections from other cluster nodes.
3.3.0-dev.414 | 2020-10-13 13:49:05 -0700
* Update cmake submodule to pull in fix for building libkqueue (Tim Wojtulewicz, Corelight)
* GH-1063: Update libkqueue to fix pf_ring-zc failures (Tim Wojtulewicz, Corelight)
3.3.0-dev.411 | 2020-10-13 13:38:38 -0700
* Remove unused LoginConn type and variable in Conn.h (Tim Wojtulewicz, Corelight)
3.3.0-dev.409 | 2020-10-13 13:32:18 -0700
* Make it possible to pass command line options through to scripts. (Seth Hall, Corelight)
A new ``zeek_script_args`` variable contains a list of arguments passed
to a script. E.g. either when explicitly executing Zeek like
``zeek -- myscript.zeek -arg1 -arg2``, or when using Zeek to interpret
executable scripts that contain a hashbang line at the top like::
#!/usr/local/zeek/bin/zeek --
3.3.0-dev.403 | 2020-10-13 10:50:12 -0700
* Add new Pcap::findalldevs() BIF (Seth Hall, Corelight)
* Remove superfluous RuleCondition destructors (Jon Siwek, Corelight)
* Silence Clang's warning about ignoring GCC's maybe-uninitialized warning (Jon Siwek, Corelight)
* Add reference to network_time_init from zeek_init docs (Jon Siwek, Corelight)
3.3.0-dev.390 | 2020-10-12 17:43:15 -0700
* Improve documentation for zeek_init event scheduling pitfalls (Jon Siwek, Corelight)
3.3.0-dev.388 | 2020-10-12 17:02:20 -0700
* Add CaptureLoss::Too_Little_Traffic notice (Vlad Grigorescu)
* Add CaptureLoss::initial_watch_interval for a quick read on cluster health after startup. (Vlad Grigorescu)
* Improve capture-loss.zeek documentation. (Vlad Grigorescu)
* Fix whitespace in capture-loss.zek (Vlad Grigorescu)
3.3.0-dev.381 | 2020-10-12 11:15:29 -0700
* GH-779: Add "udp-state" signature condition (Jon Siwek, Corelight)
It accepts "originator" or "responder" states as a way to enforce that
the signature only matches packets in the associated direction.
The "established" state is rejected as an error since it doesn't
have a useful meaning like it does for the "tcp-state" condition.
* Rename RuleConditionTCPState::TCPState enum values (Jon Siwek, Corelight)
* Rename "tcp-state" signature parser tokens to not be TCP-specific (Jon Siwek, Corelight)
3.3.0-dev.377 | 2020-10-08 12:07:42 -0700
* GH-1200: ignore a maybe-uninitialized warning (Jon Siwek, Corelight)
3.3.0-dev.374 | 2020-10-07 10:44:11 -0700
* Fix multipart MIME leak of sub-part found after closing-boundary (Jon Siwek, Corelight)
After detecting a closing-boundary for a given multipart MIME entity, it
enters into an "end of data" state, however any subsequent boundary
delimiter could still cause the allocation of a sub-entity object that
is never released due to cleanup logic being bypassed upon finding the
"end of data" state already reached.
This change prevents allocation/processing of sub-entities after the
"end of data" state is reached (e.g. from detecting a multipart
closing-boundary). This new behavior still aligns with RFC 2046
expectations:
"There appears to be room for additional information prior to the first
boundary delimiter line and following the final boundary delimiter line.
These areas should generally be left blank, and implementations must
ignore anything that appears before the first boundary delimiter line or
after the last one."
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26027
(Link to details becomes public 30 days after patch release)
* Btests for log filter policy hooks (Christian Kreibich, Corelight)
* Btest baseline updates to reflect new logging policy hooks (Christian Kreibich, Corelight)
* Migrate existing use of filter predicates to policy hooks (Christian Kreibich, Corelight)
* Support for log filter policy hooks (Christian Kreibich, Corelight)
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
3.3.0-dev.365 | 2020-10-06 12:19:49 -0700
* GH-425: Avoid temporary value while coercing records already of the right type. (Robin Sommer, Corelight)
The combination of this commit with the previous one now lets the examples
in GH-425 all execute with the same performance.
* GH-425: Optimize record constructor expression. (Robin Sommer, Corelight)
We remove the inheritance from UnaryExpr because we know the type of the
operand precisely and can skip a temporary when evaluating the expression.
* Unify type comparisions for records. (Robin Sommer, Corelight)
For records, same_type(r1, r2) would not check if the fields'
attributes match as well. That seems like an oversight, and some
callers of same_type() did indeed add that check on their end. This
commit moves the check into same_type() itself. That generally doesn't
seem make any differences except for a couple of places validating
code, which we update a bit. That in turn leans to slightly different
(better?) error messages for a couple of test cases.
3.3.0-dev.361 | 2020-10-06 10:13:37 -0700
* logging/ascii: Support leftover log rotation in non-supervisor setups (Arne Welzel, Corelight)
Allow enabling leftover log rotation through
LogAscii::enable_leftover_log_rotation and redef this for the
logger node in a supervisor setup individually.
3.3.0-dev.359 | 2020-10-02 10:49:35 -0700
* GH-1180: Add external dependency include paths to `zeek-config --include_dir` (Jon Siwek, Corelight)
This helps ensure successful compilation of a plugin that includes a
Zeek header which includes an external dependency header even if the
plugin itself doesn't necessarily depend on that external library.
3.3.0-dev.357 | 2020-10-02 10:26:11 -0700
* Switch one's complement checksum implementation (Jon Siwek, Corelight)
Borrows the `in_cksum` code from tcpdump, which borrowed from FreeBSD.
It handles unaligned data better and also unrolls the inner loop to
process 16 two-byte values at a time versus 2 one-byte values at a time
in the previous version. Generally measured as ~1.5x faster in a
release build. The new API should generally be more amenable to any
future optimization explorations since all relevant data blocks are
available within a single call rather than spread across multiple.
3.3.0-dev.355 | 2020-09-29 17:08:44 -0700
* Improve broker/ssl_auth_failure.zeek test (Jon Siwek, Corelight)
There was a race that previously may cause it to not truly test
whether the connecting-process exited due to SSL authentication
failure or just because the listening-process wasn't available yet
(e.g. due to process scheduling variability).
3.3.0-dev.352 | 2020-09-25 15:09:29 -0700
* GH-160: change find-filtered-trace to ignore pcaps with any non-TCP traffic (Robin Sommer, Corelight)
3.3.0-dev.350 | 2020-09-25 14:37:58 -0700
* Add dce_rpc_request_stub and dce_rpc_response_stub events (Yacin Nadji, Corelight)
* Fix namespace warning in fuzzer code (Tim Wojtulewicz, Corelight)
3.3.0-dev.346 | 2020-09-24 16:03:28 -0700
* Fix a Sphinx warning about misformatted packet analyzer comment (Jon Siwek, Corelight)
* Add Zeekygen documentation support for packet analyzers (Jon Siwek, Corelight)
* Move packet_mgr to the zeek namespace (Tim Wojtulewicz, Corelight)
3.3.0-dev.341 | 2020-09-24 08:16:45 -0700
* Fix negative-value-left-shift undefined behavior in patricia trie (Jon Siwek, Corelight)
* Improve negation of ConstExpr (Jon Siwek, Corelight)
* Instead of creating a NegExpr for negation of a literal/constant,
a ConstExpr is now created directly.
* For negation of integer literals, there's now an additional check
for whether the integer would be outside the range of possible 'int'
values. This can also help prevent the undefined behavior due to
overflow as a result of trying to represent the minimum 'int' value of
-9223372036854775808 as a literal in a script -- the unsigned value is
cast to signed yielding INT64_MIN, then INT64_MIN is negated.
* Avoid signed integer overflow when combining SMB header PID bits (Jon Siwek, Corelight)
Such an overflow invokes undefined behavior.
* Avoid unary negation of INT64_MIN in modp_litoa10 (Jon Siwek, Corelight)
Overlow can occur in that case, which is undefined behavior.
* Avoid double-to-int conversion overflows in modp_dtoa functions (Jon Siwek, Corelight)
Those methods already had a fallback to use sprintf() for large values
except:
* The check-for-large-value was unnecessarily done after many
operations that aren't relevant to the check and those operations can
result in a conversion overflow (undefined behavior).
* The check-for-large-value was using the literal value for a
32-bit INT_MAX instead of just using INT_MAX. For a platform where
`int` is less than 32-bits, the same conversion overflow from the
previous point could still occur (undefined behavior).
* The check-for-large-value was not inclusive of INT_MAX.
In a case where the conversion of INT_MAX itself to a double
can't be represented exactly, it's implementation-defined whether
the closest higher or closest lower representable-value is selected.
If the higher value is selected, then a `double` value comparing equal
to INT_MAX-as-converted-to-double would cause an overflow of an `int`
upon conversion (undefined behavior).
* Fix divide-by-zero in Entropy analyzer (Jon Siwek, Corelight)
* Fix divide-by-zero in stats/profiling memory usage calculation (Jon Siwek, Corelight)
* Fix uninitialized field in POP3 fuzzer (Jon Siwek, Corelight)
* Add framework for running UndefinedBehaviorSanitizer in CI (Jon Siwek, Corelight)
3.3.0-dev.329 | 2020-09-23 11:32:06 -0700
* Update NEWS (Tim Wojtulewicz, Corelight)
* Fix build on FreeBSD, which was missing full definition of sockaddr for ARP (Tim Wojtulewicz, Corelight)
* Update hashes for external testing repos (Tim Wojtulewicz, Corelight)
* Fix handling of session analysis.
This reverts commit 0e51be18ac59d7a176f18780f1f8df85561449b9. (Jan Grashoefer)
* Fix Sessions::DoNextPacket to ignore ARP packets and not report weirds for them (Tim Wojtulewicz, Corelight)
* Packet analysis cleanup. (Jan Grashoefer)
* Minor cleanup (Tim Wojtulewicz, Corelight)
* Simplify packet analyzer config. (Jan Grashoefer)
* Make default packet analyzer definition explicit. (Jan Grashoefer)
* Add explicit root analyzer for packet analysis. (Jan Grashoefer)
* Extend packet analysis test. (Jan Grashoefer)
* Simplify MPLS analysis. (Jan Grashoefer)
* Improve packet analysis data flow. (Jan Grashoefer)
* Improve packet analyzer API. (Jan Grashoefer)
* Rename DefaultAnalyzer to IP. (Jan Grashoefer)
* Move ARP analysis into packet analyzer. (Jan Grashoefer)
* Small cleanup of packet analysis. (Jan Grashoefer)
* Fix some build issues from the rebase (Tim Wojtulewicz, Corelight)
* Remove encap_hdr_size (replaced by skip analyzer). (Jan Grashoefer)
* Add skip analyzer test and update baselines. (Jan Grashoefer)
* Allow to overwrite packet analysis mappings. (Jan Grashoefer)
* Add SkipAnalyzer.
This is WIP: The test case would require a new pcap or the possibility
to overwrite analyzer mappings. The CustomEncapsulationSkip method and
the corresponding options need to be removed. (Jan Grashoefer)
* Suggested code improvements for packet analysis. (Jan Grashoefer)
* Improve naming in packet analysis. (Jan Grashoefer)
* Migrate all packet analyzers to new API. (Jan Grashoefer)
* Remove magic identifiers from Ethernet analyzer. (Jan Grashoefer)
* Bring back default packet analysis.
Default analyzers can be configured per packet analyzer by omitting the
identifier in the ConfigEntry. (Jan Grashoefer)
* Further simplified the packet analysis API.
This is still WIP and includes the following changes:
* Dispatchers are now part of analyzers (moving dispatching logic from
the manager to the analyzers)
* All available analyzers are instantiated on start up
* Removal of configuration class (Jan Grashoefer)
* Move dispatching into packet analyzers.
WIP that updates only the Ethernet analyzer. (Jan Grashoefer)
* Move cur_pos from packet into packet manager loop. (Jan Grashoefer)
* Remove packet_analysis/Defines.h
- Replace uses of identifier_t with uint32_t
- Replace repeated usage of tuple type for Analysis results with type alias (Tim Wojtulewicz, Corelight)
* Move VectorDispatcher to be the only dispatcher (Tim Wojtulewicz, Corelight)
* Remove Manager::Reset() method (Tim Wojtulewicz, Corelight)
* Remove enabled state from Components, ability to enable/disable from Manager (Tim Wojtulewicz, Corelight)
* Use shared_ptr instead of raw pointers in packet_analysis for analyzers and dispatchers (Tim Wojtulewicz, Corelight)
* Whitespace fixes from review (Tim Wojtulewicz, Corelight)
* Reorganize some pointer handling (Tim Wojtulewicz, Corelight)
* Fix ordering of debug logger strings to match the enum (Tim Wojtulewicz, Corelight)
* Fixes for various btest issues
- Fix handling of truncated ethernet headers, fix core.truncation test output
- Update commit hashes for external private test repo (Tim Wojtulewicz, Corelight)
* Merge ProtocolAnalyzerSet into Manager, remove AnalyzerSet base class (Tim Wojtulewicz, Corelight)
* Minor fixes for packet analyzer renaming. (Jan Grashoefer)
* Renamed LL-Analyzers to Packet Analyzers. (Jan Grashoefer)
* Initial implementation of Lower-Level analyzers (Peter Oettig)
3.3.0-dev.285 | 2020-09-22 16:01:51 -0700
* Improve fa_file/fa_metadata documentation (Duffy O'Craven, Corelight)
3.3.0-dev.284 | 2020-09-22 14:10:36 -0700
* Avoid race condition in control/id_value btest (Jon Siwek, Corelight)
3.3.0-dev.282 | 2020-09-22 11:08:56 -0700
* GH-927: Fix circumvention of evaluation order in 'when' conditions (Jon Siwek, Corelight)
3.3.0-dev.279 | 2020-09-21 15:22:45 -0700
* CI: Move to Fedora 32 (Christian Kreibich, Corelight)
Fedora 30 is EOL.
3.3.0-dev.275 | 2020-09-17 13:39:46 -0700
* GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet. (Jon Siwek, Corelight)
3.3.0-dev.273 | 2020-09-17 13:11:44 -0700
* GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided. (Jon Siwek, Corelight)
* Avoid run-time cost of find-filtered-trace.zeek for live traffic (Jon Siwek, Corelight)
* Remove connection_successful and successful_connection_remove events
Related to https://github.com/zeek/zeek/issues/1119 (Jon Siwek, Corelight)
3.3.0-dev.269 | 2020-09-17 11:42:38 -0700
* GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index. (Jon Siwek, Corelight)
* GH-1159: Fix vector-of-interval multiplication/division arithmetic
Those operations done between a vector-of-interval and a
vector-of-arithmetic-type previously threw a runtime expression error
due to an incorrect coercion being used internally. (Jon Siwek, Corelight)
3.3.0-dev.265 | 2020-09-17 11:24:42 -0700
* Avoid passing address of member in packed struct #1074
This appeases -Waddress-of-packed-member warnings in some
compiler/platform combinations via use of local variables. (Christian Kreibich, Corelight)
3.3.0-dev.262 | 2020-09-14 12:27:52 -0700
* Improve notice/suppression-cluster btest to avoid race potential (Jon Siwek, Corelight)
3.3.0-dev.260 | 2020-09-10 11:53:03 -0700
* Fix generate-docs Action to stage autogenerated script-reference files (Jon Siwek, Corelight)
3.3.0-dev.258 | 2020-09-09 10:46:37 -0700
* Improve Reporter weird-sampling-whitelist getters/setters (Jon Siwek, Corelight)
- getter methods return const-ref
- setter methods pass by value and std::move()
- ranged-for loops over the whitelists access by const-ref
* Fix memory leak caused by re-entering GTPv1 parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated GTPv1 frame maps to the same
Connection as the outer packet and also contains another GTPv1
encapsulation, the same GTPv1 analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
* Fix memory leak caused by re-entering AYIYA parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated AYIYA frame maps to the same
Connection as the outer packet and also contains another AYIYA
encapsulation, the same AYIYA analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25256
3.3.0-dev.254 | 2020-09-08 17:21:16 -0700
* GH-343: Change HTTP DPD signatures to trigger analyzer independent of peer state. (Robin Sommer, Corelight)
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).
3.3.0-dev.252 | 2020-09-08 17:04:19 -0700
* GH-623: Extend weird sampling with option to track selected weirds globally. (Robin Sommer, Corelight)
The new set "sampling_global_list" lists weirds to rate-limit
globally instead of per connection/flow.
3.3.0-dev.249 | 2020-09-04 18:30:19 -0700
* Exclude installing "zeek -> ." include dir symlink (Jon Siwek, Corelight)
The symlink only exists for use within the source-tree and isn't needed
for the install-tree.
3.3.0-dev.247 | 2020-09-04 17:24:28 -0700
* GH-165: Fix global initializations that indirectly use builtin types (Jon Siwek, Corelight)
* Improve how primary/top-level BIFs get initialized (Jon Siwek, Corelight)
* GH-1122: Allow initializing globals with calls to subdir BIFs (Jon Siwek, Corelight)
* GH-1122: Improve error for global record initialization exceptions (Jon Siwek, Corelight)
3.3.0-dev.241 | 2020-09-03 15:02:29 -0700
* Change a script comment to use "code-block" directive (Jon Siwek, Corelight)
3.3.0-dev.240 | 2020-09-02 20:51:13 -0700
* Improve CI benchmark script's error handling/messaging (Jon Siwek, Corelight)
3.3.0-dev.238 | 2020-09-02 11:39:25 -0700
* GH-955: Prohibit `fallthrough` in typecasting `switch` cases (Jon Siwek, Corelight)
3.3.0-dev.236 | 2020-09-01 22:33:05 -0700
* Fix/finalize the generate-docs Github Action (Jon Siwek, Corelight)
3.3.0-dev.230 | 2020-09-01 20:32:01 -0700
* Fix generate-docs action to not fail when there's no changes (Jon Siwek, Corelight)
3.3.0-dev.229 | 2020-09-01 19:49:36 -0700
* Update generate-docs action to use access token for submodule update (Jon Siwek, Corelight)
3.3.0-dev.228 | 2020-09-01 18:37:33 -0700
* Update generate-docs action to install Sphinx dependency (Jon Siwek, Corelight)
3.3.0-dev.227 | 2020-09-01 17:58:22 -0700
* Add missing configure step for doc-generation action (Jon Siwek, Corelight)
3.3.0-dev.226 | 2020-09-01 17:53:55 -0700
* Improve documentation for rfind_str() (Jon Siwek, Corelight)
* GH-1149: Add GitHub Action to automate generation of zeek-docs (Jon Siwek, Corelight)
3.3.0-dev.224 | 2020-09-01 11:57:44 -0700
* Reformat strings.bif Zeekygen comments to fix Sphinx warnings (Jon Siwek, Corelight)
3.3.0-dev.221 | 2020-08-31 17:31:21 -0700
* GH-174: Treat ambiguous attribute duplication as an error (Tim Wojtulewicz, Corelight)
For example, a &default=1 and a &default=2 attribute are not valid when
used together, but two duplicate &log attributes together are acceptable
although redundant.
3.3.0-dev.216 | 2020-08-31 14:57:57 -0700
* Simplify a broker btest (Jon Siwek, Corelight)
3.3.0-dev.215 | 2020-08-28 16:51:40 -0700
* GH-926: Improve type-checking for ternary conditional operator (Jon Siwek, Corelight)
* Unbreak build on power architectures by fixing small cmake issue (Johanna Amann, Corelight)
3.3.0-dev.211 | 2020-08-27 18:00:55 -0700
* Potentially improve intel/remove-item-cluster btest (Jon Siwek, Corelight)
3.3.0-dev.210 | 2020-08-27 15:34:14 -0700
* GH-158: Ensure packet-filter framework is loaded if -f option is passed (Tim Wojtulewicz, Corelight)
3.3.0-dev.208 | 2020-08-27 11:12:22 -0700
* GH-1145: Fix a umask issue for processing-status file (Vlad Grigorescu)
3.3.0-dev.206 | 2020-08-26 11:20:58 -0700
* Rename bro-bif.h to zeek-bif.h (Tim Wojtulewicz, Corelight)
* Fix uses of bro.pac to use zeek.pac (Tim Wojtulewicz, Corelight)
* Rename the bro-related binpac files to zeek (Tim Wojtulewicz, Corelight)
3.3.0-dev.202 | 2020-08-26 10:59:59 -0700
* GH-1145: Have mkdir in ensure_dir respect umask. (Vlad Grigorescu)
This also aligns with the mkdir BIF.
3.3.0-dev.200 | 2020-08-25 19:47:02 -0700
* Update plugin btests for namespace changes (Tim Wojtulewicz, Corelight)
* Remove namespace qualifications where unnecessary. (Tim Wojtulewicz, Corelight)
3.3.0-dev.196 | 2020-08-25 17:04:12 -0700
* Fix closing timestamp of rotated log files in supervised-cluster mode (Jon Siwek, Corelight)
3.3.0-dev.195 | 2020-08-25 14:32:47 -0700
* Improve an Intel framework btest (Jon Siwek, Corelight)
Relying on non-atomic `cp` operation may not cooperate well with the
Input framework file re-reading logic.
3.3.0-dev.194 | 2020-08-24 19:20:29 -0700
* Mark Continuation.h and PacketDumper.h as deprecated (Tim Wojtulewicz, Corelight)
* Remove vector iterator type aliases from SmithWaterman code, fix uses of them (Tim Wojtulewicz, Corelight)
* Fix warning with usage of fmt() (Tim Wojtulewicz, Corelight)
3.3.0-dev.190 | 2020-08-24 14:43:06 -0700
* Make set_processing_status() signal-safe.
Closes #574. (Robin Sommer, Corelight)
3.3.0-dev.188 | 2020-08-24 14:30:43 -0700
* GH-710: Check that &expire_func is a function and not hook/event (Jon Siwek, Corelight)
3.3.0-dev.186 | 2020-08-24 14:28:25 -0700
* Fix a case where PktSrc gets processed twice in one runloop iteration
For a non-live PktSrc, it had a special-case to be considered "ready"
every iteration, but additionally every 1 in 100 iterations (the polling
frequency), if there were no other "ready" IOSources, it would get added
to the "ready" set a 2nd time.
This commit completely excludes PktSrc from being processed during the
1/100 runloop iteration where a Poll() happens. That exclusion is
desirable for a second reason: if reading a pcap happens to do its final
Process() during that 1/100 polling-iteration and there's other
IOSources ready to process like EventMgr/TimerMgr, those sources have
logic to advance network-time to current-time if a PktSrc is no longer
open. So in such a case, PktSrc::Process() closes, then
EventMgr::Process() sees there's no longer an active PktSrc and advances
to current-time, then EventMgr::Drain() happens and may dispatch
various events that were previous scheduled, with those events now
unexpectedly seeing a network_time() returning current-time. (Jon Siwek, Corelight)
3.3.0-dev.184 | 2020-08-24 14:27:31 -0700
* GH-594: Improve table initialization type-check error messages (Jon Siwek, Corelight)
3.3.0-dev.182 | 2020-08-21 13:46:16 -0700
* Fix `zeek -NN` not printing canonical names for file analyzers (Tim Wojtulewicz, Corelight)
3.3.0-dev.180 | 2020-08-21 13:28:23 -0700
* Remove definition-less ThreadVal<->BrokerData conversion functions (Jon Siwek, Corelight)
Their presence/deprecation may cause "undefined reference" build
failures for some compilers (GCC 9.3.0 in my case).
3.3.0-dev.177 | 2020-08-21 08:36:48 -0700
* Add missing namespace to usage of get_exe_path in fuzzer (Tim Wojtulewicz, Corelight)
* Rename methods in RunState to remove 'net' from their names (Tim Wojtulewicz, Corelight)
* Move zeek::net namespace to zeek::run_state namespace.
This also moves all of the code from Net.{h,cc} to RunState.{h,cc} and marks Net.h as deprecated (Tim Wojtulewicz, Corelight)
* Move ScannedFile class and associated globals into ScannedFile.h and out of Net.h and scan.l (Tim Wojtulewicz, Corelight)
* Rename types in ZeekList.h to be consistent with the style guide (Tim Wojtulewicz, Corelight)
* Move NetVar from zeek to zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Remove PRI_PTR_COMPAT macros (Tim Wojtulewicz, Corelight)
* Fix indentation of namespaced aliases (Tim Wojtulewicz, Corelight)
* Move zeek-setup code into namespaces (Tim Wojtulewicz, Corelight)
* Move ZeekList types to zeek namespace (Tim Wojtulewicz, Corelight)
* Move __RegisterBif from zeek::detail::plugin to zeek::plugin::detail (Tim Wojtulewicz, Corelight)
* Remove unimplemented zeek_magic_path/bro_magic_path method (Tim Wojtulewicz, Corelight)
* Move all plugin classes into zeek::plugin::detail namespaces (Tim Wojtulewicz, Corelight)