Commit Graph

148 Commits

Author SHA1 Message Date
d11a2a5b8d Update nodelist import to exclude systems managed by us 2023-09-23 23:15:42 +10:00
ff04de52b5 Rework TIC processing to use Storage::disk(). Implemented handling of replaces and files that already exist 2023-09-23 22:40:17 +10:00
2ae24b9955 Move fido configuation items into fido namespace. If keeping packets move them into a date aligned subdir 2023-09-22 15:35:08 +10:00
612efda945 Process packet seenby/path/via lines when saving echomail/netmail 2023-09-20 21:37:18 +10:00
5e67be5ba1 Alert message for echomails with security violations is using wrong address 2023-09-19 11:29:08 +10:00
cc04ddd7b3 More work to ensure messages from a node are valid for the domain, and fix domain flatten to check for zone if one is supplied 2023-09-17 15:54:47 +10:00
708d9a9f67 More work to decommission rogue_path 2023-09-16 21:39:34 +10:00
a991db788e For AddressPoll, force using our file cache - seems sometimes the key doesnt release with memcached 2023-09-15 14:28:07 +10:00
096e37ef35 Removed packet cache, it wasnt used and not needed since we can queue large packets. Renamed to for consistent variable when using Packet::process() 2023-09-15 08:14:27 +10:00
2f878b6e64 Added filearea import 2023-09-14 23:42:25 +10:00
ec5c28a03e Added ignore_crc option to nodelist import 2023-09-14 23:06:02 +10:00
ff8c370d86 Move packet processing into a job 2023-09-13 20:58:22 +10:00
67c1be209d Logging changes when processing TIC files 2023-09-09 00:10:45 +10:00
6d199345ac Enable nodelist import test mode 2023-09-09 00:10:45 +10:00
6498e364fb More TIC processing fixes 2023-09-08 21:04:24 +10:00
63a9c06fb0 Fix for bug introduced in previous commit, where tic files where queued without the address prefix 2023-09-07 12:25:44 +10:00
3cd0deada0 Added subject so that it works with job:list 2023-09-05 22:46:49 +12:00
a5784de9a8 Put TIC processing on it's own queue 2023-08-15 12:25:54 +10:00
74a56d1e17 Now correctly showing packet receive time, and sending Notifications based on the packet address, not the node address 2023-08-06 13:43:10 +10:00
c59f8ee0ed Improvements to Echoarea Import, area names are now uppercase and descriptions use ucfirst() 2023-08-03 22:11:59 +10:00
f281575b15 Enable netmail forwarding for users that receive messages directed at clrghouz 2023-08-03 17:31:42 +10:00
fb2471a05b Fix our echomail sender is not the message originator 2023-07-29 17:48:09 +10:00
cd140971e2 Implemented echoarea/filearea security 2023-07-29 13:17:36 +10:00
f1ccca25ea Change public to show on echo/file areas to enable limiting whats shown in the activity graph 2023-07-28 13:22:39 +10:00
e97a106178 Add exception class when scheduled tasks fail and exception hasnt been caught 2023-07-27 21:09:57 +10:00
4e44e2e266 Added system polling 2023-07-26 19:44:07 +10:00
17fe7e910d Multiple enhancements to interactive messages, moved messages to Notifications, send netmail back when invalid packet password 2023-07-23 20:21:02 +10:00
02a3963a12 Fix netmail flags for netmails originating here 2023-07-20 23:12:26 +10:00
8d9bde9692 Fix messages to points, and fix netmail coming from systems which dont include an Origin line 2023-07-20 22:16:38 +10:00
c1ec4eff36 Optimised our sending and receiving of items 2023-07-18 23:00:13 +10:00
a3302b4012 Complete of logging received and sent packet names, and display them in the web ui for each node 2023-07-15 22:10:05 +10:00
61ab0614b6 Correctly storing netmail flags (intransit, local, recv) with senders ID and packet name 2023-07-15 10:46:19 +10:00
52055c5217 Update nodelist parsing now that we have mailer_system 2023-07-09 22:03:15 +10:00
faeec61aeb Fix for limiting AKAs, show parent PID that forked child, other minor fixes 2023-07-08 18:00:23 +10:00
ccdce6bb62 Move mailer details into a separate table 2023-07-07 23:59:04 +10:00
ad36da0bb1 Code cleanup, no functional changes 2023-07-04 11:33:28 +10:00
b1b86ca04a Implemented file sending during BINKP and EMSI sessions 2023-06-22 22:13:47 +10:00
58341db0fb Show netmails to admins, record netmail path in the DB 2023-06-18 23:33:26 +10:00
Deon George
04be81852a Save robot processed messages 2023-01-25 16:35:58 +11:00
Deon George
3dc18a3f1f Change default internal caching to off, and added test to make sure things are cached if on. More debugging with packet name for netmails 2023-01-25 16:26:10 +11:00
Deon George
63e3397aee Add packet name for incoming netmails, dont send back messages to sender for areafix messages 2023-01-24 23:16:11 +11:00
Deon George
c1ab14f2e6 Fix echoarea display with the oldest message 2023-01-01 14:33:52 +11:00
Deon George
d825f71a15 Skip messages that originate from a node in a zone we dont know. 2023-01-01 13:53:51 +11:00
Deon George
06f814395c More debugging to know more about failed echomail parsing because of no zone. 2023-01-01 13:35:56 +11:00
Deon George
91fc61e170 Fix nodelist import, when parsing regions 2022-12-19 23:34:42 +11:00
Deon George
702a4e4f37 Attempt to catch DB errors and abort the transaction, so that the queue continues to run. 2022-11-13 15:17:22 +11:00
Deon George
e7ebaa85cb Fix nodelist crc's that have 8 chars 2022-11-13 15:17:22 +11:00
Deon George
b158bb5a7d Add domain to TIC processing, for nodelists that dont have a domain 2022-11-13 15:17:22 +11:00
Deon George
9f0fa242e6 When deleting an Address change active to FALSE 2022-11-13 15:17:22 +11:00
Deon George
6284016400 Auto create FTN address for intransit netmail when the destination is not known. 2022-11-06 14:40:03 +11:00
Deon George
531654724a Enable specifying a domain for nodelist import, and throwing exceptions when errors occur 2022-11-05 15:17:11 +11:00
Deon George
9721db185b Ensure Echomail is initialised correctly 2022-11-05 10:17:14 +11:00
Deon George
b45e2c6dd8 Support processing nodelists from file repository and assume nodelists are zip fiels 2022-11-05 10:17:14 +11:00
Deon George
2790381a30 Determine if an inbound file is a nodelist 2022-11-05 10:17:14 +11:00
Deon George
34f289953d Implementation fixes for TIC processing 2022-11-05 10:17:14 +11:00
Deon George
7571a2cd7d Enable moving inbound files to s3 and logging in DB 2022-11-04 17:30:18 +11:00
Deon George
029a8a9d73 Added file areas and TIC processing 2022-11-02 22:01:45 +11:00
Deon George
da85e85774 Cast empty Collections to null, Cast strings to zstd compressed strings, add msg_src to echomails processed, fix duplicate seenbys 2022-11-02 21:20:43 +11:00
Deon George
daba4a78d3 Fix echoarea name is converted to UC, add netmail VIA lines for outgoing netmail 2022-02-17 23:55:37 +11:00
Deon George
f142284a2b Understand some gated messages, that are gated from a different zone 2022-02-12 11:35:24 +11:00
Deon George
d660d5a6df Move DomainController::NODE* to Address::NODE*, make role mandatory in the database, change logic so that mail generated by the host comes from a node address. 2022-02-03 13:23:58 +11:00
Deon George
efa7195633 Add DBID back to messages, add path/seen-by to generated messages, other minor cosmetic fixes 2022-01-22 23:08:46 +11:00
Deon George
6f1d47a6ab Fixes to message processing, now that we are using cockroachdb 2022-01-15 17:11:51 +11:00
Deon George
424d6ef39d Switchout DB to CockroachDB 2022-01-15 14:58:41 +11:00
Deon George
871430edf8 Replace path from duplicate messages when the existing path is blank 2021-11-25 21:22:36 +11:00
Deon George
ebd1cf8732 Enable duplicates to update seen-by's before being discarded 2021-11-24 22:34:40 +11:00
Deon George
02c23f047c Fix for recording seen-by, add md5 to message body, fix incorrect log of dispatching messages 2021-11-21 16:53:56 +11:00
Deon George
53a96a8dbd Change dupe checking as per FTS-0009.001 2021-10-18 23:10:15 +11:00
Deon George
b246f0051f Fix polling, presenting our address. Fix logging address. 2021-10-18 23:08:08 +11:00
Deon George
83e06908b0 Multiple exception bug fixes identified during testing 2021-10-07 23:32:37 +11:00
Deon George
82e3283d6d Move address polling to a Job, better catch Socket connection refused errors 2021-09-20 20:39:03 +10:00
Deon George
2cd6db2060 Dont launch bot processing for rescanned messages 2021-09-13 23:02:39 +10:00
Deon George
5e8b590f17 Move process functions to Netmail/Echomail folders, optimize Netmail creation when processing messages 2021-09-12 22:14:04 +10:00
Deon George
9462e66a1c Some logging fixes 2021-09-12 00:07:02 +10:00
Deon George
fcdde10512 Auto create nodes that are discovered by message packets 2021-09-11 23:32:10 +10:00
Deon George
04e8a899d4 Also ignore nodelist lines that are blank 2021-09-11 19:26:11 +10:00
Deon George
ff2b246d31 For nodelist import, find based on mailer_address first 2021-09-11 18:00:39 +10:00
Deon George
72bc9790d1 Add option to not process bots during import 2021-09-11 00:40:52 +10:00
Deon George
dbbfe46cb9 Echomail export 2021-09-08 22:07:19 +10:00
Deon George
01d54794a8 Rename PacketProcess, fix Packet::open now using Zone not Domain 2021-08-30 00:24:40 +10:00
Deon George
9fb6d191d0 Change the way we figure out zones in packets, some packet testing, fix Echomail import 2021-08-29 23:58:12 +10:00
Deon George
12f9ee1960 Changed nodelist import processing since we have a DB contraint on mailer address and port 2021-08-28 13:36:42 +10:00
Deon George
6b8704b1cf Added Echoarea NA file import 2021-08-28 00:10:42 +10:00
Deon George
d86d90cdeb Leverage Redis and queue to handle large packets 2021-08-24 23:42:03 +10:00
Deon George
e0269c9718 Fix processing packets with echomail messages from points 2021-08-22 16:45:02 +10:00
Deon George
8c992ad0f9 Enhance nodelist import 2021-08-22 00:20:22 +10:00
Deon George
9f762a642c Added SEEN-BY/PATH processing, dont show networks that are not public, minor other formatting 2021-08-19 23:35:48 +10:00
Deon George
bcf3297fdb Check for duplicate echomails, validate system password 2021-08-19 00:20:34 +10:00
Deon George
b9abd17139 Dont process messages from robots 2021-08-15 00:21:04 +10:00
Deon George
eb0405f019 Added echoareas and fileareas 2021-08-11 23:45:49 +10:00
Deon George
b179b1b3e9 Mail routing parent/children, domain name validation, nodelist import changes and other fixes 2021-08-08 01:14:11 +10:00
Deon George
d937547599 Some message optimisation, added Echomail processing 2021-07-31 00:44:23 +10:00
Deon George
ee30ef92c3 New algorithm for calculating packet name, EMSI/BINKP inbound processing tested, Netmail rejection and intransit processing 2021-07-24 00:53:35 +10:00
Deon George
9dcfe6b17d BINKP responding to incoming netmail pings 2021-07-18 22:11:34 +10:00
Deon George
a0d3c8d8ab Start of processing packets - implemented PING Responce to Netmail 2021-07-16 16:39:02 +10:00
Deon George
980f57e808 Renamed import nodelist job, dont display address id's to unauthed users, removed debugging from importnodelist. 2021-06-26 10:55:02 +10:00
Deon George
aa06296963 Fixes for nodelist importing 2021-06-26 10:35:02 +10:00
Deon George
64215ebcea Import nodelists 2021-06-25 22:21:19 +10:00