Entrepreneur, developer, seasoned system and network engineer, aspiring hobbyist pilot, husband and father.

  • 4 posts
  • 0 comments
Joined 3 years ago
Cake day: June 5th, 2023

cross-posted from: https://sh.itjust.works/post/66524166

Good day SJW community,

I wanted to provide everyone with an update regarding the stability issues we experienced over the past 24 hours.

Beginning Wednesday, September 9, we observed a significant spike in incoming requests originating from a large number of IP addresses associated with networks operated by Baidu, Tencent and OneCable Network LLC.

The traffic patterns pointed to coordinated web scraping activity distributed across large IP ranges. This approach allowed the scrapers to circumvent the per-IP rate limits we have in place by spreading requests across many different addresses. The requests were primarily targeting comments and post content, suggesting the activity was focused on large-scale data collection, potentially for AI model-related purposes.

We identified this activity by analyzing traffic patterns at the Autonomous System Number (ASN) level and reviewing the average number of requests per minute generated by each network.

At the same time, we identified a separate source of abnormal traffic that appeared to be coming from another actor. These requests were performing expensive search queries that placed significant load on the server and affected its ability to respond to legitimate requests in a timely manner.

This traffic was using the following user agent:

NexusGlobalBot/2.0 (SupabaseEdgeRuntime/1.76.0)

After blocking this agent and its associated request patterns, we observed a significant reduction in server load, bringing utilization much closer to normal operating levels.

This is not the first time we have encountered this type of activity over the past several months and unfortunately, it is unlikely to be the last. We will continue improving our monitoring, rate limiting, and mitigation measures to reduce the impact of abusive or excessive automated traffic while maintaining reliable access for legitimate users.

Thank you for your continued support and trust.

To the members of SJW, our friends across the Fediverse, and the curious passersby, this update is for you.

Now that we are well into 2026, I wanted to look back at how 2025 unfolded and share some insight into SJW’s finances, infrastructure, traffic trends, and priorities for the year ahead.

2025 Finances

One of the biggest changes in 2025 was the introduction of member-supported funding.

In January of 2025, I opened donations to help cover the cost of operating SJW. The goal was to begin moving away from a model funded entirely by me and toward a community-supported model that can remain financially sustainable over the long term.

Throughout 2025, we received $3,239.41 in net donations and spent $1,396.14 on operational expenses.

During the year, we also experienced a few hardware failures, most notably a failed power supply and hard drive. Thanks to the redundancy built into our servers, neither failure resulted in a service disruption.

I was able to repurpose a compatible power supply from a decommissioned server at no cost. I also personally covered the replacement hard drive before prices went through the roof. While donation funds could have covered these expenses, I wanted to preserve as much of the balance as possible for a larger hardware upgrade.

All amounts below are in Canadian dollars.

Category Amount Additional information
Net donations received $3,239.41 January through December 2025
S3-compatible storage $69.62 Paid through July 2026
Email service $70.60 Paid through July 2026
Colocation $1,200.00 2025 infrastructure hosting
Domain name $55.92 Domain registration
Total expenses $1,396.14
Remaining balance $1,843.27 Reserved for operations and future infrastructure

This year, I hope we can raise enough to help purchase newer hardware. Unfortunately, as many of you are probably aware, the rapid growth of AI infrastructure has significantly increased the price and demand for many server components.

Traffic Trends

This is where things become particularly interesting and, at first glance, a little puzzling.

Based on our traffic logs and Cloudflare data, overall traffic approximately doubled during 2025, despite SJW adding only around 4,000 active accounts during the same period.

As of January 2026, when I first began preparing this update, Cloudflare was reporting:

  • More than 300 million pages served per month
  • Approximately 6 million unique visitors over a 30-day period

These are enormous numbers, but they almost certainly do not represent six million individual people.

I suspect that at least 50 to 60 percent of this activity, and possibly more, comes from bots, scrapers, automated crawlers, vulnerability scanners, archival services, and other automated systems. Some of this activity may also come from services gathering public information for traditional search engines and newer AI-powered search tools.

I recently implemented Anubis to help reduce abusive automated traffic. While it cannot prevent every bot or scraper from accessing the site, it makes large-scale automated collection more difficult and has helped us begin addressing the problem.

Hardware and Infrastructure

Given these traffic trends and some of the performance issues we have encountered, our current hardware is reaching the point where I need to seriously consider replacing or upgrading it.

I recently changed our Lemmy deployment to use horizontal scaling. In simpler terms, we now run multiple Lemmy service instances in parallel and distribute incoming traffic between them. This has helped improve the instance’s ability to handle periods of heavier activity.

However, software improvements can only take us so far. Newer hardware would give us more capacity, better performance, and additional room to build resilience into the platform.

I would also like to explore bringing our primary S3-compatible object storage in-house while maintaining an appropriate off-site backup strategy. This would reduce our reliance on third-party storage providers while giving us more control over privacy, performance, and redundancy.

2026 Priorities

Our main infrastructure priorities for 2026 are:

  • Upgrade or replace aging server hardware
  • Add further redundancy to critical services
  • Continue improving horizontal scaling
  • Reduce the impact of abusive automated traffic
  • Explore self-hosted object storage
  • Deepen our collaboration with Fedecan

Fedecan

As some of you may remember, we previously shared our intention to join Fedecan, an organization with which we share common values and a similar vision for the Canadian Fediverse.

That transition has now been finalized. SJW is associated with a registered Canadian nonprofit organization and can benefit from its banking, administrative, and nonprofit structure.

The initial banking and legal integration is complete, but our collaboration will continue to deepen as we work toward a larger shared vision and develop common guidelines.

Donations are now processed through Fedecan, but funds donated specifically to SJW are tracked separately and remain reserved for SJW’s operating costs, infrastructure improvements, and future service expansion.

Donations

As outlined above, one of our main goals this year is to upgrade our existing hardware and improve the resilience of the services supporting SJW.

I am still evaluating the available hardware options and will share more specific information once the upgrade plan and expected costs have been finalized.

If you have the means and would like to support SJW, donations of any size are greatly appreciated. Every contribution helps us maintain the instance, prepare for future growth, and reduce how much of the operating cost must be covered personally.

Regardless of how much we raise, I’m committed to keeping SJW running. I will continue looking for suitable replacement hardware and am prepared to personally contribute toward any shortfall required to support the next generation of our infrastructure.

If you’d like you can contribute through either of the following platforms:

Thank You

In closing, I want to thank our admins and mods, who continue to volunteer their time to help make SJW a safe and welcoming place.

I also want to thank our members who have chosen to make the Fediverse, and SJW in particular, their home. Whether you donate, contribute to discussions, report problems, help other members, or simply spend time here, you are part of what keeps this community alive.

Thank you for being you.

And if nobody has reminded you lately: you are great, you are loved, and most importantly, you are enough.

TheDude ✌️

cross-posted from: https://lemmy.ca/post/67952523

This is the first big step in the process to develop comprehensive guidelines for the Fedecan non-profit and the various platforms.

While this will mostly involve converting tacit knowledge and experience into an explicit written form, we expect that this process will inevitably bring up some points of disagreement on the best way to deal with different issues. We ask everyone participating in these discussions to please contribute constructively and in good faith. We encourage you to bring up any concerns or issues you have with the proposed structure and drafted guidelines, so that we can work together to fix them early on. However, in order to keep a productive environment for those discussions, we will be pruning any comment chains that devolve into personal attacks, slap fights, etc.

To help ground your feedback, consider these thought experiments when evaluating a potential guideline:

  • Veil of ignorance: Would it still feel fair to you if you switched places with someone else on the platform (ex. a new user, a moderator, an admin, a member of a vulnerable group, etc.)?
  • Equal Applicability: These rules will be enforced uniformly on everyone. A poorly written rule that helps “your side” today, can easily harm “your side” in the future as circumstances change.

The full guidelines, including governance details like the annual review cycle, can be found on the website: https://fedecan.ca/en/guidelines/

We plan to structure the guidelines as follows:

diagram of the tiers that are described below

Tier 1: Fedecan Rules

Internal Conduct

These rules apply to Fedecan team members (directors, officers, admins, and anyone with elevated access). They set expectations for how team members should act.

Universal Rules

These are the baseline rules that apply to every user on every Fedecan platform. They cover the things that are prohibited by Canadian law (threats, hate speech, CSAM, non-consensual intimate imagery) as well as universal policy rules (privacy/doxxing, harassment, fraud, content that could cause harm, labelling of sensitive content, etc.).

Tier 2: Platform-Specific

Each platform has different functionality and norms, so this is where we can be more specific with the rules. The threadiverse platforms (lemmy.ca, piefed.ca, sh.itjust.works) share similar rules around community creation, moderation, vote manipulation, and content labelling. Pixelfed has its own rules tailored to its platform.

Tier 3: Community-Level Rule Templates

These are optional templates that communities can link to, or use as a starting point for their own rules. The idea is that moderators can point users to a clearly written explanation of why a rule exists, and any relevant exceptions, rather than trying to fit everything into the sidebar. Additionally, if many communities are enforcing a particular rule in the same way, then users will have an easier time understanding and following them.

The post title standards template has been drafted, and we plan to add more as the need arises. I have a few others that are in the works, but they have some overlap with the other sections, so I thought that it would be better to let people discuss first.

cross-posted from: https://sh.itjust.works/post/35952239

Hey all,

Today, I’m writing to share our intention to join Fedecan, a Canadian not-for-profit organization committed to supporting the Fediverse. We believe this partnership will foster collaboration among like-minded individuals who share a common goal: to create a safe, welcoming space where people can connect freely.

Together we will help deliver a fast, reliable, and consistent experience for all and continue to be free from corporate influence and guided by the values of openness, community, and trust.

Who is Fedecan

Fedecan is a registered non-profit organization based in Canada, with the goal to provide a safe and welcoming space for people to connect with each other on the Fediverse. More information can be found here.

You may also know them by their Canadian lemmy instance lemmy.ca.

What does this mean for sh.itjust.works?

From the outside everything will stay much the same, especially with the admin team. Internally, we will collaborate on tasks related to non-profit compliance, policies, banking and common infrastructure elements such as backup/disaster recovery infrastructure. We will continue to operate separate equipment and instances in geographically isolated locations under different names. We will work towards aligning on safety and security practices in order to ensure that data is secure.

From an organizational structure, sh.itjust.works will fall under the Fedecan umbrella and will share common bylaws, policies, methodologies on best practices, security and legal considerations.

Where do donations go?

Donations to sh.itjust.works will continue to support our mission and objectives exclusively. However, users will also have the option to donate directly to Fedecan, which will allocate funds amongst its projects including sh.itjust.works. Operational costs related to running the non-profit will be shared among projects and these expenses will be transparently disclosed in annual reports published on the Fedecan website.

Option to withdrawal

sh.itjust.works will have the right to withdraw from the Fedecan umbrella should our mutual goals no longer be aligned. In such a case, a predefined provision and action plan will be in place to ensure a smooth transition back to independence.

Why Now?

As the Fediverse continues to grow, we believe it’s important to collaborate more closely with others who share our values. Joining Fedecan allows us to do just that, strengthening our operation through a non-profit while staying true to our mission.

Timeline / Next Steps

Over the coming weeks, we’ll begin the process of integrating with Fedecan on the organization side. You won’t notice many changes but we’ll keep you informed throughout the process.

Looking Ahead

We’re excited about this next chapter and the opportunities it brings. By joining forces, we’re reinforcing our commitment to the Fediverse and to the principles that brought us all together in the first place—openness, community, transparency, and trust.

Our core mission remains unchanged. We’re still independent in spirit and practice and we remain committed to being a space that’s not driven by profit, but by people.

I invite your questions or concerns on this thread or on our https://matrix.to/#/#sh.itjust.works:matrix.org)

Stay connected,

–The sh.itjust.works and Fedecan Team