Crowdwide

Crowdwide is a social discovery platform built around communities and curiosity instead of follower counts. Post, publish articles, run polls, join open or private communities, chat one-to-one or in groups, and get a "For you" feed that deliberately makes room for new voices and new communities.

It is an Express MVC app with EJS views and MongoDB, using session-based authentication, optional TOTP two-factor, and email through the Gmail API.

Live: https://www.crowdwide.run.place

Features

Posting and content

• Posts, articles and polls, with drafts and scheduled publishing.
• Word limits set by admins (defaults: 60 words for a post, 300 for an article).
• Up to two media files per post (images, video, audio) with alt text, a caption and a transcript for each file; images keep their aspect ratio, and video seeks smoothly (HTTP range support).
• Rich formatting in the body: **bold**, *italic*, # headings, - lists, paragraph breaks.
• Hashtags and @mentions with autocomplete, content warnings with click-to-reveal, quote posts and linked post replies.
• Link previews for the first link in a post (title, description, image).
• Edit and delete controls on your own posts and comments.

Feed and discovery

• A "For you" feed that blends people you follow, communities you have joined, and a deliberate share (35-40%) of posts from outside your network, so new accounts and communities get real reach.
• The feed learns from what you do: likes, reactions, comments, saves, posts you open, hashtags you post, and searches.
• Feed tabs for your community, posts (new and viral) and articles (new and viral).
• /explore for communities, viral posts, popular people and new members; /people for people to follow (shared interests, mutual network, trending creators, new joiners).
• Search across people, communities, hashtags and posts, with recent and popular searches.
• Trending hashtags, RSS feeds for the site, profiles and communities.

Communities

• Open or private communities with banners, avatars, hashtags, guidelines, member lists and owner/moderator roles.
• Join requests for private communities, banned-word lists, optional review-before-posting, pinned posts and per-community feeds.
• Private communities are private everywhere: their posts, members and RSS feed are visible only to members across the community page, feeds, search, profiles, the public API and every RSS feed.

Profiles

• Banner, avatar, bio, links, hashtags and privacy controls.
• Followers and following, saved posts, likes, comments and activity.
• Activity stats on every profile, plus a private Stats tab with posts per month, post mix, average likes and top posts.

Messaging and notifications

• Direct messages and group chat (up to 50 members) with GIFs, sharing of posts and articles into chats, group admin tools (add/remove members, rename, group picture, invite links).
• In-app notifications with grouping, per-type preferences, and opt-in browser push notifications per device.
• Security emails for new sign-ins, password changes and 2FA changes.

Accounts and security

• Email verification, password reset, five-attempt login lockouts, 75-day session/device records with remote log-out, and TOTP two-factor with recovery codes.
• Registration bot protection: an arithmetic challenge (or Cloudflare Turnstile when configured), an invisible honeypot and a minimum-fill-time check.
• CSRF protection, Helmet security headers, rate limits, upload validation and optional ClamAV virus scanning.
• Data export and account deletion from Settings.

Moderation and safety

• Reports with optional evidence screenshots, moderator review with recommendations, and admin approval.
• Warnings, temporary posting restrictions and suspensions, each with reasons and durations; suspensions cannot be shortened by unrelated login lockouts.
• Appeals for suspensions (public form), restrictions and warnings (Settings > Moderation); admins approve or deny, and approval lifts the action automatically.
• Spam detection, an audit log, and a per-user account history timeline.
• An admin console at /admin (users, communities, posts, reports, appeals, audit log, site settings including word limits, suspension defaults and review thresholds) and a moderator console at /moderator.

Public pages and API

• /about (with live community charts), /about/developer, /privacy, /terms, /community-guidelines, /accessibility, /contact, /help, /guide and /docs.
• A read-only public API at /api/v1/posts plus RSS feeds - see API.md.
• Every page has a title, description, canonical URL, Open Graph and Twitter metadata; robots.txt and sitemap.xml are generated by the app.

Operations

• Structured JSON logging, error alert emails, database-aware /health, backup and restore verification scripts, CI, staging mode, and a free-tier keep-alive ping. See DEPLOYMENT.md.

Quick start

Requirements

• Node.js 22 or newer
• MongoDB running locally or a MongoDB Atlas connection string

Install and run

npm install
cp .env.example .env

Set MONGODB_URI and a long, unique SESSION_SECRET in .env. Keep APP_URL=http://localhost:3000 for local development. Then start the app:

npm run dev

Open http://localhost:3000. To run the test suite, use npm test. For a production-style start, use npm start.

Email delivery is optional during local development. Without mail credentials, verification codes and security email previews are written to the server log. See Email delivery to configure Gmail OAuth2 or SMTP.

Without mail configured, verification codes and security emails print to the server log instead of sending - fine for local development, not for anything real users will hit.

For Render (or a similar platform), set NODE_ENV=production and TRUST_PROXY_HOPS=1. Crowdwide trusts one reverse-proxy hop so Express and express-rate-limit can safely read the platform's X-Forwarded-For header. Don't set the Express trust-proxy value to true unless you fully control the deployment topology - see DEPLOYMENT.md for the rest of what a real deployment needs (CI, backups, virus scanning, alerting).

Environment variables

Every variable Crowdwide reads is listed in .env.example with a one-line comment. Grouped by what breaks if you skip them:

Required to run at all

• MONGODB_URI - a MongoDB connection string. MongoDB Atlas has a free tier; or run mongod locally for development.
• SESSION_SECRET - any long random string, used to sign session cookies. Generate one with node -e "console.log(require('crypto').randomBytes(32).toString('hex'))".
• APP_URL - the public HTTPS URL of the deployment (or http://localhost:3000 locally). Used for canonical links, Open Graph tags, the sitemap, RSS feed links, and API response URLs.

Required for real email delivery (without these, codes/alerts just log to the console - see Email via Google Cloud / Gmail API below for the full walkthrough)

• MAIL_PROVIDER=gmail
• GMAIL_USER - the Gmail address that will send mail.
• GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET - from a Google Cloud OAuth 2.0 Client ID.
• GOOGLE_REFRESH_TOKEN - obtained via a one-time OAuth authorization (walkthrough below).
• MAIL_FROM - same address as GMAIL_USER.

Optional - each feature degrades gracefully to "off" without it

• VAPID_PUBLIC_KEY, VAPID_PRIVATE_KEY, VAPID_SUBJECT - browser push notifications. Generate a pair with npx web-push generate-vapid-keys.
• ADMIN_ALERT_EMAIL - unhandled server errors email this address (rate-limited). Any address you actually check.
• CLAMAV_HOST/CLAMAV_PORT or CLAMAV_SOCKET, plus optional CLAMAV_REQUIRED/CLAMAV_FAIL_OPEN - upload virus scanning against a ClamAV daemon. See DEPLOYMENT.md.
• Translation works with no setup (free built-in providers; TRANSLATE_FREE_FALLBACK=off disables). Optional: LIBRETRANSLATE_URL and optional LIBRETRANSLATE_API_KEY - post/comment translation via a self-hosted LibreTranslate service. An optional GOOGLE_TRANSLATE_API_KEY enables paid fallback translation.
• GCS_PROJECT_ID, GCS_BUCKET, GOOGLE_APPLICATION_CREDENTIALS, MEDIA_CDN_URL - direct-to-cloud media uploads via Google Cloud Storage, instead of (or alongside) MongoDB GridFS. See Extended media storage below.
• MONGO_DB_URL_0, MONGO_DB_URL_1, ... - additional MongoDB clusters to spread media storage across. See Extended media storage below.
• TRUST_PROXY_HOPS - see Run locally above.
• GIPHY_API_KEY (and optional GIF_RATING) - GIF search in chat, group chat and comments. Without a key the GIF buttons are hidden.
• TURNSTILE_SITE_KEY, TURNSTILE_SECRET_KEY - use Cloudflare Turnstile for the sign-up CAPTCHA instead of the built-in arithmetic question.
• APP_ENV=staging, STAGING_MAIL_ALLOWLIST - staging mode for a second deployment. See DEPLOYMENT.md.
• KEEP_ALIVE_* - free-tier keep-alive ping (on by default in production).
• FEED_DISCOVERY_RATIO, POPULAR_SEARCH_MIN_USERS, STATS_MID_COMMUNITY_MIN, STATS_LARGE_COMMUNITY_MIN - feed and stats tuning.
• BACKUP_DIR, BACKUP_RETENTION_DAYS - see DEPLOYMENT.md.

Nothing in .env.example needs a paid service to get a working local setup - MongoDB Atlas, Google Cloud, and web-push key generation are all free. Mail, push, GIFs, alerting, virus scanning, and extended storage are all independently optional; the app runs and every core feature works without any of them configured.

Email via Google Cloud / Gmail API

Crowdwide sends mail through the Gmail API via OAuth2 against a Google Cloud project - not a raw Gmail address/app-password combination, and not a third-party transactional email service. If you're setting this up for the first time:

1. In Google Cloud Console, create or select a project.
2. APIs & Services > Library - enable the Gmail API for that project.
3. APIs & Services > OAuth consent screen - configure it (External is fine for a small deployment). While the app is in "Testing" mode, add the Gmail address that will send mail as a test user - Google won't let an unverified app send as an account that isn't listed here.
4. APIs & Services > Credentials > Create Credentials > OAuth client ID - type "Web application" (or "Desktop app" if you're doing the OAuth flow locally). Add https://developers.google.com/oauthplayground as an authorized redirect URI - that's the easiest way to complete step 5 without writing a callback handler yourself.
5. Get a refresh token - the fiddly one-time step:
   • Go to Google OAuth Playground.
   • Click the gear icon (top right) -> check "Use your own OAuth credentials" -> paste in the Client ID and Client Secret from step 4.
   • In the left panel, find Gmail API v1 and select the https://mail.google.com/ scope.
   • Click Authorize APIs, sign in with the Gmail address from step 3, and accept.
   • Click Exchange authorization code for tokens. The Refresh token field is what you need - it doesn't expire until you revoke it.
6. Set in .env:

   MAIL_PROVIDER=gmail
   GMAIL_USER=your-sending-address@gmail.com
   GOOGLE_CLIENT_ID=...           # from step 4
   GOOGLE_CLIENT_SECRET=...       # from step 4
   GOOGLE_REFRESH_TOKEN=...       # from step 5
   MAIL_FROM=your-sending-address@gmail.com
   

No Gmail password and no "less secure app access" toggle involved - that's the whole point of doing it this way. Keep the client secret and refresh token out of source control; in production, put them in your host's secret manager (Render's environment groups, Google Secret Manager, etc.), not in a committed .env file.

If mail is unconfigured, verification codes, password reset links, and security alerts just print to the server log ([Crowdwide mail preview] ...) instead of sending - useful for local development, not something to leave on in production, since real users won't see their verification code anywhere.

Structure

• server.js - process entry point (env loading, DB connect, listen)
• src/app.js - the Express app itself, separated out so tests can import it without binding a port
• src/config - database configuration
• src/controllers - request handlers
• src/middleware - auth guards, upload handling, security (CSRF, rate limits, virus scanning hook)
• src/models - MongoDB/Mongoose schemas
• src/routes - web and auth route definitions
• src/services - mail, push notifications, link previews, GIFs, feed ranking, public stats, structured logging, error alerting, virus scanning, publication scheduling, keep-alive, environment/staging
• src/utils - small shared helpers (hashtags, spam detection, posting restrictions, private-community visibility)
• src/views - EJS pages and partials
• public - static assets, styles, service worker, and browser scripts
• test - DB-free unit tests (npm test) and end-to-end tests requiring MongoDB (npm run test:e2e) - see Testing below
• scripts - operational scripts (backup, restore verification)

Public pages and SEO

The site includes /about, /about/developer, /privacy, /terms, /community-guidelines, /accessibility, and /contact. Every page gets a title, description, canonical URL, Open Graph and Twitter metadata, favicon, and responsive layout. The homepage also emits WebSite JSON-LD, while /robots.txt and /sitemap.xml are generated by the application.

Homepage member, post, and community totals are queried from MongoDB on each request. Top communities are ordered by member count. If MongoDB is empty or unavailable, the homepage shows an honest starter state rather than invented activity.

Dashboard feed

The authenticated dashboard has two feed modes. The normal feed builds a 20-post window with an intended 40% new voices and newer community posts, 55% posts from the largest communities, and 5% viral content ranked by likes. The personalized feed only returns posts and articles attached to communities the signed-in user has joined. The dashboard also supports publishing posts or articles, assigning them to a community, creating communities, joining communities, and following verified users.

Community creators are stored as owners and can edit community details, view recent posts, and remove members from the owner dashboard. Profiles support a 280-character bio, a 1.5MB image avatar, and public/followers-only privacy. Posts are limited to 4,000 characters; articles are limited to 50,000 characters. Post media is limited to images under 1.5MB, videos under 4MB, and audio under 2MB.

Community discovery and moderation

/explore is a discovery hub, not just a community list: alongside the searchable, category- and hashtag-filterable community grid, it surfaces viral posts, popular people, and newly joined members. Communities have their own hashtags, an optional banner (under 2MB) and profile picture (under 1.5MB), and public guidelines shown on their /communities/:slug page along with a public member list (owner/moderator/member roles visible to everyone).

Communities can be open or private. Open communities accept members immediately; private communities create a join request that the owner or a moderator must approve - this is the only difference in the join flow. Owners can edit details, hashtags, and guidelines; maintain a banned-word list; toggle "review posts before they appear" (new posts to that community land in a pending queue moderators/owners approve or reject from the manage page); add moderators by username or email (adding them as a member automatically if needed); and remove members. Posts addressed to a community require membership and are checked against that community's banned words.

Social interactions and security

Likes, bookmarks, nested/threaded comments (with reply-to-reply), share counters, hashtags, and notifications are stored in MongoDB. Posts and communities support #hashtags, searchable from /search?q=%23tag or by typing #tag in the search box. Users can follow/unfollow and block/unblock each other; blocking removes any follow relationship and filters the blocked account's posts, comments, and notifications out of your feed and search results. A profile's Followers and Following counts open a dedicated page (/u/:id/followers, /u/:id/following) with suggestions. Helmet security headers, CSRF tokens, authentication rate limits, interaction rate limits, five-attempt login lockouts, 75-day device records, and new-device email alerts are enabled. Users can configure TOTP two-factor authentication at /settings/security/2fa, which issues eight one-time recovery codes (downloadable as a .txt file) for signing in if the authenticator app is unavailable; codes can be regenerated at any time with a password confirmation.

Extended media storage (multi-cluster MongoDB)

Uploaded post media, generated image thumbnails, and profile pictures are stored in MongoDB GridFS and served through /media/:cluster/:id. By default everything lives on the primary MONGODB_URI cluster ("cluster 0") - no extra setup required.

To scale storage horizontally, add any number of extra MongoDB clusters as MONGO_DB_URL_0, MONGO_DB_URL_1, MONGO_DB_URL_2, ... in .env. Every configured cluster gets its own GridFS bucket, and uploads distribute across all of them round-robin (src/services/storageCluster.js). GET /media/status (authenticated) reports which clusters are currently connected.

Set GCS_PROJECT_ID, GCS_BUCKET, GOOGLE_APPLICATION_CREDENTIALS, and optionally MEDIA_CDN_URL to enable /media/signed-upload for direct-to-cloud uploads via Google Cloud Storage, as an alternative or addition to GridFS. The endpoint returns a V4 signed upload URL that expires after 15 minutes.

If CLAMAV_HOST/CLAMAV_PORT or CLAMAV_SOCKET is set, every upload (post media, profile pictures, community banners, report evidence) is scanned through a ClamAV daemon before it's accepted - see DEPLOYMENT.md for setup. Scanner outages block uploads by default; CLAMAV_FAIL_OPEN=true opts into allowing uploads without a completed scan.

Testing

• npm test - the DB-free unit suite (permission logic, upload validation, mailer/alerting resilience, link previews, spam detection, CAPTCHA, the private-community access gate, and more). No setup beyond npm install.
• npm run test:e2e - full route-level and end-to-end tests (signup through posting, admin permissions, the suspension-bypass regression, the full appeal lifecycle, the private-community privacy regression) against a real MongoDB - either a MONGODB_URI you provide, or an auto-downloaded in-memory instance via mongodb-memory-server.
• CI runs both automatically on every push/PR - see .github/workflows/ci.yml and DEPLOYMENT.md.

Profiles, following, and search

Every author name and avatar across the app links to a public profile at /u/:id, showing a banner image, bio, up to four custom links, join date, post count, followers/following counts, and that user's posts. Viewing your own profile shows an additional "Recent activity" feed (your latest likes and comments) and a "People to follow" panel, Twitter-style. Users can follow, unfollow, or block each other; following/blocking are reflected immediately (AJAX). The home feed mixes posts from people you follow, a second-degree "extended network" (people your follows follow, and people who follow your followers), new voices and communities, and larger communities - so growing accounts and new communities are not permanently buried under popularity, and the feed lazy-loads further posts as you scroll. /search?q= searches people, communities, hashtags, and post text in one place and is wired to the header search box and the mobile menu on every page.

Auth flow

New accounts receive a six-digit verification code and cannot access /dashboard until verified. An unverified login generates a fresh code and redirects to verification. Password reset tokens are stored with an expiry on the user document.

Email delivery

Crowdwide supports Gmail OAuth2 and generic SMTP. Configure either provider in .env; never commit OAuth tokens, passwords, or service-account credentials. The Gmail OAuth2 setup below is the recommended option for Gmail.

Gmail OAuth2 setup

1. In Google Cloud Console, create or select a project.
2. Enable the Gmail API for the project.
3. Configure the OAuth consent screen. Add the Gmail account that will send Crowdwide mail as a test user while the app is in testing mode.
4. Create an OAuth 2.0 Client ID for a Web/Desktop application and obtain a refresh token with the Gmail scope https://mail.google.com/.
5. Set MAIL_PROVIDER=gmail, GMAIL_USER, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET, and GOOGLE_REFRESH_TOKEN in .env.
6. Set MAIL_FROM to the same verified sender address as GMAIL_USER.

The mailer uses Gmail OAuth2 through Nodemailer, so no Gmail password or less-secure-app access is required. MAIL_FROM should use the verified sender address configured as GMAIL_USER. For generic SMTP, set MAIL_HOST, MAIL_PORT, MAIL_USER, MAIL_PASS, and MAIL_FROM; do not set MAIL_PROVIDER=gmail. In production, store credentials in your deployment platform's secret manager or Google Secret Manager.

Routes

• / landing page
• /auth/register, /auth/login, /auth/verify, /auth/forgot-password, /auth/reset?token=..., /auth/appeal (public suspension-appeal form), /auth/2fa
• /dashboard (/dashboard/feed/more powers lazy-loaded scrolling)
• /explore discovery hub
• /communities/:slug community detail (members-only content if private); /communities/:id/manage owner/moderator controls; /communities/:slug/rss.xml
• /u/:id public profile; /u/:id/followers, /u/:id/following; /u/:id/rss.xml
• /search?q=
• /posts/:id post detail with nested threaded comments
• /messages, /messages/:id; /groups, /groups/:id
• /notifications
• /settings/:section - profile, security (including /settings/security/2fa, /settings/security/2fa/recovery-codes), privacy, notifications, moderation (only shown if relevant), history, account
• /admin (admin console), /moderator (moderator review)
• /rss.xml; /api/v1/posts (see API.md)
• /people people to follow: common interests, mutual network, trending creators, new joiners
• /guide tips for growing a profile, a community, and staying secure
• /docs public API reference; /help frequently asked questions
• /about, /about/developer, /privacy, /terms, /community-guidelines, /accessibility, /contact

Operations

CI, staging, backups/restore verification, upload virus scanning, and error alerting are covered in DEPLOYMENT.md.

Public API

/api/v1/posts and the RSS feeds are documented in API.md.

Roadmap

todo.txt lists what has been built and what is planned next.

Contact

For product feedback, bug reports, accessibility concerns, or security questions, email developerpuneet2010@gmail.com. Do not include passwords, recovery codes, OAuth tokens, or other secrets in support messages.