profile
viewpoint

babygoat/Animation 0

Modal with key interaction animation using React

babygoat/click-to-deploy 0

Source for Google Click to Deploy solutions listed on Google Cloud Marketplace.

babygoat/concurrency-in-go-src 0

Full sourcecode for the book, "Concurrency in Go" published by O'Reilly.

babygoat/EcStore-net 0

extract and override practice

babygoat/environment-setup 0

NPM package to set up development environment while developing `twreporter` related projects.

babygoat/gcsfuse 0

A user-space file system for interacting with Google Cloud Storage

fork babygoat/keystone

node.js cms and web app framework

http://www.keystonejs.com

fork in 7 hours

startedtwreporter/orangutan-monorepo

started time in 2 days

push eventtwreporter/go-api

nickhsine

commit sha 84c1831e0dc520f86b10b462a399fb3c16b77f68

chore: update CHANGELOG.md. remove Merge Pull commits

view details

nick

commit sha ac43df3781f8e533a36758ccf41d5d496e423fc8

Merge pull request #385 from nickhsine/master chore: update CHANGELOG.md. remove Merge Pull commits

view details

Ching-Yang, Tseng

commit sha c5729ab46903dffb14d4e2f189465438971dbde4

doc: adds v2 endpoint of getting a post This patch adds the api document of describing the v2 endpoint (/v2/posts/:slug).

view details

Ching-Yang, Tseng

commit sha 0bb1f95d8d9420240a21a81ee7eeb403e2b6fad8

doc: add v2 endpoint for listing posts This patch adds the v2 endpoint document for listing posts according to following options Filter * category_id * tag_id * id Pagination * offset * limit Sort * updated_at * published_date

view details

Ching-Yang, Tseng

commit sha 5f876e9fa075447fc628e52a7e24ffb729db93a3

doc: add the v2 endpoint to get a single topic This patc adds the endopint specification of the v2 single topic infomration retrieval. Also, move the common asset data structure into asset.apib.

view details

Ching-Yang, Tseng

commit sha c74559fa7463cac0a2a48123bdd7a447f86aa1a4

doc: add the v2 endpoint of listing topic This patch adds the v2 endpoint of listing topic with respect ot following options. Pagination * offset * limit Sort * sort

view details

Ching-Yang, Tseng

commit sha 9ffbfa32bcef9895f2cc667ada8a2638d15e5150

doc: generate result document This patch generates the final html file for api document

view details

Ching-Yang, Tseng

commit sha f6aba810246cee78d04f9e06b7feb91e098c54f5

doc: add /v2/index_page endpoint This patch adds the endpoint document for /v2/index_page.

view details

Ching-Yang, Tseng

commit sha dcd8975adba21eaad4dba69093f9395d595cb79f

doc: adjust /v2/posts endpoints This patch adjusts /v2/posts & /v2/posts/:slug endpoints response according to the payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1523711569)

view details

Ching-Yang, Tseng

commit sha 1590853f052c397777ca3057c9dd602047b27951

doc: adjust /v2/topics endpoints This patch adjust /v2/topics and /v2/topics/:slug endponts with respect to the response payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1659711178)

view details

Ching-Yang, Tseng

commit sha 93af75242aed38975050cd1fa928cbaf00d589e0

doc: describe different response w.r.t parameter This patch specifies the full=true on the request description to differentiate the same response code with different response body with respect to the query parameter setup.

view details

Ching-Yang, Tseng

commit sha 7c7410760cc63ca96bc2871b51d1a6188b74fd4e

doc: add client side error for invalid slug This patch adds the client side error (404) during the single post/topic retrieval when the slug does not exist within the posts or topics collection.

view details

Ching-Yang, Tseng

commit sha e9f7f769345a0d89d457d07dfc9e4bb3ba8ed63c

doc: give relateds sample value instead of default This patch gives the relateds field sample value instead of using aglio string default value `hello, world` to approach the object string data stype.

view details

Ching-Yang, Tseng

commit sha 7f177d2606ec80c2aec7d19e973ddce84d9be589

doc: fix nested array field schema type missing This patch fixs schema of nested array fields which should be both required and fixed-type(inherit the object type).

view details

Ching-Yang, Tseng

commit sha 8d2ba673bfd88997a04d19168d2121e300382892

doc: singularize topics field This patch renames the `topics` field of a single post to `topic`. Also, add the topic field as the required field in the full post response.

view details

Ching-Yang, Tseng

commit sha aeb4a20d8c6d7132b47a1dd9df2f5b5883743507

doc: add missing required fields This patch adds the updated_at/published_date fields to required fields so that the schema of the topic/post fit the requirement. Also, rename the query parameter naming of updated_date back to updated_at.

view details

Ching-Yang, Tseng

commit sha 2637e1b114012f3c89d492da0d15402f6130d563

doc: return meta instead of empty content during post/topic list This patch replaces status no content with status ok with meta information(e.g., pagination) and empty record array.

view details

Ching-Yang, Tseng

commit sha f19d8a5ea6cc0fb71366f2731fb0c6e5ba1f49a4

api/news: replace field writters with writers This patch replaces the field `writters` with `writers` in full post data.

view details

Ching-Yang, Tseng

commit sha 6533e85511d6f7efca67b59a22841e13b1bff332

doc: fix `full` field type in Post group This patch updates the `full` field type from string to boolean. Also, mark the full field as required in the MetaOfPost model.

view details

Ching-Yang, Tseng

commit sha c14a33993395ac316eeb6f485f5b0e3b670d6d9d

doc: fix `full` field type in Topic This patch updates the `full` field type from string to boolean. Also, mark the fields as required.

view details

push time in 3 days

PR merged twreporter/go-api

Test news-v2 endpoint on staging

This patch merges master into staging to the test news-v2 endpoint

+14122 -120

0 comment

35 changed files

babygoat

pr closed time in 3 days

PR opened twreporter/go-api

Test news-v2 endpoint on staging

This patch merges master into staging to the test news-v2 endpoint

+14122 -120

0 comment

35 changed files

pr created time in 3 days

push eventtwreporter/go-api

babygoat

commit sha fea7d18078ba95b8a895e3ee13cb7cfddde2bcd0

api/news: add new model for posts This patch adds the new model for post/topic page data.

view details

Ching-Yang, Tseng

commit sha 3f6b2f11f1af8a1d121ec6cf6ada09cc0ae735fa

core: add mongo db connection with new driver

view details

Ching-Yang, Tseng

commit sha c86462242cf5b61f44b62e56cd5806d79ee66b2d

api/news: prototype v2 controller

view details

Ching-Yang, Tseng

commit sha b91b5c627ea54ef5d93bb40daadecdf99855d077

api/news: prototype the post/topic query model

view details

Ching-Yang, Tseng

commit sha 19fa0d8539c1e6b9a30f6603ea990223b796ca31

api/news: adjust storage interface w.r.t query

view details

Ching-Yang, Tseng

commit sha dbd641bcf3fe68a17c0f0113bbc9d2361eeaee51

api/news: add storage layer function signature This patch adds the function signatures of the storage layer. Also, implement the GetPosts function

view details

Ching-Yang, Tseng

commit sha 40421bbf1cc397f4ec0a1552d67b37524847f681

api/news: implement GetPosts in storage layer This patch implements the GetPosts by the mongo aggregation stages according to the query object. Since the fields of the data object requires different expansion context, provide buildLookupStages for the entrypoint of the lookup pipleine builder.

view details

Ching-Yang, Tseng

commit sha d5bd96d47729bb03232b399a774f49b00ba14a6a

api/news: implement GetTopics in storage layer This patch implements the GetTopics in storage layer.

view details

Ching-Yang, Tseng

commit sha dc4a5b6abab18c3573f186af864c34c296b8ec87

api/news: implement index page fetch This patch adapted from v1 query to implement index page fetch. Also, fix the typo of latest topic section. Note that during the latest section fetch, specify a time range filter(within a week) to further reduce the posts sorting effort.

view details

Ching-Yang, Tseng

commit sha 14d04df242f4904ae0bfb5eaa3f80a7d5c91ea8c

api/news: implement posts/topics list This patch implements the posts/topics list in v2 endpoint. Note that the index on posts collection should be added in the production database as the descendant order.

view details

Ching-Yang, Tseng

commit sha 916375667356a03eb41837f6072e087a227656dc

api/news: remove filter during index page fetch This patch removes the match filter which restricts the doucment published within a week during the fetch of latest post section. The performance improvments should be operated on collections index.

view details

Ching-Yang, Tseng

commit sha 8d37028158aaa4a85ed3a065c5dece8680263900

api/news: implement posts query filter This patch implments the filters parsed from the query parameter(category, tag, id). Note that this is a breaking change which adapted from v1 which only accepts json string to be parsed as the mongo query document directly. However, previous implementation is not intuitive in url context and storage dependent. This patch changes value to use the specific parameters and transform the parameters into mongo query respectively.

view details

Ching-Yang, Tseng

commit sha 1851f357875a7f31a516fb3f8ebbf55902556ffa

api/news: refactor query for single post retrieval This patch moves the single post query parser to a specific internal query function. Also, define the common query data structure within the /internal/query to share among the project scope.

view details

Ching-Yang, Tseng

commit sha 82878636d76d4e3ac6cb43d6277b2327798e7b65

api/news: refactor post list query This patch refactors post list query with the supported query parameters to internal/news/.

view details

Ching-Yang, Tseng

commit sha 7714683c92f723ba2ee763af3c8abb6cf7677c16

api/news: define constant string variables This patch defines constant string variables to replace string literal of the query parameters.

view details

Ching-Yang, Tseng

commit sha 79fe61a2aafc87c12a7c6a0d8dd499a137d07289

api/news: refactor single topic query parser This patch refactors the single topic query parser into internal/news/. Also, deduplicate the common body with into single private function parseSingleQuery.

view details

Ching-Yang, Tseng

commit sha 7396dcf37253873a2bf4a02694fc006545da4f3d

api/news: refactor topic list query parser This patch refactors the topic list query parser into ParseTopicListQuery in internal/news/.

view details

Ching-Yang, Tseng

commit sha 1b9a1f53fa72ccf23f7cd1cb0a521813d3042fe8

api/news: convert Query to mongoQuery object This patch adds the mongoQuery object and the convert function. mongoQuery object maps fields pagination/filter/sort to mongo specific query object with extra meta tag `mongo`. This tag denotes the corresponding field name in the mongo documents.

view details

Ching-Yang, Tseng

commit sha 83298ea7d162fb8a5bd2f8e7f18d09288521c610

api/news: convert mongoQuery to query documents This patch converts mongoQuery (filter/pagination/sort) to query documents respectively. Also, introduce internal/mongo package to define the wrapper and constant (stages/query operators) for mongo-go-driver.

view details

Ching-Yang, Tseng

commit sha 63993c4c6724ef3217fc72b0a5bd4f193b57a3c7

api/news: refactor lookup stages This patch adapts the lookup stages to the frontend requirements defined in the spreadsheet. Also, rewrite the lookup functions by defining lookup field maps for each retrieval case.

view details

push time in 3 days

PR merged twreporter/go-api

Features/news v2
+1818 -112

7 comments

29 changed files

babygoat

pr closed time in 3 days

startedM157q/m157q.github.io

started time in 4 days

push eventtwreporter/click-to-deploy

Ching-Yang, Tseng

commit sha 4a52eb64b732059ea247a0da5b7e6e1b9ec73d18

chore: add cloud-build-vm target This patch adds the cloud-build-vm target for triggering manual cloud build from local. Note that the required environment variables should be defined while making the target. Also, removes build from workerpool as we don't have dedicated build server. Finally, as a workaround of the internal ssh timeout issue of packer, disable the internal access as a workaround temporarily.

view details

Ching-Yang, Tseng

commit sha 49a79ecb4ea967875ffd27e007ca76eb6749b1f4

feat(mongo3.6): change package version This patch changes installed mongo package version to the latest stable mongo3 version.

view details

Ching-Yang, Tseng

commit sha 1a29656c516d571d16188ffc5409972871a0ccac

feat(mongo3.6): install stackdriver agent This patch adds the stackdriver logging/monitoring agents in the image and update the test to validate if the agents are installed properly.

view details

Ching-Yang, Tseng

commit sha 802d1763fd4392e9072a3841fac7d2676735d1dc

feat(mongo3.6): add stackdriver monitor This patch adds the configuration for stackdriver to monitor replicaset through mongo plugin.

view details

Ching-Yang, Tseng

commit sha 95255ff30d07215d023ec84fbbcbf19c2a7dbda4

feat(mongo3.6): customize mongo logger This patch customizes the mongo logger configuration to replace the default configuration in fluentd-catch-all-config. The mongo.logger.conf file describes the parser of the log of mongo(/var/log/mongodb/mongod.log) and mutates the severity string to corresponding log entry severity. Replace the log configuration of default fluentd with the customized one if the service is actived.(i.e, stackdriver-agent is installed).

view details

babygoat

commit sha 955e1cecb08f50aa7b425a2111366c798214cdd5

Merge pull request #1 from babygoat/twreporter/mongo Add the mongo cluster installation

view details

push time in 4 days

PR merged twreporter/click-to-deploy

Add the mongo cluster installation

Why Change

V1 mongo cluster setup has following drawbacks.

  • Lack of logging entries of the mongo cluster on stackdriver
  • Lack of metrics of the mongo cluster on stackdriver monitoring
  • Obscure VM image from marketplace adds maintenance cost

What does the PR do

  • Extend the VM image build steps for twreporter mongo cluster
    • Install the required Stackdriver-agent for logging and monitoring
    • Setup monitoring configuration of collectd for Stackdriver-agent
    • Setup customized configuration of fluentd for Stackdriver logging
    • Build the VM image from base OS image to control the package dependencies for future reference

Solution Overview

The click-to-deploy repo contains the source lists of solutions on Google Cloud Platform Marketplace which provision applications on containers(K8S) or VM(GCE) with one click. We mainly focus on the solution of mongo cluster by vm instances.

vm directory contains the following components:

  • Configuration Management (chef)
  • VM image builder(packer)
  • Validation scripts (tests)

Configuration Management

The chef directory contains the build time configuration and provisioning startup scripts described in cookbooks. Brief descriptions of the subdirectories of the solution are listed below.

  • attributes directory:
    • Descirbe the metadata of the main application (e.g, mongo) and other dependencies
  • files directory:
    • Contain configuration templates and runtime startup scripts
  • recipes directory:
    • Describe the vm image installation steps

VM image builder

The packer directory contains the configuration for Packer to build the system image. The packer utilize chef-solo provisioner to descirbe the target solutions and the installation order(run-list).

Cloud Build

The repo utilizes prebuilt docker image to build the vm image as described in the cloudbuild-vm.yaml on cloud-build. This prebuilt image comes with the required applications(packer, chef-solo) to run the configuration within Packer and Chef directories. After the target image is built, it will provision a disposable compute instance to run the tests against the image.

Build image

GCP_PROJECT_TO_RUN_CLOUD_BUILD=[PROJECT_ID] \
PACKER_LOGS_GCS_BUCKET_NAME=[LOGS_BUCKET] \
SERVICE_ACCOUNT_KEY_JSON_GCS=[SERVICE_ACCOUNT_ON_GCS] \
SOLUTION_NAME=[TARGET_NAME] make cloud-build-vm

Image Usage

Select the image during the GCE creation and create extra attached disk for the mongo data persistent. Note that the attached-disk should be named with [instance hostname]-data.

hostname: mongo-test-server
attached-disk: mongo-test-server-data

Also, configure following metadata on the instance

is_mongodb_server "True" // whether this instance of the cluster persists data
is_mongodb_arbiter "False" // whether this instance of the cluster is arbiter
mongodb_replicaset_name "rs0" // replicaset name of mongo cluster
mongodb_servers "MONGO_SERVER_1 MONGO_SERVER_2 MONGO_SERVER_3" // instances list of the mongo group
stackdriver_password "admin_password" // password for monitor user
+206 -6

3 comments

10 changed files

babygoat

pr closed time in 4 days

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha fe693fefbd51bad4046f05ed7bd6c192f8ea670f

doc: Update CHANGELOG This patch updates the CHANGELOG for summarying the changes so far.

view details

push time in 4 days

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha a4a3a1bf2a7d81d3e8ecc0d0cc8533485af20eb5

api/news: fix post list query parameter parser This patch fixes the post list query parameter parsing error.

view details

Ching-Yang, Tseng

commit sha af42caeb6227e89c6ff3ccb192a6175c790ad62b

api/news: fix index page response format This patch promotes the index page sections to "data" field directly.

view details

push time in 4 days

Pull request review commenttwreporter/go-api

Features/news v2

+package controllers++import (+	"context"+	"net/http"+	"sync"++	"github.com/gin-gonic/gin"+	"github.com/pkg/errors"+	log "github.com/sirupsen/logrus"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type newsV2Storage interface {+	GetFullPosts(context.Context, *news.Query) ([]news.Post, error)+	GetMetaOfPosts(context.Context, *news.Query) ([]news.MetaOfPost, error)+	GetFullTopics(context.Context, *news.Query) ([]news.Topic, error)+	GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error)++	GetPostCount(context.Context, *news.Query) (int, error)+	GetTopicCount(context.Context, *news.Query) (int, error)+}++func NewNewsV2Controller(s newsV2Storage) *newsV2Controller {+	return &newsV2Controller{s}+}++type newsV2Controller struct {+	Storage newsV2Storage+}++func (nc *newsV2Controller) GetPosts(c *gin.Context) {+	var err error++	ctx, cancel := context.WithTimeout(c, globals.Conf.News.PostPageTimeout)+	defer cancel()++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParsePostListQuery(c)++	posts, err := nc.Storage.GetMetaOfPosts(ctx, q)++	if err != nil {+		return+	}++	total, err := nc.Storage.GetPostCount(ctx, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": posts, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetAPost(c *gin.Context) {+	var post interface{}+	var err error++	ctx, cancel := context.WithTimeout(c, globals.Conf.News.PostPageTimeout)+	defer cancel()++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSinglePostQuery(c)++	if q.Full {+		var posts []news.Post+		posts, err = nc.Storage.GetFullPosts(ctx, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	} else {+		var posts []news.MetaOfPost+		posts, err = nc.Storage.GetMetaOfPosts(ctx, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	}++	if err != nil {+		return+	}++	if post == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the post from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": post})+}++func (nc *newsV2Controller) GetTopics(c *gin.Context) {+	var err error++	ctx, cancel := context.WithTimeout(c, globals.Conf.News.TopicPageTimeout)+	defer cancel()++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseTopicListQuery(c)++	topics, err := nc.Storage.GetMetaOfTopics(ctx, q)++	if err != nil {+		return+	}++	total, err := nc.Storage.GetTopicCount(ctx, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": topics, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetATopic(c *gin.Context) {+	var topic interface{}+	var err error++	ctx, cancel := context.WithTimeout(c, globals.Conf.News.TopicPageTimeout)+	defer cancel()++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSingleTopicQuery(c)++	if q.Full {+		var topics []news.Topic+		topics, err = nc.Storage.GetFullTopics(ctx, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	} else {+		var topics []news.MetaOfTopic+		topics, err = nc.Storage.GetMetaOfTopics(ctx, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	}++	// server side error+	if err != nil {+		return+	}++	if topic == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the topic from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": topic})+}++type job struct {+	Name  string+	Type  string+	Query *news.Query+}++type result struct {+	Name    string+	Content interface{}+	Error   error+}++const (+	typePost  = "post"+	typeTopic = "topic"+)++func (nc *newsV2Controller) GetIndexPage(c *gin.Context) {+	ctx, cancel := context.WithTimeout(c, globals.Conf.News.IndexPageTimeout)+	defer cancel()++	jobs := nc.getIndexPageJobs()++	var err error+	results := make(map[string]interface{})++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	for result := range nc.fetchjobs(ctx, jobs) {+		select {+		case <-ctx.Done():+			err = errors.WithStack(ctx.Err())+			break+		default:+			if result.Error != nil {+				err = result.Error+				break+			}+			results[result.Name] = result.Content+			if len(results) == len(jobs) {+				break+			}+		}+	}+	if err != nil {+		return+	}+	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": results}})++}++func (nc *newsV2Controller) getIndexPageJobs() []job {+	// v1 index page section jobs+	jobs := []job{+		{+			Name:  news.LatestSection,+			Type:  typePost,+			Query: news.NewQuery(news.WithLimit(6)),+		}, {+			Name: news.EditorPicksSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterIsFeatured(true),+				news.WithLimit(6),+				news.WithSortUpdatedAt(false)),+		}, {+			Name:  news.LatestTopicSection,+			Type:  typeTopic,+			Query: news.NewQuery(news.WithLimit(1)),+		}, {+			Name: news.ReviewsSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterCategoryIDs(news.Review.ID),+				news.WithLimit(4)),+		}, {+			Name: news.PhotoSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterCategoryIDs(news.Photography.ID),+				news.WithLimit(6)),+		}, {+			Name:  news.InfographicSection,+			Type:  typePost,+			Query: news.NewQuery(news.WithFilterStyle("interactive"), news.WithLimit(6)),+		}, {+			Name:  news.TopicsSection,+			Type:  typeTopic,+			Query: news.NewQuery(news.WithOffset(1), news.WithLimit(4)),+		},+	}++	// v1 categories in index page+	for _, v := range []news.Category{+		news.HumanRightsAndSociety,+		news.EnvironmentAndEducation,+		news.PoliticsAndEconomy,+		news.CultureAndArt,+		news.International,+		news.LivingAndMedicalCare,+	} {+		jobs = append(jobs, job{+			v.Name,+			typePost,+			news.NewQuery(news.WithFilterCategoryIDs(v.ID), news.WithLimit(1)),+		})+	}++	return jobs+}++func (nc *newsV2Controller) fetchjobs(ctx context.Context, jobs []job) <-chan result {+	resultStream := make(chan result)++	var wg sync.WaitGroup++	wg.Add(len(jobs))+	for _, j := range jobs {+		go func(job job) {+			defer wg.Done()+			switch job.Type {+			case typePost:+				posts, err := nc.Storage.GetMetaOfPosts(ctx, job.Query)+				result := result{+					Name:    job.Name,+					Content: posts,+					Error:   err,+				}+				resultStream <- result+			case typeTopic:+				topics, err := nc.Storage.GetMetaOfTopics(ctx, job.Query)+				result := result{+					Name:    job.Name,+					Content: topics,+					Error:   err,+				}+				resultStream <- result+			}+		}(j)+	}++	go func() {+		defer close(resultStream)+		wg.Wait()+	}()+	return resultStream+}++func (nc *newsV2Controller) helperCleanup(c *gin.Context, err error) {+	if err != nil {+		switch {+		case errors.Is(err, context.DeadlineExceeded):

Yes, you should have at least go version > 1.13.0. That's why I upgraded the go runtime of the image to the latest version.

babygoat

comment created time in 5 days

Pull request review commenttwreporter/go-api

Features/news v2

+package controllers++import (+	"context"+	"net/http"+	"runtime"+	"sync"++	"github.com/gin-gonic/gin"+	"github.com/pkg/errors"+	log "github.com/sirupsen/logrus"+	"twreporter.org/go-api/internal/news"+)++type newsV2Storage interface {+	GetFullPosts(context.Context, *news.Query) ([]news.Post, error)+	GetMetaOfPosts(context.Context, *news.Query) ([]news.MetaOfPost, error)+	GetFullTopics(context.Context, *news.Query) ([]news.Topic, error)+	GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error)++	GetPostCount(context.Context, *news.Query) (int, error)+	GetTopicCount(context.Context, *news.Query) (int, error)+}++func NewNewsV2Controller(s newsV2Storage) *newsV2Controller {+	return &newsV2Controller{s}+}++type newsV2Controller struct {+	Storage newsV2Storage+}++func (nc *newsV2Controller) GetPosts(c *gin.Context) {+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParsePostListQuery(c)++	// TODO(babygoat): config context with proper timeout+	posts, err := nc.Storage.GetMetaOfPosts(c, q)++	if err != nil {+		return+	}++	// TODO(babygoat): config context with proper timeout+	total, err := nc.Storage.GetPostCount(c, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": posts, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetAPost(c *gin.Context) {+	var post interface{}+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSinglePostQuery(c)++	if q.Full {+		var posts []news.Post+		// TODO(babygoat): config context with proper timeout+		posts, err = nc.Storage.GetFullPosts(c, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	} else {+		var posts []news.MetaOfPost+		// TODO(babygoat): config context with proper timeout+		posts, err = nc.Storage.GetMetaOfPosts(c, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	}++	if err != nil {+		return+	}++	if post == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the post from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": post})+}++func (nc *newsV2Controller) GetTopics(c *gin.Context) {+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseTopicListQuery(c)++	// TODO(babygoat): config context with proper timeout+	topics, err := nc.Storage.GetMetaOfTopics(c, q)++	if err != nil {+		return+	}++	// TODO(babygoat): config context with proper timeout+	total, err := nc.Storage.GetTopicCount(c, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": topics, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetATopic(c *gin.Context) {+	var topic interface{}+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSingleTopicQuery(c)++	if q.Full {+		var topics []news.Topic+		// TODO(babygoat): config context with proper timeout+		topics, err = nc.Storage.GetFullTopics(c, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	} else {+		var topics []news.MetaOfTopic+		// TODO(babygoat): config context with proper timeout+		topics, err = nc.Storage.GetMetaOfTopics(c, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	}++	// server side error+	if err != nil {+		return+	}++	if topic == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the topic from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": topic})+}++type job struct {+	Name  string+	Type  string+	Query *news.Query+}++type result struct {+	Name    string+	Content interface{}+	Error   error+}++const (+	typePost  = "post"+	typeTopic = "topic"+)++func (nc *newsV2Controller) GetIndexPage(c *gin.Context) {+	// TODO(babygoat): config context with proper timeout+	ctx := c++	jobs := nc.getIndexPageJobs()+	jobStream := nc.preparejobs(ctx, jobs)++	workers := make([]<-chan result, runtime.NumCPU())+	for i, _ := range workers {+		workers[i] = nc.fetchjobs(ctx, jobStream)+	}++	var err error+	results := make(map[string]interface{})++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()+	for result := range nc.merge(ctx, workers...) {+		select {+		case <-ctx.Done():+			err = errors.WithStack(ctx.Err())+			break+		default:+			if result.Error != nil {+				err = result.Error+				break+			}+			results[result.Name] = result.Content+			if len(results) == len(jobs) {+				break+			}+		}+	}+	if err != nil {+		return+	}+	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": results}})++}++func (nc *newsV2Controller) getIndexPageJobs() []job {+	// v1 index page section jobs+	jobs := []job{+		{+			Name:  news.LatestSection,+			Type:  typePost,+			Query: news.NewQuery(news.WithLimit(6)),+		}, {+			Name: news.EditorPicksSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterIsFeatured(true),+				news.WithLimit(6),+				news.WithSortUpdatedAt(false)),+		}, {+			Name:  news.LatestTopicSection,+			Type:  typeTopic,+			Query: news.NewQuery(news.WithLimit(1)),+		}, {+			Name: news.ReviewsSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterCategoryIDs(news.Review.ID),+				news.WithLimit(4)),+		}, {+			Name: news.PhotoSection,+			Type: typePost,+			Query: news.NewQuery(+				news.WithFilterCategoryIDs(news.Photography.ID),+				news.WithLimit(6)),+		}, {+			Name:  news.InfographicSection,+			Type:  typePost,+			Query: news.NewQuery(news.WithFilterStyle("interactive"), news.WithLimit(6)),+		}, {+			Name:  news.TopicsSection,+			Type:  typeTopic,+			Query: news.NewQuery(news.WithOffset(1), news.WithLimit(4)),+		},+	}++	// v1 categories in index page+	jobs = append(jobs, func(categories ...news.Category) []job {+		var jobs []job+		for _, v := range categories {+			jobs = append(jobs, job{+				v.Name,+				typePost,+				news.NewQuery(news.WithFilterCategoryIDs(v.ID), news.WithLimit(1)),+			})+		}+		return jobs+	}(+		news.HumanRightsAndSociety,+		news.EnvironmentAndEducation,+		news.PoliticsAndEconomy,+		news.CultureAndArt,+		news.International,+		news.LivingAndMedicalCare,+	)...)

updated.

babygoat

comment created time in 8 days

pull request commenttwreporter/go-api

Features/news v2

Updated. Sorry for accidentally force-pushed the commits. Change starts from 9c32bde

babygoat

comment created time in 8 days

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha c5729ab46903dffb14d4e2f189465438971dbde4

doc: adds v2 endpoint of getting a post This patch adds the api document of describing the v2 endpoint (/v2/posts/:slug).

view details

Ching-Yang, Tseng

commit sha 0bb1f95d8d9420240a21a81ee7eeb403e2b6fad8

doc: add v2 endpoint for listing posts This patch adds the v2 endpoint document for listing posts according to following options Filter * category_id * tag_id * id Pagination * offset * limit Sort * updated_at * published_date

view details

Ching-Yang, Tseng

commit sha 5f876e9fa075447fc628e52a7e24ffb729db93a3

doc: add the v2 endpoint to get a single topic This patc adds the endopint specification of the v2 single topic infomration retrieval. Also, move the common asset data structure into asset.apib.

view details

Ching-Yang, Tseng

commit sha c74559fa7463cac0a2a48123bdd7a447f86aa1a4

doc: add the v2 endpoint of listing topic This patch adds the v2 endpoint of listing topic with respect ot following options. Pagination * offset * limit Sort * sort

view details

Ching-Yang, Tseng

commit sha 9ffbfa32bcef9895f2cc667ada8a2638d15e5150

doc: generate result document This patch generates the final html file for api document

view details

Ching-Yang, Tseng

commit sha f6aba810246cee78d04f9e06b7feb91e098c54f5

doc: add /v2/index_page endpoint This patch adds the endpoint document for /v2/index_page.

view details

Ching-Yang, Tseng

commit sha dcd8975adba21eaad4dba69093f9395d595cb79f

doc: adjust /v2/posts endpoints This patch adjusts /v2/posts & /v2/posts/:slug endpoints response according to the payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1523711569)

view details

Ching-Yang, Tseng

commit sha 1590853f052c397777ca3057c9dd602047b27951

doc: adjust /v2/topics endpoints This patch adjust /v2/topics and /v2/topics/:slug endponts with respect to the response payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1659711178)

view details

Ching-Yang, Tseng

commit sha 93af75242aed38975050cd1fa928cbaf00d589e0

doc: describe different response w.r.t parameter This patch specifies the full=true on the request description to differentiate the same response code with different response body with respect to the query parameter setup.

view details

Ching-Yang, Tseng

commit sha 7c7410760cc63ca96bc2871b51d1a6188b74fd4e

doc: add client side error for invalid slug This patch adds the client side error (404) during the single post/topic retrieval when the slug does not exist within the posts or topics collection.

view details

Ching-Yang, Tseng

commit sha e9f7f769345a0d89d457d07dfc9e4bb3ba8ed63c

doc: give relateds sample value instead of default This patch gives the relateds field sample value instead of using aglio string default value `hello, world` to approach the object string data stype.

view details

Ching-Yang, Tseng

commit sha 7f177d2606ec80c2aec7d19e973ddce84d9be589

doc: fix nested array field schema type missing This patch fixs schema of nested array fields which should be both required and fixed-type(inherit the object type).

view details

Ching-Yang, Tseng

commit sha 8d2ba673bfd88997a04d19168d2121e300382892

doc: singularize topics field This patch renames the `topics` field of a single post to `topic`. Also, add the topic field as the required field in the full post response.

view details

Ching-Yang, Tseng

commit sha aeb4a20d8c6d7132b47a1dd9df2f5b5883743507

doc: add missing required fields This patch adds the updated_at/published_date fields to required fields so that the schema of the topic/post fit the requirement. Also, rename the query parameter naming of updated_date back to updated_at.

view details

Ching-Yang, Tseng

commit sha 2637e1b114012f3c89d492da0d15402f6130d563

doc: return meta instead of empty content during post/topic list This patch replaces status no content with status ok with meta information(e.g., pagination) and empty record array.

view details

Ching-Yang, Tseng

commit sha f19d8a5ea6cc0fb71366f2731fb0c6e5ba1f49a4

api/news: replace field writters with writers This patch replaces the field `writters` with `writers` in full post data.

view details

Ching-Yang, Tseng

commit sha 6533e85511d6f7efca67b59a22841e13b1bff332

doc: fix `full` field type in Post group This patch updates the `full` field type from string to boolean. Also, mark the full field as required in the MetaOfPost model.

view details

Ching-Yang, Tseng

commit sha c14a33993395ac316eeb6f485f5b0e3b670d6d9d

doc: fix `full` field type in Topic This patch updates the `full` field type from string to boolean. Also, mark the fields as required.

view details

babygoat

commit sha 0edf28910123b4ae87f31aa0c57d854c16a8f80d

Merge pull request #387 from babygoat/posts-v2-api-doc Posts v2 api doc

view details

babygoat

commit sha fea7d18078ba95b8a895e3ee13cb7cfddde2bcd0

api/news: add new model for posts This patch adds the new model for post/topic page data.

view details

push time in 8 days

startedgoogle/go-cloud

started time in 12 days

push eventtwreporter/go-api

Ching-Yang, Tseng

commit sha c5729ab46903dffb14d4e2f189465438971dbde4

doc: adds v2 endpoint of getting a post This patch adds the api document of describing the v2 endpoint (/v2/posts/:slug).

view details

Ching-Yang, Tseng

commit sha 0bb1f95d8d9420240a21a81ee7eeb403e2b6fad8

doc: add v2 endpoint for listing posts This patch adds the v2 endpoint document for listing posts according to following options Filter * category_id * tag_id * id Pagination * offset * limit Sort * updated_at * published_date

view details

Ching-Yang, Tseng

commit sha 5f876e9fa075447fc628e52a7e24ffb729db93a3

doc: add the v2 endpoint to get a single topic This patc adds the endopint specification of the v2 single topic infomration retrieval. Also, move the common asset data structure into asset.apib.

view details

Ching-Yang, Tseng

commit sha c74559fa7463cac0a2a48123bdd7a447f86aa1a4

doc: add the v2 endpoint of listing topic This patch adds the v2 endpoint of listing topic with respect ot following options. Pagination * offset * limit Sort * sort

view details

Ching-Yang, Tseng

commit sha 9ffbfa32bcef9895f2cc667ada8a2638d15e5150

doc: generate result document This patch generates the final html file for api document

view details

Ching-Yang, Tseng

commit sha f6aba810246cee78d04f9e06b7feb91e098c54f5

doc: add /v2/index_page endpoint This patch adds the endpoint document for /v2/index_page.

view details

Ching-Yang, Tseng

commit sha dcd8975adba21eaad4dba69093f9395d595cb79f

doc: adjust /v2/posts endpoints This patch adjusts /v2/posts & /v2/posts/:slug endpoints response according to the payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1523711569)

view details

Ching-Yang, Tseng

commit sha 1590853f052c397777ca3057c9dd602047b27951

doc: adjust /v2/topics endpoints This patch adjust /v2/topics and /v2/topics/:slug endponts with respect to the response payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1659711178)

view details

Ching-Yang, Tseng

commit sha 93af75242aed38975050cd1fa928cbaf00d589e0

doc: describe different response w.r.t parameter This patch specifies the full=true on the request description to differentiate the same response code with different response body with respect to the query parameter setup.

view details

Ching-Yang, Tseng

commit sha 7c7410760cc63ca96bc2871b51d1a6188b74fd4e

doc: add client side error for invalid slug This patch adds the client side error (404) during the single post/topic retrieval when the slug does not exist within the posts or topics collection.

view details

Ching-Yang, Tseng

commit sha e9f7f769345a0d89d457d07dfc9e4bb3ba8ed63c

doc: give relateds sample value instead of default This patch gives the relateds field sample value instead of using aglio string default value `hello, world` to approach the object string data stype.

view details

Ching-Yang, Tseng

commit sha 7f177d2606ec80c2aec7d19e973ddce84d9be589

doc: fix nested array field schema type missing This patch fixs schema of nested array fields which should be both required and fixed-type(inherit the object type).

view details

Ching-Yang, Tseng

commit sha 8d2ba673bfd88997a04d19168d2121e300382892

doc: singularize topics field This patch renames the `topics` field of a single post to `topic`. Also, add the topic field as the required field in the full post response.

view details

Ching-Yang, Tseng

commit sha aeb4a20d8c6d7132b47a1dd9df2f5b5883743507

doc: add missing required fields This patch adds the updated_at/published_date fields to required fields so that the schema of the topic/post fit the requirement. Also, rename the query parameter naming of updated_date back to updated_at.

view details

Ching-Yang, Tseng

commit sha 2637e1b114012f3c89d492da0d15402f6130d563

doc: return meta instead of empty content during post/topic list This patch replaces status no content with status ok with meta information(e.g., pagination) and empty record array.

view details

Ching-Yang, Tseng

commit sha f19d8a5ea6cc0fb71366f2731fb0c6e5ba1f49a4

api/news: replace field writters with writers This patch replaces the field `writters` with `writers` in full post data.

view details

Ching-Yang, Tseng

commit sha 6533e85511d6f7efca67b59a22841e13b1bff332

doc: fix `full` field type in Post group This patch updates the `full` field type from string to boolean. Also, mark the full field as required in the MetaOfPost model.

view details

Ching-Yang, Tseng

commit sha c14a33993395ac316eeb6f485f5b0e3b670d6d9d

doc: fix `full` field type in Topic This patch updates the `full` field type from string to boolean. Also, mark the fields as required.

view details

babygoat

commit sha 0edf28910123b4ae87f31aa0c57d854c16a8f80d

Merge pull request #387 from babygoat/posts-v2-api-doc Posts v2 api doc

view details

push time in 13 days

PR merged twreporter/go-api

Posts v2 api doc

Adds api documents for following endpoints according to spreadsheet

  • [x] /v2/posts/:slug
  • [x] /v2/posts
  • [x] /v2/topics/:slug
  • [x] /v2/topics
  • [x] /v2/index_page

Also, after the index_page & index_page response shrinking, do we still need these endpoints? IMO, frontend can send the batch queries to backend posts/topics endpoints with corresponding query parameters.

Pros: each payloads are small. Cons: Increase both (BE/FE) server file I/O (socket connections)

+12304 -2

5 comments

6 changed files

babygoat

pr closed time in 13 days

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data+            + meta (meta, fixed-type, required)+            + records (array[MetaOfTopic], fixed-type, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Topic [/v2/topics/{slug}{?full}]+Contain meta(brief) or full information of a topic with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-topic` (required) - Topic slug+    + full: `true` (optional) - Whether to retrieve a topic with full information+        + Default: `false`++### Get a single topic [GET]+Get a single topic with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfTopic, required)+++ Response 404 (application/json)+    +    + Attributes+        + status: fail (required)+        + data (required)+            + slug: Cannot find the topic from the slug (required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request with full=true+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullTopic, required)++# Data Structures++## FullTopic++ include MetaOfTopic++ relateds_background: in-row++ relateds_format: `#5E5E41`++ title_position: center++ lead_video (video, required)++ headline: topic headline (required)++ subtitle: topic subtitle (required)++ description (paragraphs, required)++ team_description (paragraphs, required)++ og_title: topic og title (required)++## MetaOfTopic++ id: 5edf118c3e631f0600198935 (required)++ slug: `a-slug-of-the-topic` (required)++ title: topic title (required)++ `short_title`: short title (required)++ published_date: 2020-06-8T16:00:00Z (required)++ og_description: topic description (required)++ og_image (image, required)++ leading_image (image, required)++ leading_image_portrait (image, required)++ relateds (array, fixed-type, required)+    + 5edf118c3e631f0600198935

updated.

babygoat

comment created time in 13 days

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data+            + meta (meta, fixed-type, required)+            + records (array[MetaOfTopic], fixed-type, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Topic [/v2/topics/{slug}{?full}]+Contain meta(brief) or full information of a topic with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-topic` (required) - Topic slug+    + full: `true` (optional) - Whether to retrieve a topic with full information+        + Default: `false`++### Get a single topic [GET]+Get a single topic with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfTopic, required)+++ Response 404 (application/json)+    +    + Attributes+        + status: fail (required)+        + data (required)+            + slug: Cannot find the topic from the slug (required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request with full=true+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullTopic, required)++# Data Structures++## FullTopic++ include MetaOfTopic++ relateds_background: in-row++ relateds_format: `#5E5E41`++ title_position: center++ lead_video (video, required)++ headline: topic headline (required)++ subtitle: topic subtitle (required)++ description (paragraphs, required)++ team_description (paragraphs, required)++ og_title: topic og title (required)

updated

babygoat

comment created time in 13 days

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data+            + meta (meta, fixed-type, required)+            + records (array[MetaOfPost], fixed-type, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 404 (application/json)+    +    + Attributes+        + status: fail (required)+        + data (required)+            + slug: Cannot find the post from the slug (required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request with full=true+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullPost, required)++# Data Structures++## FullPost++ include MetaOfPost++ brief (paragraphs, required)++ content (paragraphs, required)++ copyright: copyrighted (required)++ designers (array[author], fixed-type, required)++ engineers (array[author], fixed-type, required)++ extend_byline: extended lines (required)++ leading_image_description: description (required)++ og_title: og title (required)++ photographers (array[author], fixed-type, required)++ relateds (array, fixed-type, required)+    + 5edf118c3e631f0600198935++ topic (required)+    + title: topic title (required)+    + `short_title`: topic short title (required)+    + slug: `the-slug-of-the-topic` (required)+    + relateds (array, fixed-type, required)+        + 5edf118c3e631f0600198935++ updated_at: `2020-06-8T16:00:00Z` (required)++ writers (array[author], fixed-type, required)++ hero_image_size: normal (required)++ full: true (required)++## MetaOfPost++ id: 5edf118c3e631f0600198935 (required)++ style: article:v2:default (required)++ slug: `a-slug-of-the-post` (required)++ leading_image_portrait (image, required)++ hero_image  (image, required)++ og_image (image, required)++ og_description: post description (required)++ title: post title (required)++ subtitle: post subtitle (required)++ categories (array[category], fixed-type, required)++ published_date: `2020-06-8T16:00:00Z` (required)++ is_external: false (boolean, required)++ tags (array[tag], fixed-type, required)

updated.

babygoat

comment created time in 13 days

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 6533e85511d6f7efca67b59a22841e13b1bff332

doc: fix `full` field type in Post group This patch updates the `full` field type from string to boolean. Also, mark the full field as required in the MetaOfPost model.

view details

Ching-Yang, Tseng

commit sha c14a33993395ac316eeb6f485f5b0e3b670d6d9d

doc: fix `full` field type in Topic This patch updates the `full` field type from string to boolean. Also, mark the fields as required.

view details

push time in 13 days

pull request commenttwreporter/click-to-deploy

Add the mongo cluster installation

I have submitted the PR for the detailed installation steps. Please see the wiki.

babygoat

comment created time in 16 days

create barnchbabygoat/click-to-deploy

branch : twreporter/mongo

created branch time in 20 days

fork babygoat/click-to-deploy

Source for Google Click to Deploy solutions listed on Google Cloud Marketplace.

fork in 20 days

create barnchbabygoat/click-to-deploy

branch : twreporter/mongo

created branch time in 20 days

Pull request review commenttwreporter/go-api

Features/news v2

+package controllers++import (+	"context"+	"net/http"+	"runtime"+	"sync"++	"github.com/gin-gonic/gin"+	"github.com/pkg/errors"+	log "github.com/sirupsen/logrus"+	"twreporter.org/go-api/internal/news"+)++type newsV2Storage interface {+	GetFullPosts(context.Context, *news.Query) ([]news.Post, error)+	GetMetaOfPosts(context.Context, *news.Query) ([]news.MetaOfPost, error)+	GetFullTopics(context.Context, *news.Query) ([]news.Topic, error)+	GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error)++	GetPostCount(context.Context, *news.Query) (int, error)+	GetTopicCount(context.Context, *news.Query) (int, error)+}++func NewNewsV2Controller(s newsV2Storage) *newsV2Controller {+	return &newsV2Controller{s}+}++type newsV2Controller struct {+	Storage newsV2Storage+}++func (nc *newsV2Controller) GetPosts(c *gin.Context) {+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParsePostListQuery(c)++	// TODO(babygoat): config context with proper timeout+	posts, err := nc.Storage.GetMetaOfPosts(c, q)++	if err != nil {+		return+	}++	// TODO(babygoat): config context with proper timeout+	total, err := nc.Storage.GetPostCount(c, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": posts, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetAPost(c *gin.Context) {+	var post interface{}+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSinglePostQuery(c)++	if q.Full {+		var posts []news.Post+		// TODO(babygoat): config context with proper timeout+		posts, err = nc.Storage.GetFullPosts(c, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	} else {+		var posts []news.MetaOfPost+		// TODO(babygoat): config context with proper timeout+		posts, err = nc.Storage.GetMetaOfPosts(c, q)+		if len(posts) > 0 {+			post = posts[0]+		}+	}++	if err != nil {+		return+	}++	if post == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the post from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": post})+}++func (nc *newsV2Controller) GetTopics(c *gin.Context) {+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseTopicListQuery(c)++	// TODO(babygoat): config context with proper timeout+	topics, err := nc.Storage.GetMetaOfTopics(c, q)++	if err != nil {+		return+	}++	// TODO(babygoat): config context with proper timeout+	total, err := nc.Storage.GetTopicCount(c, q)+	if err != nil {+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": topics, "meta": gin.H{+		"total":  total,+		"offset": q.Offset,+		"limit":  q.Limit,+	}}})+}++func (nc *newsV2Controller) GetATopic(c *gin.Context) {+	var topic interface{}+	var err error++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()++	q := news.ParseSingleTopicQuery(c)++	if q.Full {+		var topics []news.Topic+		// TODO(babygoat): config context with proper timeout+		topics, err = nc.Storage.GetFullTopics(c, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	} else {+		var topics []news.MetaOfTopic+		// TODO(babygoat): config context with proper timeout+		topics, err = nc.Storage.GetMetaOfTopics(c, q)+		if len(topics) > 0 {+			topic = topics[0]+		}+	}++	// server side error+	if err != nil {+		return+	}++	if topic == nil {+		c.JSON(http.StatusNotFound, gin.H{"status": "fail", "data": gin.H{"slug": "Cannot find the topic from the slug"}})+		return+	}++	c.JSON(http.StatusOK, gin.H{"status": "success", "data": topic})+}++type job struct {+	Name  string+	Type  string+	Query *news.Query+}++type result struct {+	Name    string+	Content interface{}+	Error   error+}++const (+	typePost  = "post"+	typeTopic = "topic"+)++func (nc *newsV2Controller) GetIndexPage(c *gin.Context) {+	// TODO(babygoat): config context with proper timeout+	ctx := c++	jobs := nc.getIndexPageJobs()+	jobStream := nc.preparejobs(ctx, jobs)++	workers := make([]<-chan result, runtime.NumCPU())+	for i, _ := range workers {+		workers[i] = nc.fetchjobs(ctx, jobStream)+	}++	var err error+	results := make(map[string]interface{})++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()+	for result := range nc.merge(ctx, workers...) {+		select {+		case <-ctx.Done():+			err = errors.WithStack(ctx.Err())+			break+		default:+			if result.Error != nil {+				err = result.Error+				break+			}+			results[result.Name] = result.Content+			if len(results) == len(jobs) {+				break+			}+		}+	}

Can you elaborate more details?

These are general and idiomatic concurrent patterns including fan-out, multiplexing(nc.prepareJobs, nc.fetchJobs) and fan-in (nc.merge). Here are some reference for these common pattern in golang

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

+package storage++import (+	"context"++	"github.com/pkg/errors"+	"go.mongodb.org/mongo-driver/bson"+	"go.mongodb.org/mongo-driver/mongo"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type fetchResult struct {+	Content interface{}+	Error   error+}++type mongoStorage struct {+	*mongo.Client+}++func NewMongoV2Storage(client *mongo.Client) *mongoStorage {+	return &mongoStorage{client}+}++func (m *mongoStorage) GetFullPosts(ctx context.Context, q *news.Query) ([]news.Post, error) {+	var posts []news.Post++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())

This is to deal with the case if timeout occurs and the m.getFullPosts returns earlier than the ctx.Done(). However, since the m.getFullPosts always return result, it seems to me that this can be saved as well.

babygoat

comment created time in a month

pull request commenttwreporter/go-api

Features/news v2

Updated.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

+package storage++import (+	"context"++	"github.com/pkg/errors"+	"go.mongodb.org/mongo-driver/bson"+	"go.mongodb.org/mongo-driver/mongo"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type fetchResult struct {+	Content interface{}+	Error   error+}++type mongoStorage struct {+	*mongo.Client+}++func NewMongoV2Storage(client *mongo.Client) *mongoStorage {+	return &mongoStorage{client}+}++func (m *mongoStorage) GetFullPosts(ctx context.Context, q *news.Query) ([]news.Post, error) {+	var posts []news.Post++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())+		}+		posts = result.Content.([]news.Post)+	}++	return posts, nil+}++func (m *mongoStorage) getFullPosts(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)

The result channel will be closed by the anonymous function below from another goroutine. As the context was passed to that goroutine, it would either return or timeout and then close the channel properly. Also, we only transfer single result not multiple results, hence, we don't need to declare the buffer channel.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

+package storage++import (+	"context"++	"github.com/pkg/errors"+	"go.mongodb.org/mongo-driver/bson"+	"go.mongodb.org/mongo-driver/mongo"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type fetchResult struct {+	Content interface{}+	Error   error+}++type mongoStorage struct {+	*mongo.Client+}++func NewMongoV2Storage(client *mongo.Client) *mongoStorage {+	return &mongoStorage{client}+}++func (m *mongoStorage) GetFullPosts(ctx context.Context, q *news.Query) ([]news.Post, error) {+	var posts []news.Post++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())

Yes, the ok served as indicator whether the channel was closed or not. This assertion is idempotent.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)

Yes, the above error handling section tries to deal with the cases which the assertion would be failed. If these were done properly, then I believed validation here can be saved.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)+		for i := 0; i < len(topics); i++ {+			topics[i].Full = true+		}+	}++	return topics, nil }-func (m *mongoStorage) GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error) {-	return nil, nil++func (m *mongoStorage) getFullTopics(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)+	go func(ctx context.Context, stages []bson.D) {+		defer close(result)+		cursor, err := m.Database(globals.Conf.DB.Mongo.DBname).Collection(news.ColTopics).Aggregate(ctx, stages)+		if err != nil {+			result <- fetchResult{Error: errors.WithStack(err)}+		}+		defer cursor.Close(ctx)++		var topics []news.Topic+		for cursor.Next(ctx) {+			var topic news.Topic+			err := cursor.Decode(&topic)+			if err != nil {+				result <- fetchResult{Error: errors.WithStack(err)}+			}+			topics = append(topics, topic)+		}+		result <- fetchResult{Content: topics}+	}(ctx, stages)+	return result }++func (m *mongoStorage) GetMetaOfTopics(ctx context.Context, q *news.Query) ([]news.MetaOfTopic, error) {+	var topics []news.MetaOfTopic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupMetaOfTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getMetaOfTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.MetaOfTopic)+	}++	return topics, nil+}++func (m *mongoStorage) getMetaOfTopics(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)+	go func(ctx context.Context, stages []bson.D) {+		defer close(result)+		cursor, err := m.Database(globals.Conf.DB.Mongo.DBname).Collection(news.ColTopics).Aggregate(ctx, stages)+		if err != nil {+			result <- fetchResult{Error: errors.WithStack(err)}+		}+		defer cursor.Close(ctx)++		var topics []news.MetaOfTopic+		for cursor.Next(ctx) {+			var topic news.MetaOfTopic+			err := cursor.Decode(&topic)+			if err != nil {+				result <- fetchResult{Error: errors.WithStack(err)}+			}+			topics = append(topics, topic)+		}

done

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)+		for i := 0; i < len(topics); i++ {+			topics[i].Full = true+		}+	}++	return topics, nil }-func (m *mongoStorage) GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error) {-	return nil, nil++func (m *mongoStorage) getFullTopics(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)+	go func(ctx context.Context, stages []bson.D) {+		defer close(result)+		cursor, err := m.Database(globals.Conf.DB.Mongo.DBname).Collection(news.ColTopics).Aggregate(ctx, stages)+		if err != nil {+			result <- fetchResult{Error: errors.WithStack(err)}+		}+		defer cursor.Close(ctx)++		var topics []news.Topic+		for cursor.Next(ctx) {+			var topic news.Topic+			err := cursor.Decode(&topic)+			if err != nil {+				result <- fetchResult{Error: errors.WithStack(err)}+			}+			topics = append(topics, topic)+		}

done

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

+package storage++import (+	"context"++	"github.com/pkg/errors"+	"go.mongodb.org/mongo-driver/bson"+	"go.mongodb.org/mongo-driver/mongo"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type fetchResult struct {+	Content interface{}+	Error   error+}++type mongoStorage struct {+	*mongo.Client+}++func NewMongoV2Storage(client *mongo.Client) *mongoStorage {+	return &mongoStorage{client}+}++func (m *mongoStorage) GetFullPosts(ctx context.Context, q *news.Query) ([]news.Post, error) {+	var posts []news.Post++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())+		}

done

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

+package storage++import (+	"context"++	"github.com/pkg/errors"+	"go.mongodb.org/mongo-driver/bson"+	"go.mongodb.org/mongo-driver/mongo"+	"twreporter.org/go-api/globals"+	"twreporter.org/go-api/internal/news"+)++type fetchResult struct {+	Content interface{}+	Error   error+}++type mongoStorage struct {+	*mongo.Client+}++func NewMongoV2Storage(client *mongo.Client) *mongoStorage {+	return &mongoStorage{client}+}++func (m *mongoStorage) GetFullPosts(ctx context.Context, q *news.Query) ([]news.Post, error) {+	var posts []news.Post++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())+		}+		posts = result.Content.([]news.Post)+	}++	return posts, nil+}++func (m *mongoStorage) getFullPosts(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)+	go func(ctx context.Context, stages []bson.D) {+		defer close(result)+		cursor, err := m.Database(globals.Conf.DB.Mongo.DBname).Collection(news.ColPosts).Aggregate(ctx, stages)+		if err != nil {+			result <- fetchResult{Error: errors.WithStack(err)}+		}+		defer cursor.Close(ctx)++		var posts []news.Post+		for cursor.Next(ctx) {+			var post news.Post+			err := cursor.Decode(&post)+			if err != nil {+				result <- fetchResult{Error: errors.WithStack(err)}+			}+			posts = append(posts, post)+		}+		result <- fetchResult{Content: posts}+	}(ctx, stages)+	return result+}++func (m *mongoStorage) GetMetaOfPosts(ctx context.Context, q *news.Query) ([]news.MetaOfPost, error) {+	var posts []news.MetaOfPost++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupMetaOfPost)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getMetaOfPosts(ctx, stages):+		if !ok {+			return nil, errors.WithStack(ctx.Err())+		}

done

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (nc *newsV2Controller) GetATopic(c *gin.Context) { 	c.JSON(http.StatusOK, gin.H{"status": "success", "data": topic}) } +type job struct {+	Name  string+	Type  string+	Query *news.Query+}++type result struct {+	Name    string+	Content interface{}+	Error   error+}++const (+	typePost  = "post"+	typeTopic = "topic"+)+ func (nc *newsV2Controller) GetIndexPage(c *gin.Context) {+	// TODO(babygoat): config context with proper timeout+	ctx := c++	jobs := nc.getIndexPageJobs()+	jobStream := nc.preparejobs(ctx, jobs)++	workers := make([]<-chan result, runtime.NumCPU())+	for i, _ := range workers {+		workers[i] = nc.fetchjobs(ctx, jobStream)+	}++	var err error+	results := make(map[string]interface{})++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()+	for result := range nc.merge(ctx, workers...) {+		select {+		case <-ctx.Done():+			err = errors.WithStack(ctx.Err())+			break+		default:+			if result.Error != nil {+				err = result.Error+				break+			}+			results[result.Name] = result.Content+			if len(results) == len(jobs) {+				break+			}+		}+	}+	if err != nil {+		return+	}+	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": results}})++}++func (nc *newsV2Controller) getIndexPageJobs() []job {+	const statePublished = "published"++	// v1 index page section jobs+	jobs := []job{+		{+			Name:  news.LatestSection,+			Type:  typePost,+			Query: news.NewQuery(news.FilterState(statePublished), news.Limit(6)),

done

babygoat

comment created time in a month

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha b74a6f3b45243e1e9ee813ee4b7aa28e145be383

api/news: bail out error and handle cursor error This patch improves the storage layer of news to return ASAP once error occurs. Also, adds the mongo cursor fetch error handle case to return with the wrapped error.

view details

Ching-Yang, Tseng

commit sha 6203ef0401d1ac792d87df7db2d9195a435ec8a6

api/news: group Option functions This patch define the Option type for closure function fun(*Query) so that all the option functions can be grouped under the Option type. Also, prefix these functions with `With` denote the option usage.

view details

Ching-Yang, Tseng

commit sha 4343f143b6515a937dcf03beb9884354afaf9b69

api/news: append full flag on FullPosts This patch appends the full flag on FullPosts.

view details

push time in a month

startedidank/explainshell

started time in a month

fork babygoat/click-to-deploy

Source for Google Click to Deploy solutions listed on Google Cloud Marketplace.

fork in a month

startedsudokar/generator-tf-module

started time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)

I omit the ok in purpose as the possible error cases are handled in the above switch cases.

As I proposed before, does there any case I miss?

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (nc *newsV2Controller) GetATopic(c *gin.Context) { 	c.JSON(http.StatusOK, gin.H{"status": "success", "data": topic}) } +type job struct {+	Name  string+	Type  string+	Query *news.Query+}++type result struct {+	Name    string+	Content interface{}+	Error   error+}++const (+	typePost  = "post"+	typeTopic = "topic"+)+ func (nc *newsV2Controller) GetIndexPage(c *gin.Context) {+	// TODO(babygoat): config context with proper timeout+	ctx := c++	jobs := nc.getIndexPageJobs()+	jobStream := nc.preparejobs(ctx, jobs)++	workers := make([]<-chan result, runtime.NumCPU())+	for i, _ := range workers {+		workers[i] = nc.fetchjobs(ctx, jobStream)+	}++	var err error+	results := make(map[string]interface{})++	defer func() {+		if err != nil {+			nc.helperCleanup(c, err)+		}+	}()+	for result := range nc.merge(ctx, workers...) {+		select {+		case <-ctx.Done():+			err = errors.WithStack(ctx.Err())+			break+		default:+			if result.Error != nil {+				err = result.Error+				break+			}+			results[result.Name] = result.Content+			if len(results) == len(jobs) {+				break+			}+		}+	}+	if err != nil {+		return+	}+	c.JSON(http.StatusOK, gin.H{"status": "success", "data": gin.H{"records": results}})++}++func (nc *newsV2Controller) getIndexPageJobs() []job {+	const statePublished = "published"++	// v1 index page section jobs+	jobs := []job{+		{+			Name:  news.LatestSection,+			Type:  typePost,+			Query: news.NewQuery(news.FilterState(statePublished), news.Limit(6)),

This is actually a idiomatic way to use called functional options pattern as promoted by one of the golang creator, Rob Pike, and Dave Cheney. The reason why this pattern is preferred to method chaining in golang is that other language like java or python can throw error in the middle of the chain. However, golang code tends to return error rather throw error in the middle. Since the chaining method requires the function to return receiver itself, there is no way to break in the middle of chain but to record error within the receiver object (e.g., gorm.db.Error in gorm).

In our case, the goal is to create an immutable object pointer(*Query). There is no such drawback from the aforementioned issue so the decision to use functional option is subjective and I believed this is more like an overloading constructor (func NewQuery(options func(*Query)) *Query). Chaining methods, instead, are setters which might be sort of contradict to our goal semantically.

Functional option do have some drawbacks (e.g., package api bloating/not easy to learn what to configure) as discussed in the google-cloud-go issue . I'm not sure if the problems you referred to the package level api are within these issues scope. In the former issue, I cannot foresee the requirements for adding more filters or sort but the current number (7) seems OK. In the latter one, I think I can define the type of func(*Query) (i.e, type Option func(*Query)) so that these functions will be grouped under Option in godoc. What's more, I can prefix the function with With to indicate that this is actually a option function. (e.g., func WithLimit(limit int) Option.

What do u think?

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)+		for i := 0; i < len(topics); i++ {+			topics[i].Full = true+		}+	}++	return topics, nil }-func (m *mongoStorage) GetMetaOfTopics(context.Context, *news.Query) ([]news.MetaOfTopic, error) {-	return nil, nil++func (m *mongoStorage) getFullTopics(ctx context.Context, stages []bson.D) <-chan fetchResult {+	result := make(chan fetchResult)+	go func(ctx context.Context, stages []bson.D) {+		defer close(result)+		cursor, err := m.Database(globals.Conf.DB.Mongo.DBname).Collection(news.ColTopics).Aggregate(ctx, stages)+		if err != nil {+			result <- fetchResult{Error: errors.WithStack(err)}+		}+		defer cursor.Close(ctx)++		var topics []news.Topic+		for cursor.Next(ctx) {+			var topic news.Topic+			err := cursor.Decode(&topic)+			if err != nil {+				result <- fetchResult{Error: errors.WithStack(err)}+			}+			topics = append(topics, topic)+		}

Good gotcha! I forgot the error handling here.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Features/news v2

 func (m *mongoStorage) getMetaOfPosts(ctx context.Context, stages []bson.D) <-ch 	return result } -func (m *mongoStorage) GetFullTopics(context.Context, *news.Query) ([]news.Topic, error) {-	return nil, nil+func (m *mongoStorage) GetFullTopics(ctx context.Context, q *news.Query) ([]news.Topic, error) {+	var topics []news.Topic++	mq := news.NewMongoQuery(q)++	// build aggregate stages from query+	stages := news.BuildQueryStatements(mq)+	// build lookup(join) stages according to required fields+	stages = append(stages, news.BuildLookupStatements(news.LookupFullTopic)...)++	select {+	case <-ctx.Done():+		return nil, errors.WithStack(ctx.Err())+	case result, ok := <-m.getFullTopics(ctx, stages):+		switch {+		case !ok:+			return nil, errors.WithStack(ctx.Err())+		case result.Error != nil:+			return nil, errors.WithStack(result.Error)+		}+		topics = result.Content.([]news.Topic)

I believed the switch statements above catches all the error cases AFAIK. case !ok catches the context timeout if the storage layer returned early. result.Error catches the operation errors through the api.

Was there any case I miss?

babygoat

comment created time in a month

startederikgrinaker/toydb

started time in a month

push eventbabygoat/environment-setup

babygoat

commit sha 84628ff687ef68bb1880b497b9cbd498e6ab9623

Merge pull request #3 from babygoat/master Initialize mysql with the event scheduler enabled

view details

dependabot[bot]

commit sha b21e8bc7b4b580be9a7f1f3aa86255fb19930cf8

Bump eslint-utils from 1.3.1 to 1.4.3 Bumps [eslint-utils](https://github.com/mysticatea/eslint-utils) from 1.3.1 to 1.4.3. - [Release notes](https://github.com/mysticatea/eslint-utils/releases) - [Commits](https://github.com/mysticatea/eslint-utils/compare/v1.3.1...v1.4.3) Signed-off-by: dependabot[bot] <support@github.com>

view details

dependabot[bot]

commit sha f0a487cd6fc5513f8210290659fea90ad5940621

Bump acorn from 6.1.1 to 6.4.1 Bumps [acorn](https://github.com/acornjs/acorn) from 6.1.1 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](https://github.com/acornjs/acorn/compare/6.1.1...6.4.1) Signed-off-by: dependabot[bot] <support@github.com>

view details

nick

commit sha 3cfd49bd6a3dbe74592749f3969926ab10c1bdce

Update README.md Fix missing command

view details

dependabot[bot]

commit sha f9311989fcf10ecfe3ef5d64acfc9098138d4d71

Bump lodash from 4.17.11 to 4.17.15 Bumps [lodash](https://github.com/lodash/lodash) from 4.17.11 to 4.17.15. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](https://github.com/lodash/lodash/compare/4.17.11...4.17.15) Signed-off-by: dependabot[bot] <support@github.com>

view details

dependabot[bot]

commit sha 11ac69e91a389b2d906cd4f04a311efdfb268094

Bump js-yaml from 3.13.0 to 3.14.0 Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 3.13.0 to 3.14.0. - [Release notes](https://github.com/nodeca/js-yaml/releases) - [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md) - [Commits](https://github.com/nodeca/js-yaml/compare/3.13.0...3.14.0) Signed-off-by: dependabot[bot] <support@github.com>

view details

nick

commit sha bd3ee7c87ed3f0497d8a811ea1b037d10231db6d

Merge pull request #8 from twreporter/dependabot/npm_and_yarn/lodash-4.17.15 Bump lodash from 4.17.11 to 4.17.15

view details

nick

commit sha fe25b2df567ba2b1bbe85ba3955ae804190bcf54

Merge pull request #9 from twreporter/dependabot/npm_and_yarn/js-yaml-3.14.0 Bump js-yaml from 3.13.0 to 3.14.0

view details

nick

commit sha fa16f0fd31f79c54955c4a55a0cf5ba2f0ac8bc3

Merge pull request #7 from twreporter/dependabot/npm_and_yarn/acorn-6.4.1 Bump acorn from 6.1.1 to 6.4.1

view details

nick

commit sha e7fba810d41aa884cbafcac52d513ae484d12ffb

Merge pull request #6 from twreporter/dependabot/npm_and_yarn/eslint-utils-1.4.3 Bump eslint-utils from 1.3.1 to 1.4.3

view details

Ching-Yang, Tseng

commit sha 3dc5572378df5bb94bf0f49549b207180c5f604c

Upgrade mongo version to 3.6.18 This patch upgrades the mongo version to the latest stable mongo server version 3.6.18. Also, bump the version to 1.0.0-beta8

view details

push time in a month

pull request commenttwreporter/go-api

Posts v2 api doc

Updated

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 404 (application/json)+    +    + Attributes+        + status: fail (required)+        + data (required)+            + slug: Cannot find the post from the slug (required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request with full=true+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullPost, required)++# Data Structures++## FullPost++ include MetaOfPost++ brief (paragraphs, required)++ content (paragraphs, required)++ copyright: copyrighted (required)++ designers (array[author], fixed-type, required)++ engineers (array[author], fixed-type, required)++ extend_byline: extended lines (required)++ leading_image_description: description (required)++ og_title: og title (required)++ photographers (array[author], fixed-type, required)++ relateds (array, fixed-type, required)+    + 5edf118c3e631f0600198935++ topic (required)+    + title: topic title (required)+    + `short_title`: topic short title (required)+    + slug: `the-slug-of-the-topic` (required)+    + relateds (array, fixed-type, required)+        + 5edf118c3e631f0600198935++ updated_at: `2020-06-8T16:00:00Z` (required)++ writters (array[author], fixed-type, required)

fix

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)+++ Response 204

update

babygoat

comment created time in a month

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 2637e1b114012f3c89d492da0d15402f6130d563

doc: return meta instead of empty content during post/topic list This patch replaces status no content with status ok with meta information(e.g., pagination) and empty record array.

view details

Ching-Yang, Tseng

commit sha f19d8a5ea6cc0fb71366f2731fb0c6e5ba1f49a4

api/news: replace field writters with writers This patch replaces the field `writters` with `writers` in full post data.

view details

push time in a month

pull request commenttwreporter/go-api

Features/news v2

@nickhsine you can start reviewing from commit 6bbdf17. Those pre-commits serve as the proof-of-concept of using the mongo-go-driver and performance testing but lack of full structure design.

babygoat

comment created time in a month

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 2eaf29c356fe17208b6ff985fc24c2ae975cfbdd

api/news: fix test fail from default query changed This patch fixes the test fail resulting from the default query change with state published.

view details

push time in a month

push eventbabygoat/go-api

nickhsine

commit sha 1be3f110dd60119f59d7ea84ec82cd080ddb5219

api/mail: update success donation email template Support both token donation success and prime donation success

view details

nickhsine

commit sha 83b7f799c8a224ae6a69d24bcffa1269a1cb79c4

doc: update CHANGELOG.md

view details

nick

commit sha 815b123a33b37de5bfce4fee072ad16839de6dd3

Merge pull request #377 from nickhsine/donation-email-temp refactor: update success donation email template

view details

Taylor Fang

commit sha 03da19a4cefb92d5bfaf7ba599c1a532ceaf462d

doc: update docs to add `receipt_header` field

view details

Taylor Fang

commit sha 9098549594a8ffb632f4eb07f959afadbcc0cb24

doc: update docs according to review comment This patch promote `receipt_header` to the top-level

view details

Taylor Fang

commit sha 90541811e753357c275ace3a627f8a6e3202aef2

doc: fix json format

view details

Tai-Jiun Fang

commit sha 6405beec1847d927fbb9c7604979c9e8666a76e0

Merge pull request #378 from taylrj/add-receipt-title doc: update docs to add `receipt_header` field

view details

Taylor Fang

commit sha 73dad896910642ad47a0e8129fcc11ea3a6c5880

chore: donations table schema change This patch updates schemas of table `pay_by_prime_donations`, `periodic_donations`, `pay_by_card_token_donations`, 'pay_by_other_method_donations' by adding `receipt_header` column. Add corresponding `000006_add_receipt_header.down.sql` and `000006_add_receipt_header.up.sql` for migration.

view details

Taylor Fang

commit sha 95830c578df9273185a82d6821c9d3dce91ad3e3

doc: update CHANGELOG.md

view details

Tai-Jiun Fang

commit sha a25b664fe854d2f1bb36d93cc8aaff79df85a783

Merge pull request #380 from taylrj/update-schema chore: donations table schema change

view details

nickhsine

commit sha 776be53d2b205ade960dd08043fb66bd7d9eb47e

chore: release v6.0.4

view details

nick

commit sha ae1ec86060ebb4699e81382f92d8fa424e8259b0

Merge pull request #383 from nickhsine/master chore: release v6.0.4

view details

nickhsine

commit sha 84c1831e0dc520f86b10b462a399fb3c16b77f68

chore: update CHANGELOG.md. remove Merge Pull commits

view details

nick

commit sha ac43df3781f8e533a36758ccf41d5d496e423fc8

Merge pull request #385 from nickhsine/master chore: update CHANGELOG.md. remove Merge Pull commits

view details

babygoat

commit sha 1e639dd6ceb4f094cd55aebd52785098195a6203

api/news: add new model for posts This patch adds the new model for post/topic page data.

view details

Ching-Yang, Tseng

commit sha 7ae9c6cf1c644331e39975e7c50c7c8da8889b7d

core: add mongo db connection with new driver

view details

Ching-Yang, Tseng

commit sha 93d10d520793a3a8ddeae916516e07e21266f60c

api/news: prototype v2 controller

view details

Ching-Yang, Tseng

commit sha f97a0bfdc87dab1570ebeed34b5a3dd367eee068

api/news: prototype the post/topic query model

view details

Ching-Yang, Tseng

commit sha d6329d0eea34ead3c8d49f04381b2a00bed7fc33

api/news: adjust storage interface w.r.t query

view details

Ching-Yang, Tseng

commit sha 82f430a535b4f214a4d6c2794a082d2e6c278359

api/news: add storage layer function signature This patch adds the function signatures of the storage layer. Also, implement the GetPosts function

view details

push time in a month

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)+++ Response 204

OK.

babygoat

comment created time in a month

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)

Sure. I will update the pagination format.

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 404 (application/json)+    +    + Attributes+        + status: fail (required)+        + data (required)+            + slug: Cannot find the post from the slug (required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request with full=true+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullPost, required)++# Data Structures++## FullPost++ include MetaOfPost++ brief (paragraphs, required)++ content (paragraphs, required)++ copyright: copyrighted (required)++ designers (array[author], fixed-type, required)++ engineers (array[author], fixed-type, required)++ extend_byline: extended lines (required)++ leading_image_description: description (required)++ og_title: og title (required)++ photographers (array[author], fixed-type, required)++ relateds (array, fixed-type, required)+    + 5edf118c3e631f0600198935++ topic (required)+    + title: topic title (required)+    + `short_title`: topic short title (required)+    + slug: `the-slug-of-the-topic` (required)+    + relateds (array, fixed-type, required)+        + 5edf118c3e631f0600198935++ updated_at: `2020-06-8T16:00:00Z` (required)++ writters (array[author], fixed-type, required)

I will update both codes and document in the later commits

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfTopic], fixed-type, required)+++ Response 204

see below.

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_at` - sort by updated_at ascending+            + `-updated_at` - sort by updated_at descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], fixed-type, required)+++ Response 204

No, as stated in the RFC standard. Response with 204 status code cannot have any message body.

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_date` - sort by updated_date ascending+            + `-updated_date` - sort by updated_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request+    + Parameters+        + full: true (boolean, optional)

done

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfTopic], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Topic [/v2/topics/{slug}{?full}]+Contain meta(brief) or full information of a topic with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-topic` (required) - Topic slug+    + full: `true` (optional) - Whether to retrieve a topic with full information+        + Default: `false`++### Get a single topic [GET]+Get a single topic with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfTopic, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullTopic, required)++# Data Structures++## FullTopic++ include MetaOfTopic++ relateds_background: in-row++ relateds_format: `#5E5E41`++ title_position: center++ lead_video (video, required)++ headline: topic headline (required)++ subtitle: topic subtitle (required)++ description (paragraphs, required)++ team_description (paragraphs, required)++ og_title: topic og title (required)++## MetaOfTopic++ id: 5edf118c3e631f0600198935 (required)++ slug: `a-slug-of-the-topic` (required)++ title: topic title (required)++ `short_title`: short title (required)++ published_date: 2020-06-8T16:00:00Z (required)++ og_description: topic description (required)++ og_image (image, required)++ leading_image (image, required)++ leading_image_portrait (image, required)++ relateds (array[string], required)

done

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_date` - sort by updated_date ascending+            + `-updated_date` - sort by updated_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullPost, required)++# Data Structures++## FullPost++ include MetaOfPost++ brief (paragraphs, required)++ content (paragraphs, required)++ copyright: copyrighted (required)++ designers (array[author], fixed-type)++ engineers (array[author], fixed-type)++ extend_byline: extended lines (required)++ leading_image_description: description (required)++ og_title: og title (required)++ photographers (array[author], fixed-type)++ relateds (array[string], required)++ topics

done

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_date` - sort by updated_date ascending+            + `-updated_date` - sort by updated_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+

done

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfTopic], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Topic [/v2/topics/{slug}{?full}]+Contain meta(brief) or full information of a topic with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-topic` (required) - Topic slug+    + full: `true` (optional) - Whether to retrieve a topic with full information+        + Default: `false`++### Get a single topic [GET]+Get a single topic with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfTopic, required)+

done

babygoat

comment created time in 2 months

pull request commenttwreporter/go-api

Posts v2 api doc

Updated.

babygoat

comment created time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 93af75242aed38975050cd1fa928cbaf00d589e0

doc: describe different response w.r.t parameter This patch specifies the full=true on the request description to differentiate the same response code with different response body with respect to the query parameter setup.

view details

Ching-Yang, Tseng

commit sha 7c7410760cc63ca96bc2871b51d1a6188b74fd4e

doc: add client side error for invalid slug This patch adds the client side error (404) during the single post/topic retrieval when the slug does not exist within the posts or topics collection.

view details

Ching-Yang, Tseng

commit sha e9f7f769345a0d89d457d07dfc9e4bb3ba8ed63c

doc: give relateds sample value instead of default This patch gives the relateds field sample value instead of using aglio string default value `hello, world` to approach the object string data stype.

view details

Ching-Yang, Tseng

commit sha 7f177d2606ec80c2aec7d19e973ddce84d9be589

doc: fix nested array field schema type missing This patch fixs schema of nested array fields which should be both required and fixed-type(inherit the object type).

view details

Ching-Yang, Tseng

commit sha 8d2ba673bfd88997a04d19168d2121e300382892

doc: singularize topics field This patch renames the `topics` field of a single post to `topic`. Also, add the topic field as the required field in the full post response.

view details

Ching-Yang, Tseng

commit sha aeb4a20d8c6d7132b47a1dd9df2f5b5883743507

doc: add missing required fields This patch adds the updated_at/published_date fields to required fields so that the schema of the topic/post fit the requirement. Also, rename the query parameter naming of updated_date back to updated_at.

view details

push time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Topics++## Topic List [/topics{?sort,offset,limit}]+A list contains meta(brief) information of the selected topics.++## Get a list of topics [GET]+++ Parameters+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfTopic], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Topic [/v2/topics/{slug}{?full}]+Contain meta(brief) or full information of a topic with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-topic` (required) - Topic slug+    + full: `true` (optional) - Whether to retrieve a topic with full information+        + Default: `false`++### Get a single topic [GET]+Get a single topic with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfTopic, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request+    + Parameters+        + full: true (boolean, optional)+++ Response 200 (application/json)++    + Body++    + Attributes+        + status: success (required)+        + data (FullTopic, required)++# Data Structures++## FullTopic++ include MetaOfTopic++ relateds_background: in-row++ relateds_format: `#5E5E41`++ title_position: center++ lead_video (video, required)++ headline: topic headline (required)++ subtitle: topic subtitle (required)++ description (paragraphs, required)++ team_description (paragraphs, required)++ og_title: topic og title (required)++## MetaOfTopic++ id: 5edf118c3e631f0600198935 (required)++ slug: `a-slug-of-the-topic` (required)++ title: topic title (required)++ `short_title`: short title (required)++ published_date: 2020-06-8T16:00:00Z (required)++ og_description: topic description (required)++ og_image (image, required)++ leading_image (image, required)++ leading_image_portrait (image, required)++ relateds (array[string], required)

Sure. It's string default

babygoat

comment created time in 2 months

Pull request review commenttwreporter/go-api

Posts v2 api doc

+# Group Posts++## Post List [/v2/posts{?category_id,tag_id,id,sort,offset,limit}]+A list contains meta(brief) information of the selected posts.++## Get a list of posts [GET]+++ Parameters+    + `category_id`: `5edf118c3e631f0600198935` (optional) - Search for posts of the categories referenced by the category_id+    + `tag_id`: `5edf118c3e631f0600198935` (optional) - Search for posts with the tags referenced by the tag ids+    + id: `5edf118c3e631f0600198935` (optional) - Search for posts with the referenced ids+    + sort: `-published_date` (optional) - which field to sort by+        + Default: `-published_date`+        + Members+            + `published_date` - sort by published_date ascending+            + `-published_date` - sort by published_date descending+            + `updated_date` - sort by updated_date ascending+            + `-updated_date` - sort by updated_date descending+    + offset: `0` (integer, optional) - The number of posts to skip+        + Default: `0`+    + limit: `10` (integer, optional) - The maximum number of posts to return+        + Default: `10`+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (array[MetaOfPost], required)+++ Response 204+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)++## Post [/v2/posts/{slug}{?full}]+A post contains meta(brief) or full information of a post with the slug specified.+++ Parameters+    + slug: `a-slug-of-a-post` (required) - Post slug+    + full: `true` (optional) - Whether to retrieve a post with full information+        + Default: `false`++## Get a single post [GET]+Get a single post with the given slug+++ Response 200 (application/json)++    + Attributes+        + status: success (required)+        + data (MetaOfPost, required)+++ Response 500 (application/json)++    + Attributes+        + status: error (required)+        + message: Unexpected error. (required)+++ Response 504 (application/json)++    + Attributes+        + status: error (required)+        + message: Query upstream server timeout. (required)+++ Request+    + Parameters+        + full: true (boolean, optional)

Aglio does not support this feature to render different query parameter requests but the core parser (drafter) for mson did. (However, different request body works but I didn't want to take this too far to change the request type.) AFAIK, the only way to workaround this is to use the request description. How do you think? 螢幕快照 2020-06-19 上午11 05 20

babygoat

comment created time in 2 months

pull request commenttwreporter/go-api

Posts v2 api doc

Update according to the discussion in spreadsheet(https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1659711178).

babygoat

comment created time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha f6aba810246cee78d04f9e06b7feb91e098c54f5

doc: add /v2/index_page endpoint This patch adds the endpoint document for /v2/index_page.

view details

Ching-Yang, Tseng

commit sha dcd8975adba21eaad4dba69093f9395d595cb79f

doc: adjust /v2/posts endpoints This patch adjusts /v2/posts & /v2/posts/:slug endpoints response according to the payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1523711569)

view details

Ching-Yang, Tseng

commit sha 1590853f052c397777ca3057c9dd602047b27951

doc: adjust /v2/topics endpoints This patch adjust /v2/topics and /v2/topics/:slug endponts with respect to the response payload discussion spreadsheet. (https://docs.google.com/spreadsheets/d/1qNuS2urtRuiu0-s41NF22_YT-eTCMgE2ag9lquuo3hs/edit?pli=1#gid=1659711178)

view details

push time in 2 months

pull request commenttwreporter/go-api

Posts v2 api doc

Another view whether to adopt the /v2/index_page endpoint

Pros: use redis cache for the whole page Cons: cache ttl might result in delayed new post appearance

babygoat

comment created time in 2 months

PR opened twreporter/go-api

Reviewers
Posts v2 api doc

Adds api documents for following endpoints according to spreadsheet

  • [x] /v2/posts/:slug
  • [x] /v2/posts
  • [x] /v2/topics/:slug
  • [x] /v2/topics

Also, after the index_page & index_page response shrinking, do we still need these endpoints? IMO, frontend can send the batch queries to backend posts/topics endpoints with corresponding query parameters.

Pros: each payloads are small. Cons: Increase both (BE/FE) server file I/O (socket connections)

+3290 -2

0 comment

5 changed files

pr created time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 9ffbfa32bcef9895f2cc667ada8a2638d15e5150

doc: generate result document This patch generates the final html file for api document

view details

push time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha c5729ab46903dffb14d4e2f189465438971dbde4

doc: adds v2 endpoint of getting a post This patch adds the api document of describing the v2 endpoint (/v2/posts/:slug).

view details

Ching-Yang, Tseng

commit sha 0bb1f95d8d9420240a21a81ee7eeb403e2b6fad8

doc: add v2 endpoint for listing posts This patch adds the v2 endpoint document for listing posts according to following options Filter * category_id * tag_id * id Pagination * offset * limit Sort * updated_at * published_date

view details

Ching-Yang, Tseng

commit sha 5f876e9fa075447fc628e52a7e24ffb729db93a3

doc: add the v2 endpoint to get a single topic This patc adds the endopint specification of the v2 single topic infomration retrieval. Also, move the common asset data structure into asset.apib.

view details

Ching-Yang, Tseng

commit sha c74559fa7463cac0a2a48123bdd7a447f86aa1a4

doc: add the v2 endpoint of listing topic This patch adds the v2 endpoint of listing topic with respect ot following options. Pagination * offset * limit Sort * sort

view details

push time in 2 months

create barnchbabygoat/go-api

branch : posts-v2-api-doc

created branch time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

OK, then I can keep the field.

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

What is the exact use case? Indicate the response is ready? Or response is full article?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

What is the exact use case? Indicate the response is ready? Or response is full article?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

What is the exact use case? Indicate the response is ready? Or response is full article?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

I suppose the point is that the field is used directly as the indicator. From the perspective of api response, the full field only indicates the response contains full post but not whether the response ready. It seemed to me that whether the response is ready or not should be obtained from other indicator like http response code or the isFetching.

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ *+ *  @property {Object[]} description.api_data+ *  @property {Object[]} team_description.api_data+ *  @property {Object} description+ *  @property {Object} leading_video+ *  @property {Object} team_description+ *  @property {ObjectID[]} relateds - slugs of related posts

The latter one is what I referred.

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ *+ *  @property {Object[]} description.api_data+ *  @property {Object[]} team_description.api_data+ *  @property {Object} description+ *  @property {Object} leading_video+ *  @property {Object} team_description+ *  @property {ObjectID[]} relateds - slugs of related posts+ *  @property {string} headline+ *  @property {string} og_title+ *  @property {string} published_date+ *  @property {string} relateds_background+ *  @property {string} relateds_format+ *  @property {string} subtitle+ *  @property {string} title_position+ */++/**+ *  FullPost type definition+ *  @typedef {Object} FullPost+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug

Sorry. False alarm

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

Why should we use this field as indicators instead of other fields in full post/topic only?

nickhsine

comment created time in 2 months

pull request commenttwreporter/go-api

Features/news v2

Listing tests

posts (~2600)

  • v1: ~180ms
  • v2: ~50ms (with createIndex on publishedDate)

topics

  • v1: ~70ms
  • v2: ~10ms

Note that the createIndex on the dataset will drastically improve the operations performance (match/sort). This should be done along with the cluster migration.

babygoat

comment created time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha 6de7d2bfe127a2eb3555ff2398b59deac877c2ba

api/news: implement posts/topics list This patch implements the posts/topics list in v2 endpoint. Note that the index on posts collection should be added in the production database as the descendant order.

view details

Ching-Yang, Tseng

commit sha c9f4edea8dbcb6a2b34d3c4ac4ac940661a6044e

api/news: remove filter during index page fetch This patch removes the match filter which restricts the doucment published within a week during the fetch of latest post section. The performance improvments should be operated on collections index.

view details

Ching-Yang, Tseng

commit sha e95249b92c26b7c8c0740d61e49cbd871176921e

api/news: implement posts query filter This patch implments the filters parsed from the query parameter(category, tag, id). Note that this is a breaking change which adapted from v1 which only accepts json string to be parsed as the mongo query document directly. However, previous implementation is not intuitive in url context and storage dependent. This patch changes value to use the specific parameters and transform the parameters into mongo query respectively.

view details

push time in 2 months

starteddavecheney/high-performance-go-workshop

started time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha c1eb151bf5f194dec39137b60f2bd03b8f7fda95

api/news: implement index page fetch This patch adapted from v1 query to implement index page fetch. Also, fix the typo of latest topic section. Note that during the latest section fetch, specify a time range filter(within a week) to further reduce the posts sorting effort.

view details

push time in 2 months

pull request commenttwreporter/twreporter-npm-packages

Redux refactoring

To summarize the response requirement, we could still leverage the meta/full fetch.

Post

post.id
post.style 
post.slug 
post.leading_image_portrait
post.hero_image 
post.og_image
post.og_description
post.title
post.subtitle
post.categories
post.published_date

Full

post.brief
post.categories
post.content
post.copyright
post.designers
post.engineers 
post.extend_byline
post.hero_image 
post.id
post.is_external
post.leading_image_portrait
post.leading_image_description
post.og_description
post.og_image
post.og_title
post.photographers
post.published_date
post.relateds([]ObjectID)
post.slug
post.style
post.subtitle
post.title
post.topics(topic_name, slug,  relateds(objectID))
post.updated_at
post.writters

Topic

meta

topic.id 
topic.leading_image
topic.og_description
topic.og_image 
topic.slug 
topic.title
topic.topic_name
topic.relateds ([]objectID)

Full

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ *+ *  @property {Object[]} description.api_data+ *  @property {Object[]} team_description.api_data+ *  @property {Object} description+ *  @property {Object} leading_video+ *  @property {Object} team_description+ *  @property {ObjectID[]} relateds - slugs of related posts

objectID

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url

Do we need the original image info?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order

Where is the sort_order used?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright

Where does the property be used?

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ *+ *  @property {Object[]} description.api_data+ *  @property {Object[]} team_description.api_data+ *  @property {Object} description+ *  @property {Object} leading_video+ *  @property {Object} team_description+ *  @property {ObjectID[]} relateds - slugs of related posts+ *  @property {string} headline+ *  @property {string} og_title+ *  @property {string} published_date+ *  @property {string} relateds_background+ *  @property {string} relateds_format+ *  @property {string} subtitle+ *  @property {string} title_position+ */++/**+ *  FullPost type definition+ *  @typedef {Object} FullPost+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug

Same above.

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} title+ *  @property {string} topic_name+ *  @property {string} updated_at+ */++/**+ *  FullTopic type definition+ *  @typedef {Object} FullTopic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

Same above

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full+ *  @property {bool} is_external+ *  @property {string} id+ *  @property {string} og_description+ *  @property {string} published_date+ *  @property {string} style+ *  @property {string} subtitle+ *  @property {string} title+ */++/**+ *  Topic type definition+ *  @typedef {Object} Topic+ *  @property {Image} leading_image+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {bool} full

Same above.

nickhsine

comment created time in 2 months

Pull request review commenttwreporter/twreporter-npm-packages

Redux refactoring

 const _ = {   merge, } +/**+ *  Mongodb ObjectID type definition+ *  @typedef {string} ObjectID+ */++/**+ *  Slug type defintion+ *  @typedef {string} Slug+ */++/**+ * ResizedTarget type definition+ * @typedef {Object} ResizedTarget+ * @property {string} url+ * @property {number} height+ * @property {number} width+ */++/**+ *  Image type definition+ *  @typedef {Object} Image+ *  @property {string} id+ *  @property {string} description+ *  @property {string} copyright+ *  @property {string} filetype+ *  @property {number} height+ *  @property {number} width+ *  @property {string} url+ *  @property {Object} resized_targets+ *  @property {ResizedTarget} resized_targets.tiny+ *  @property {ResizedTarget} resized_targets.w400+ *  @property {ResizedTarget} resized_targets.mobile+ *  @property {ResizedTarget} resized_targets.tablet+ *  @property {ResizedTarget} resized_targets.desktop+ */++/**+ *  Tag type definition+ *  @typedef {Object} Tag+ *  @property {string} id+ *  @property {string} name+ */++/**+ *  Category type definition+ *  @typedef {Object} Category+ *  @property {string} id+ *  @property {string} name+ *  @property {number} sort_order+ */++/**+ *  Author type definition+ *  @typedef {Object} Author+ *  @property {string} id+ *  @property {string} name+ *  @property {string} job_title+ */++/**+ *  Post type definition+ *  @typedef {Object} Post+ *  @property {Category[]} categories+ *  @property {Image} hero_image+ *  @property {Image} leading_image_portrait+ *  @property {Image} og_image+ *  @property {Slug} slug+ *  @property {Tag[]} tags+ *  @property {bool} full

Do we need full property?

nickhsine

comment created time in 2 months

pull request commenttwreporter/go-api

Features/news v2

More performance tests

index_page on 6/05

  • v1: ~1.2 s
  • v2 ~0.6 s

loadtesting on the 6/05 dataset

  • v1
Requests      [total, rate, throughput]         1500, 50.03, 5.54
Duration      [total, attack, wait]             37.707s, 29.981s, 7.726s
Latencies     [min, mean, 50, 90, 95, 99, max]  92.944µs, 4.522s, 5.317s, 5.976s, 7.138s, 22.863s, 29.94s
Bytes In      [total, mean]                     371769, 247.85
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           13.93%
Status Codes  [code:count]                      0:328  200:209  500:963
  • v2
Requests      [total, rate, throughput]         1500, 50.04, 12.76
Duration      [total, attack, wait]             40.69s, 29.977s, 10.712s
Latencies     [min, mean, 50, 90, 95, 99, max]  106.257µs, 3.779s, 1.053s, 11.161s, 13.702s, 17.225s, 19.54s
Bytes In      [total, mean]                     4170840, 2780.56
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           34.60%
Status Codes  [code:count]                      0:285  200:519  500:696

Note that the success rate improves from (14% -> 35%). The major bottleneck of the local load testing is the limited file descriptors (i.e., socket, see ulimit -n) which greatly affects the response time.

babygoat

comment created time in 2 months

push eventbabygoat/go-api

Ching-Yang, Tseng

commit sha d59bf01523462085410e4d889e17030c500b9696

api/news: implement index page fetch This patch adapted from v1 query to implement index page fetch. Also, fix the typo of latest topic section. Note that during the latest section fetch, specify a time range filter(within a week) to further reduce the posts sorting effort.

view details

push time in 2 months

more