Large refactoring projects: OpenIM automation, scripting, and openimctl refactoring (#825)

* fix: fix bin tools path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: fix golang release file path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: fix golang release file path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: fix scripts and optimize

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: fix scripts path module

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: sync script code

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add lib and start scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* ci: add copyright scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* ci: add go-docs file and  copyright scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add scripts cross ower

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: Formatting code make lint path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: Formatting code make lint path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: Formatting code make lint path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: Formatting code make lint path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: chat scripts path bug

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: channge smail images

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add makefile feature

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add config and images log

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: Migrate directory to remove docker to images

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* style: formatting style Code

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: set opneim's bash logs

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: option scripts and docs

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add git cherry

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add git cherry

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save all bash and docs labels

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: scripts feature extend

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add config path config

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add config path config

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add feat scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add save scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add save scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add sctips help

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add start sctips help

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save scripts file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save all file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add openim server template file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add alot of system design

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save all file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save all file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add env config options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more robot details

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more module explain

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add scripts environment details design

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add openim msgtransfer scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add openim msgtransfer scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more design scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add file save

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add file save

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add rpc build and start

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add rpc build and start

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add rpc build and start

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save all images file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add scripts set

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add test options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: fix config path file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: update config file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: update config file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: update config file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: update config file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add readme docs

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save build scripts

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add all actions file

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add chat scripts name

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add all compose

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: commit tag

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save server code

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add docker compose fix

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add docker compose fix

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add docker compose fix

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: save server code

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: optimize dockerfile option

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: optimize dockerfile option

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: optimize dockerfile option

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add all options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add all options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more scrips

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add more options

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add config path

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: Add some optimizations

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* fix: Add some optimizations

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* Delete go.work.sum

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* Update .env

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add docker compose fix

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: add docker compose fix

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

* feat: delele go work sum

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>

---------

Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>
This commit is contained in:
Xinwei Xiong
2023-08-23 09:09:51 +08:00
committed by GitHub
parent ff9b258229
commit 58f591e5f6
255 changed files with 12457 additions and 4111 deletions

15
test/README.md Normal file
View File

@@ -0,0 +1,15 @@
## Run the Tests
To run a single test or set of tests, you'll need the [Ginkgo](https://github.com/onsi/ginkgo) tool installed on your
machine:
```console
go install github.com/onsi/ginkgo/ginkgo@latest
```
```shell
ginkgo --help
--focus value
If set, ginkgo will only run specs that match this regular expression. Can be specified multiple times, values are ORed.
```

48
test/jwt/main.go Normal file
View File

@@ -0,0 +1,48 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"fmt"
"github.com/golang-jwt/jwt/v4"
)
func main() {
rawJWT := `eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJpYW0uYXV0aHoubWFybW90ZWR1LmNvbSIsImV4cCI6MTYwNDEyODQwMywiaWF0IjoxNjA0MTI4NDAyLCJpc3MiOiJpYW1jdGwiLCJraWQiOiJpZDEifQ.Itr5u4C-nTeA01qbjjl7RzuPD-aSQazsJZY_Z25aGnI`
// Verify the token
claims := &jwt.MapClaims{}
parsedT, err := jwt.ParseWithClaims(rawJWT, claims, func(token *jwt.Token) (interface{}, error) {
// Validate the alg is HMAC signature
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
}
if kid, ok := token.Header["kid"].(string); ok {
fmt.Println("kid", kid)
}
return []byte("key1"), nil
})
if err != nil || !parsedT.Valid {
fmt.Println("token valid failed", err)
return
}
fmt.Println("ok")
}

64
test/testdata/README.md vendored Normal file
View File

@@ -0,0 +1,64 @@
# Test Data for OpenIM Server
This directory (`testdata`) contains various JSON formatted data files that are used for testing the OpenIM Server.
## Structure
```bash
testdata/
├── README.md # 描述该目录下各子目录和文件的作用
├── db/ # 存储模拟的数据库数据
│ ├── users.json # 用户的模拟数据
│ └── messages.json # 消息的模拟数据
├── requests/ # 存储模拟的请求数据
│ ├── login.json # 模拟登陆请求
│ ├── register.json # 模拟注册请求
│ └── sendMessage.json # 模拟发送消息请求
└── responses/ # 存储模拟的响应数据
├── login.json # 模拟登陆响应
├── register.json # 模拟注册响应
└── sendMessage.json # 模拟发送消息响应
```
Here is an overview of what each subdirectory or file represents:
- `db/` - This directory contains mock data mimicking the actual database contents.
- `users.json` - Represents a list of users in the system. Each entry contains user-specific information such as user ID, username, password hash, etc.
- `messages.json` - Contains a list of messages exchanged between users. Each message entry includes the sender's and receiver's user IDs, message content, timestamp, etc.
- `requests/` - This directory contains mock requests that a client might send to the server.
- `login.json` - Represents a user login request. It includes fields such as username and password.
- `register.json` - Mimics a user registration request. Contains details such as username, password, email, etc.
- `sendMessage.json` - Simulates a message sending request from a user to another user.
- `responses/` - This directory holds the expected server responses for the respective requests.
- `login.json` - Represents a successful login response from the server. It typically includes a session token and user-specific information.
- `register.json` - Simulates a successful registration response from the server, usually containing the new user's ID, username, etc.
- `sendMessage.json` - Depicts a successful message sending response from the server, confirming the delivery of the message.
## JSON Format
All the data files in this directory are in JSON format. JSON (JavaScript Object Notation) is a lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate.
Here is a simple example of what a JSON file might look like:
```bash
"users": [
{
"id": 1,
"username": "user1",
"password": "password1"
},
{
"id": 2,
"username": "user2",
"password": "password2"
}
]
```
In this example, "users" is an array of user objects. Each user object has an "id", "username", and "password".

0
test/testdata/db/messages.json vendored Normal file
View File

0
test/testdata/db/users.json vendored Normal file
View File

0
test/testdata/requests/login.json vendored Normal file
View File

0
test/testdata/requests/register.json vendored Normal file
View File

View File

0
test/testdata/responses/login.json vendored Normal file
View File

0
test/testdata/responses/register.json vendored Normal file
View File

View File

27
test/typecheck/README.md Normal file
View File

@@ -0,0 +1,27 @@
# OpenIM Typecheck
OpenIM Typecheck 为所有 Go 构建平台进行跨平台源代码类型检查。
## 优点
- **速度**OpenIM 完整编译大约需要 3 分钟,而使用 Typecheck 只需数秒。
- **资源消耗**:与需要 >40GB 的 RAM 不同Typecheck 只需 <8GB RAM
## 实现
OpenIM Typecheck 使用 Go 内置的解析和类型检查库 (`go/parser` `go/types`)。然而这些库并不是 go 编译器所使用的偶尔会出现不匹配的情况但总的来说它们是相当接近的
## 错误处理
如果错误不会阻止构建可以忽略
**`go/types` 报告的错误 `go build` 不会**
- **真正的错误**根据规范
- 应尽量修复如果无法修复或正在进行中例如已被外部引用的代码则可以忽略
- 例如闭包中的未使用变量
- **不真实的错误**
- 应忽略并在适当的情况下向上游报告
- 例如staging generated 类型之间的类型检查不匹配
**`go build` 报告的错误但我们不会**
- CGo 错误包括语法和链接器错误

10
test/typecheck/go.mod Normal file
View File

@@ -0,0 +1,10 @@
module github.com/OpenIMSDK/Open-IM-Server/test/typecheck
go 1.20
require golang.org/x/tools v0.12.0
require (
golang.org/x/mod v0.12.0 // indirect
golang.org/x/sys v0.11.0 // indirect
)

7
test/typecheck/go.sum Normal file
View File

@@ -0,0 +1,7 @@
golang.org/x/mod v0.12.0 h1:rmsUpXtvNzj340zd98LZ4KntptpfRHwpFOHG188oHXc=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sys v0.11.0 h1:eG7RXZHdqOJ1i+0lgLgCpSXAp6M3LYlAo6osgSi0xOM=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/tools v0.12.0 h1:YW6HUoUmYBpwSgyaGaZq1fHjrBjX1rlpZ54T6mu2kss=
golang.org/x/tools v0.12.0/go.mod h1:Sc0INKfu04TlqNoRA1hgpFZbhYXHPr4V5DzpSBTPqQM=

320
test/typecheck/typecheck.go Normal file
View File

@@ -0,0 +1,320 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// do a fast type check of kubernetes code, for all platforms.
package main
import (
"flag"
"fmt"
"io"
"log"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"golang.org/x/tools/go/packages"
)
var (
verbose = flag.Bool("verbose", false, "print more information")
cross = flag.Bool("cross", true, "build for all platforms")
platforms = flag.String("platform", "", "comma-separated list of platforms to typecheck")
timings = flag.Bool("time", false, "output times taken for each phase")
defuses = flag.Bool("defuse", false, "output defs/uses")
serial = flag.Bool("serial", false, "don't type check platforms in parallel (equivalent to --parallel=1)")
parallel = flag.Int("parallel", 2, "limits how many platforms can be checked in parallel. 0 means no limit.")
skipTest = flag.Bool("skip-test", false, "don't type check test code")
tags = flag.String("tags", "", "comma-separated list of build tags to apply in addition to go's defaults")
ignoreDirs = flag.String("ignore-dirs", "", "comma-separated list of directories to ignore in addition to the default hardcoded list including staging, vendor, and hidden dirs")
// When processed in order, windows and darwin are early to make
// interesting OS-based errors happen earlier.
crossPlatforms = []string{
"linux/amd64", "windows/386",
"darwin/amd64", "darwin/arm64",
"linux/arm", "linux/386",
"windows/amd64", "linux/arm64",
"linux/ppc64le", "linux/s390x",
"windows/arm64",
}
// directories we always ignore
standardIgnoreDirs = []string{
// Staging code is symlinked from vendor/k8s.io, and uses import
// paths as if it were inside of vendor/. It fails typechecking
// inside of staging/, but works when typechecked as part of vendor/.
"staging",
// OS-specific vendor code tends to be imported by OS-specific
// packages. We recursively typecheck imported vendored packages for
// each OS, but don't typecheck everything for every OS.
"vendor",
"_output",
// This is a weird one. /testdata/ is *mostly* ignored by Go,
// and this translates to kubernetes/vendor not working.
// edit/record.go doesn't compile without gopkg.in/yaml.v2
// in $GOSRC/$GOROOT (both typecheck and the shell script).
"pkg/kubectl/cmd/testdata/edit",
// Tools we use for maintaining the code base but not necessarily
// ship as part of the release
"hack/tools",
}
)
func newConfig(platform string) *packages.Config {
platSplit := strings.Split(platform, "/")
goos, goarch := platSplit[0], platSplit[1]
mode := packages.NeedName | packages.NeedFiles | packages.NeedTypes | packages.NeedSyntax | packages.NeedDeps | packages.NeedImports | packages.NeedModule
if *defuses {
mode = mode | packages.NeedTypesInfo
}
env := append(os.Environ(),
"CGO_ENABLED=1",
fmt.Sprintf("GOOS=%s", goos),
fmt.Sprintf("GOARCH=%s", goarch))
tagstr := "selinux"
if *tags != "" {
tagstr = tagstr + "," + *tags
}
flags := []string{"-tags", tagstr}
return &packages.Config{
Mode: mode,
Env: env,
BuildFlags: flags,
Tests: !(*skipTest),
}
}
type collector struct {
dirs []string
ignoreDirs []string
}
func newCollector(ignoreDirs string) collector {
c := collector{
ignoreDirs: append([]string(nil), standardIgnoreDirs...),
}
if ignoreDirs != "" {
c.ignoreDirs = append(c.ignoreDirs, strings.Split(ignoreDirs, ",")...)
}
return c
}
func (c *collector) walk(roots []string) error {
for _, root := range roots {
err := filepath.Walk(root, c.handlePath)
if err != nil {
return err
}
}
sort.Strings(c.dirs)
return nil
}
// handlePath walks the filesystem recursively, collecting directories,
// ignoring some unneeded directories (hidden/vendored) that are handled
// specially later.
func (c *collector) handlePath(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
name := info.Name()
// Ignore hidden directories (.git, .cache, etc)
if (len(name) > 1 && (name[0] == '.' || name[0] == '_')) || name == "testdata" {
if *verbose {
fmt.Printf("DBG: skipping dir %s\n", path)
}
return filepath.SkipDir
}
for _, dir := range c.ignoreDirs {
if path == dir {
if *verbose {
fmt.Printf("DBG: ignoring dir %s\n", path)
}
return filepath.SkipDir
}
}
// Make dirs into relative pkg names.
// NOTE: can't use filepath.Join because it elides the leading "./"
pkg := path
if !strings.HasPrefix(pkg, "./") {
pkg = "./" + pkg
}
c.dirs = append(c.dirs, pkg)
if *verbose {
fmt.Printf("DBG: added dir %s\n", path)
}
}
return nil
}
func (c *collector) verify(plat string) ([]string, error) {
errors := []packages.Error{}
start := time.Now()
config := newConfig(plat)
rootPkgs, err := packages.Load(config, c.dirs...)
if err != nil {
return nil, err
}
// Recursively import all deps and flatten to one list.
allMap := map[string]*packages.Package{}
for _, pkg := range rootPkgs {
if *verbose {
serialFprintf(os.Stdout, "pkg %q has %d GoFiles\n", pkg.PkgPath, len(pkg.GoFiles))
}
allMap[pkg.PkgPath] = pkg
if len(pkg.Imports) > 0 {
for _, imp := range pkg.Imports {
if *verbose {
serialFprintf(os.Stdout, "pkg %q imports %q\n", pkg.PkgPath, imp.PkgPath)
}
allMap[imp.PkgPath] = imp
}
}
}
keys := make([]string, 0, len(allMap))
for k := range allMap {
keys = append(keys, k)
}
sort.Strings(keys)
allList := make([]*packages.Package, 0, len(keys))
for _, k := range keys {
allList = append(allList, allMap[k])
}
for _, pkg := range allList {
if len(pkg.GoFiles) > 0 {
if len(pkg.Errors) > 0 && (pkg.PkgPath == "main" || strings.Contains(pkg.PkgPath, ".")) {
errors = append(errors, pkg.Errors...)
}
}
if *defuses {
for id, obj := range pkg.TypesInfo.Defs {
serialFprintf(os.Stdout, "%s: %q defines %v\n",
pkg.Fset.Position(id.Pos()), id.Name, obj)
}
for id, obj := range pkg.TypesInfo.Uses {
serialFprintf(os.Stdout, "%s: %q uses %v\n",
pkg.Fset.Position(id.Pos()), id.Name, obj)
}
}
}
if *timings {
serialFprintf(os.Stdout, "%s took %.1fs\n", plat, time.Since(start).Seconds())
}
return dedup(errors), nil
}
func dedup(errors []packages.Error) []string {
ret := []string{}
m := map[string]bool{}
for _, e := range errors {
es := e.Error()
if !m[es] {
ret = append(ret, es)
m[es] = true
}
}
return ret
}
var outMu sync.Mutex
func serialFprintf(w io.Writer, format string, a ...interface{}) (n int, err error) {
outMu.Lock()
defer outMu.Unlock()
return fmt.Fprintf(w, format, a...)
}
func main() {
flag.Parse()
args := flag.Args()
if *verbose {
*serial = true // to avoid confusing interleaved logs
}
if len(args) == 0 {
args = append(args, ".")
}
c := newCollector(*ignoreDirs)
if err := c.walk(args); err != nil {
log.Fatalf("Error walking: %v", err)
}
plats := crossPlatforms[:]
if *platforms != "" {
plats = strings.Split(*platforms, ",")
} else if !*cross {
plats = plats[:1]
}
var wg sync.WaitGroup
var failMu sync.Mutex
failed := false
if *serial {
*parallel = 1
} else if *parallel == 0 {
*parallel = len(plats)
}
throttle := make(chan int, *parallel)
for _, plat := range plats {
wg.Add(1)
go func(plat string) {
// block until there's room for this task
throttle <- 1
defer func() {
// indicate this task is done
<-throttle
}()
f := false
serialFprintf(os.Stdout, "type-checking %s\n", plat)
errors, err := c.verify(plat)
if err != nil {
serialFprintf(os.Stderr, "ERROR(%s): failed to verify: %v\n", plat, err)
f = true
} else if len(errors) > 0 {
for _, e := range errors {
// Special case CGo errors which may depend on headers we
// don't have.
if !strings.HasSuffix(e, "could not import C (no metadata for C)") {
f = true
serialFprintf(os.Stderr, "ERROR(%s): %s\n", plat, e)
}
}
}
failMu.Lock()
failed = failed || f
failMu.Unlock()
wg.Done()
}(plat)
}
wg.Wait()
if failed {
os.Exit(1)
}
}

View File

@@ -0,0 +1,121 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"errors"
"flag"
"os"
"path/filepath"
"testing"
"golang.org/x/tools/go/packages"
)
// This exists because `go` is not always in the PATH when running CI.
var goBinary = flag.String("go", "", "path to a `go` binary")
func TestVerify(t *testing.T) {
// x/tools/packages is going to literally exec `go`, so it needs some
// setup.
setEnvVars(t)
tcs := []struct {
path string
expect int
}{
{"./testdata/good", 0},
{"./testdata/bad", 18},
}
for _, tc := range tcs {
c := newCollector("")
if err := c.walk([]string{tc.path}); err != nil {
t.Fatalf("error walking %s: %v", tc.path, err)
}
errs, err := c.verify("linux/amd64")
if err != nil {
t.Errorf("unexpected error: %v", err)
} else if len(errs) != tc.expect {
t.Errorf("Expected %d errors, got %d: %v", tc.expect, len(errs), errs)
}
}
}
func setEnvVars(t testing.TB) {
t.Helper()
if *goBinary != "" {
newPath := filepath.Dir(*goBinary)
curPath := os.Getenv("PATH")
if curPath != "" {
newPath = newPath + ":" + curPath
}
t.Setenv("PATH", newPath)
}
if os.Getenv("HOME") == "" {
t.Setenv("HOME", "/tmp")
}
}
func TestHandlePath(t *testing.T) {
c := collector{
ignoreDirs: standardIgnoreDirs,
}
e := errors.New("ex")
i, _ := os.Stat(".") // i.IsDir() == true
if c.handlePath("foo", nil, e) != e {
t.Error("handlePath not returning errors")
}
if c.handlePath("vendor", i, nil) != filepath.SkipDir {
t.Error("should skip vendor")
}
}
func TestDedup(t *testing.T) {
testcases := []struct {
input []packages.Error
expected int
}{{
input: nil,
expected: 0,
}, {
input: []packages.Error{
{Pos: "file:7", Msg: "message", Kind: packages.ParseError},
},
expected: 1,
}, {
input: []packages.Error{
{Pos: "file:7", Msg: "message1", Kind: packages.ParseError},
{Pos: "file:8", Msg: "message2", Kind: packages.ParseError},
},
expected: 2,
}, {
input: []packages.Error{
{Pos: "file:7", Msg: "message1", Kind: packages.ParseError},
{Pos: "file:8", Msg: "message2", Kind: packages.ParseError},
{Pos: "file:7", Msg: "message1", Kind: packages.ParseError},
},
expected: 2,
}}
for i, tc := range testcases {
out := dedup(tc.input)
if len(out) != tc.expected {
t.Errorf("[%d] dedup(%v) = '%v', expected %d",
i, tc.input, out, tc.expected)
}
}
}

View File

@@ -1,25 +1,30 @@
#!/bin/bash
# Copyright 2020 Lingfei Kong <colin404@foxmail.com>. All rights reserved.
# Use of this source code is governed by a MIT style
# license that can be found in the LICENSE file.
#!/usr/bin/env bash
# Copyright © 2023 OpenIM. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
: << EOF
The API performance test script automatically executes wrk commands, collects data, analyzes it, and calls gnuplot to plot it
Usage (to test API performance) :
Start the openim-api(port 10002)
Execute the test script: ./wrktest.sh
1. Start the openim-api(port 10002)
2. Execute the test script: ./wrktest.sh
The script will generate the data file.dat, each column meaning: concurrency QPS average response time success rate
Usage (Compare the results of 2 tests)
1. The performance test:. / wrktest. Sh openim apiserver - http://127.0.0.1:10002/healthz
Execute the command:./wrktest.sh diff apiserver.dat http.dat
2. Execute the command:./wrktest.sh diff apiserver.dat http.dat
&gt; Note: Make sure you have wrk and gnuplot installed on your system