zsh-histdb

Check-in [575d7a36ce]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:clean up for people to use
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | master | trunk
Files: files | file ages | folders
SHA3-256:575d7a36ced33c56d5e8cea005dfc6859f981b3dbb024ed8c7b4874edf7a2e8b
User & Date: tom.hinton@cse.org.uk 2017-04-13 16:48:37
Context
2017-04-17
19:23
update function names to be better check-in: 14eb286998 user: t@larkery.com tags: master, trunk
2017-04-13
16:48
clean up for people to use check-in: 575d7a36ce user: tom.hinton@cse.org.uk tags: master, trunk
16:09
copy source from elsewhere check-in: beada4156b user: tom.hinton@cse.org.uk tags: master, trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to README.org.

7
8
9
10
11
12
13
14














15

































16




17














- The working directory where the command was run
- The hostname of the machine
- A unique per-host session ID, so history from several sessions is not confused
- The exit status of the command

It is also possible to merge multiple history databases together without conflict, so long as all your machines have different hostnames.

* TODO Installation














* TODO Querying history

































* TODO Database schema




* TODO Synchronising history





















|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
- The working directory where the command was run
- The hostname of the machine
- A unique per-host session ID, so history from several sessions is not confused
- The exit status of the command

It is also possible to merge multiple history databases together without conflict, so long as all your machines have different hostnames.

* Installation

You will need ~sqlite3~ and the usual coreutils commands installed on your ~PATH~.
To load and activate history recording you need to source ~sqlite-history.zsh~ from your shell in your zsh startup files.

If you want the history to contain correct timing information, you will also need to source ~history-timer.zsh~ and run

#+BEGIN_SRC zsh
autoload -Uz add-zsh-hook
add-zsh-hook preexec _start_timer
add-zsh-hook precmd  _stop_timer
#+END_SRC

in your zsh startup files.

* Querying history
You can query the history with the ~histdb~ command.
With no arguments it will print one screenful of history on the current host.

With arguments, it will print history lines matching their concatenation.

For wildcards within a history line, you can use the ~%~ character, which is like the shell glob ~*~, so ~histdb this%that~ will match any history line containing ~this~ followed by ~that~ with zero or more characters in-between.

To search on particular hosts, directories, sessions, or time periods, see the help with ~histdb --help~.

You can also run ~histdb-top~ to see your most frequent commands, and ~histdb-top dir~ to show your favourite directory for running commands in, but these commands are really a bit useless.
** Example:

#+BEGIN_SRC text
$ histdb strace
time   ses  dir  cmd
17/03  438  ~    strace conkeror
22/03  522  ~    strace apropos cake
22/03  522  ~    strace -e trace=file s
22/03  522  ~    strace -e trace=file ls
22/03  522  ~    strace -e trace=file cat temp/people.vcf
22/03  522  ~    strace -e trace=file cat temp/gammu.log
22/03  522  ~    run-help strace
24/03  547  ~    man strace
#+END_SRC

These are all the history entries involving ~strace~ in my history.
If there was more than one screenful, I would need to say ~--limit 1000~ or some other large number.
The command does not warn you if you haven't seen all the results.
The ~ses~ column contains a unique session number, so all the ~522~ rows are from the same shell session.

To see all hosts, add ~--host~ /after/ the query terms.
To see a specific host, add ~--host hostname~.
To see all of a specific session say e.g. ~-s 522 --limit 10000~.
* Database schema
The database lives by default in ~$HOME/.histdb/zsh-history.db~.
You can look in it easily by running ~_histdb~, as this actually just fires up sqlite with the database.

For inspiration you can also use ~histdb~ with the ~-d~ argument and it will print the SQL it's running.
* Synchronising history
You should be able to synchronise the history using ~git~; a 3-way merge driver is supplied in ~histdb-merge~.

The 3-way merge will only work properly if all the computers on which you use the repository have different hostnames.

The ~histdb-commit~ command will initialize git in the histdb directory and configure the merge driver for you first time you run it.
Subsequent times it will commit all changes, pull all changes, force a merge, and push all changes back again.
The commit message is useless, so if you find that kind of thing upsetting you will need to fix it.

The reason for using ~histdb-commit~ instead of doing it by hand is that if you are running the git steps in your shell the history database will be changed each command, and so you will never be able to do a pull / merge.
* Completion
None, and I've used the names with underscores to mean something else.
* Pull requests / missing features
Happy to look at changes.
I did at one point have a reverse-isearch thing in here for searching the database interactively, but it didn't really make my life any better so I deleted it.

Added history-timer.zsh.























>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
typeset -g _STARTED
typeset -g _FINISHED

_start_timer () {
    _STARTED=$(date +%s)
    _FINISHED=$_STARTED
}

_stop_timer () {
    _FINISHED=$(date +%s)
}

Changes to sqlite-history.zsh.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
...
115
116
117
118
119
120
121
















122
123
124
125
126
127
128
...
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
which sqlite3 >/dev/null 2>&1 || return;

typeset -g HISTDB_QUERY=""
typeset -g HISTDB_FILE="${HOME}/.histdb/zsh-history.db"
typeset -g HISTDB_SESSION=""
typeset -g HISTDB_MAX_ROWID=""
typeset -g HISTDB_HOST=""

typeset -gA HISTDB_RESULT

sql_escape () {
    sed -e "s/'/''/g" <<< "$@"
}

_histdb () {
    sqlite3 "${HISTDB_FILE}" "$@"
    [[ "$?" -ne 0 ]] && echo "error in $@"
}

_histdb_init () {
    if ! [[ -e "${HISTDB_FILE}" ]]; then
        local DN="$(dirname ${HISTDB_FILE})"
        if ! [[ -d $DN ]]; then
            mkdir -p -- $DN
            pushd $DN
            git init
            git config merge.histdb.driver "$HOME/.zsh/histdb-merge %O %A %B"
            popd
        fi
        _histdb <<-EOF
create table commands (argv text, unique(argv) on conflict ignore);
create table places   (host text, dir text, unique(host, dir) on conflict ignore);
create table history  (session int,
                       command_id int references commands (rowid),
                       place_id int references places (rowid),
................................................................................
    _histdb -separator "$sep" \
            -header \
            "select count(*) as count, places.host, replace($field, '
', '
$sep$sep') as ${1:-cmd} from history left join commands on history.command_id=commands.rowid left join places on history.place_id=places.rowid group by places.host, $field order by count(*)" |
        column -t -s "$sep"
}

















histdb () {
    _histdb_init
    local -a opts
    local -a hosts
    local -a indirs
    local -a atdirs
................................................................................
               -host+::=hosts \
               -in+::=indirs \
               -at+::=atdirs \
               d h -help \
               s+::=sessions \
               -from:- -until:- -limit:-

    usage="usage:$0 [--host] [--in] [--at] [-s n]+* [--from] [--until] [--limit] query
    --host    print the host column and show all hosts (otherwise current host)
    --host=x  find entries from host x
    --in      find only entries run in the current dir or below
    --in=x    find only entries in directory x or below
    --at      like --in, but excluding subdirectories
    -s n      only show session n
    -d        debug output query that will be run







|
<












|
|
|
<
<
<
<







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|







1
2
3
4
5
6
7
8

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23




24
25
26
27
28
29
30
...
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
...
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
which sqlite3 >/dev/null 2>&1 || return;

typeset -g HISTDB_QUERY=""
typeset -g HISTDB_FILE="${HOME}/.histdb/zsh-history.db"
typeset -g HISTDB_SESSION=""
typeset -g HISTDB_MAX_ROWID=""
typeset -g HISTDB_HOST=""
typeset -g HISTDB_INSTALLED_IN="${(%):-%N}"


sql_escape () {
    sed -e "s/'/''/g" <<< "$@"
}

_histdb () {
    sqlite3 "${HISTDB_FILE}" "$@"
    [[ "$?" -ne 0 ]] && echo "error in $@"
}

_histdb_init () {
    if ! [[ -e "${HISTDB_FILE}" ]]; then
        local hist_dir="$(dirname ${HISTDB_FILE})"
        if ! [[ -d "$hist_dir" ]]; then
            mkdir -p -- "$hist_dir"




        fi
        _histdb <<-EOF
create table commands (argv text, unique(argv) on conflict ignore);
create table places   (host text, dir text, unique(host, dir) on conflict ignore);
create table history  (session int,
                       command_id int references commands (rowid),
                       place_id int references places (rowid),
................................................................................
    _histdb -separator "$sep" \
            -header \
            "select count(*) as count, places.host, replace($field, '
', '
$sep$sep') as ${1:-cmd} from history left join commands on history.command_id=commands.rowid left join places on history.place_id=places.rowid group by places.host, $field order by count(*)" |
        column -t -s "$sep"
}

histdb-commit () {
    local hist_dir="$(dirname ${HISTDB_FILE})"
    if [[ -d "$hist_dir" ]]; then
        pushd "$hist_dir"
        if [[ $(git rev-parse --is-inside-work-tree) != "true" ]]; then
            git init
            git config merge.histdb.driver "$(basename ${HISTDB_INSTALLED_IN})/histdb-merge %O %A %B"
            echo "$(basename ${HISTDB_FILE}) merge=histdb" >> .gitattributes
            git add .gitattributes
            git add "$(basename ${HISTDB_FILE})"
        fi
        git commit -am "history" && git pull --no-edit && git push
        popd
    fi
}

histdb () {
    _histdb_init
    local -a opts
    local -a hosts
    local -a indirs
    local -a atdirs
................................................................................
               -host+::=hosts \
               -in+::=indirs \
               -at+::=atdirs \
               d h -help \
               s+::=sessions \
               -from:- -until:- -limit:-

    usage="usage:$0 terms [--host] [--in] [--at] [-s n]+* [--from] [--until] [--limit]
    --host    print the host column and show all hosts (otherwise current host)
    --host=x  find entries from host x
    --in      find only entries run in the current dir or below
    --in=x    find only entries in directory x or below
    --at      like --in, but excluding subdirectories
    -s n      only show session n
    -d        debug output query that will be run