Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search
/
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading.
Please reload this page
.
Yolean
/
kubernetes-kafka
Public
Notifications
You must be signed in to change notification settings
Fork
714
Star
1.8k
Code
Issues
60
Pull requests
22
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Actions
Projects
Security and quality
Insights
Upgrade to Kafka 0.11 and test for open issues
- #30
#30
Merged
solsson
merged 107 commits into
master
Yolean/kubernetes-kafka:master
from
kafka-011
Yolean/kubernetes-kafka:kafka-011
Copy head branch name to clipboard
Jul 29, 2017
Conversation
Commits
107
(107)
Checks
Files changed
Merged
Upgrade to Kafka 0.11 and test for open issues
#30
solsson
merged 107 commits into
master
Yolean/kubernetes-kafka:master
from
kafka-011
Yolean/kubernetes-kafka:kafka-011
Copy head branch name to clipboard
Commits
Commits on Jun 23, 2017
Uses latest jre image
solsson
committed
642c620
View commit details
Copy full SHA for 642c620
Browse repository at this point
Upgrades Kafka to latest release
solsson
committed
42ea218
View commit details
Copy full SHA for 42ea218
Browse repository at this point
Upgrades Scala to latest, supported as of Kafka 0.10.2
solsson
committed
3739b24
View commit details
Copy full SHA for 3739b24
Browse repository at this point
Adds namespace to kubectl commands, matching that in yamls
solsson
committed
a385740
View commit details
Copy full SHA for a385740
Browse repository at this point
Makes PV match PVC, avoiding small initial storage becase resize is difficult in GKE
solsson
committed
1110d59
View commit details
Copy full SHA for 1110d59
Browse repository at this point
0.10.2 adds to the confusion about consumer args, #21
solsson
committed
b9340fe
View commit details
Copy full SHA for b9340fe
Browse repository at this point
Use a generic kafka image and explicitly override "eligible for deletion"
solsson
committed
cfe5cd7
View commit details
Copy full SHA for cfe5cd7
Browse repository at this point
Merge branch 'use-generic-kafka-image' into kafka-011
solsson
committed
5bf97fc
View commit details
Copy full SHA for 5bf97fc
Browse repository at this point
Upgrade to development build of the latest 0.11 RC
solsson
committed
5b8e94a
View commit details
Copy full SHA for 5b8e94a
Browse repository at this point
Modern kafka clients use a bootstrap servers list to connect...
Show description for 5ad1550
solsson
committed
5ad1550
View commit details
Copy full SHA for 5ad1550
Browse repository at this point
Commits on Jun 25, 2017
Reuses the statefulset's image in test pods, without tag duplication...
Show description for 6e8cab0
solsson
committed
6e8cab0
View commit details
Copy full SHA for 6e8cab0
Browse repository at this point
Borrows string trick from https://github.com/kubernetes/charts/
solsson
committed
15bcb87
View commit details
Copy full SHA for 15bcb87
Browse repository at this point
Merge branch 'no-roundrobin-service' into kafka-011
solsson
committed
401d65e
View commit details
Copy full SHA for 401d65e
Browse repository at this point
Upgrades zookeeper to latest
solsson
committed
a2b658a
View commit details
Copy full SHA for a2b658a
Browse repository at this point
Uses zookeeper without the bind address sed
solsson
committed
344df6e
View commit details
Copy full SHA for 344df6e
Browse repository at this point
We do need the bind address sed. This image logs the at start, to reduce the element of surprise.
solsson
committed
8897c05
View commit details
Copy full SHA for 8897c05
Browse repository at this point
Removes out-of-date zookeeper info
solsson
committed
a9b7a22
View commit details
Copy full SHA for a9b7a22
Browse repository at this point
Merge branch 'zookeeper-update' into kafka-011
solsson
committed
c39b65c
View commit details
Copy full SHA for c39b65c
Browse repository at this point
Uses the new small image, with only selected jars from the build
solsson
committed
07fc0d1
View commit details
Copy full SHA for 07fc0d1
Browse repository at this point
Default log dir is /tmp/kafka-logs so it should be changed to be inside the persistent volume mount
solsson
committed
df88b0d
View commit details
Copy full SHA for df88b0d
Browse repository at this point
Merge branch 'log-dir-inside-volume' into kafka-011
solsson
committed
a83683f
View commit details
Copy full SHA for a83683f
Browse repository at this point
Support for generic image results in a verbose startup command, make it git friendly
solsson
committed
5919d45
View commit details
Copy full SHA for 5919d45
Browse repository at this point
Adds metrics exporter for Prometheus
solsson
committed
3ab9938
View commit details
Copy full SHA for 3ab9938
Browse repository at this point
Use a specific build, github.com/solsson/dockerfiles/commit/81e8e4c20be19835d4036db1c975ff17455fe7e7
solsson
committed
a1823a8
View commit details
Copy full SHA for a1823a8
Browse repository at this point
Merge pull request #31 from Yolean/metrics-via-jmx
Show description for 4a56062
solsson
authored
4a56062
View commit details
Copy full SHA for 4a56062
Browse repository at this point
Reindents yaml to match the rest of this repo, and most examples out there
solsson
committed
4297271
View commit details
Copy full SHA for 4297271
Browse repository at this point
Uses the kafka image as Zookeeper service...
Show description for ccb9e5d
solsson
committed
ccb9e5d
View commit details
Copy full SHA for ccb9e5d
Browse repository at this point
Uses the same data path convention as zookeeper, from Confluent Platform
solsson
committed
3344799
View commit details
Copy full SHA for 3344799
Browse repository at this point
Merge partial branch 'zookeeper-data', the switch to kafka image
solsson
committed
edf7d84
View commit details
Copy full SHA for edf7d84
Browse repository at this point
Uses a named storage class so you can select volume type specifically for zoo
solsson
committed
4351e7c
View commit details
Copy full SHA for 4351e7c
Browse repository at this point
Verified the volume setup with Minikube
solsson
committed
9479e81
View commit details
Copy full SHA for 9479e81
Browse repository at this point
Updates the readme
solsson
committed
a8c8a39
View commit details
Copy full SHA for a8c8a39
Browse repository at this point
Enables metrics export to Prometheus, but they look very uninteresting.
Show description for 26173af
solsson
committed
26173af
View commit details
Copy full SHA for 26173af
Browse repository at this point
Commits on Jun 26, 2017
Makes persistence a fundamental attribute of the statefulset
solsson
committed
4fd1e5e
View commit details
Copy full SHA for 4fd1e5e
Browse repository at this point
Creates identical definitions for a non-persistent zoo statefulset
solsson
committed
225569f
View commit details
Copy full SHA for 225569f
Browse repository at this point
A cluster in three availability zones now get one persistent zk each, and two that can move automatically at node failures
solsson
committed
cb83353
View commit details
Copy full SHA for cb83353
Browse repository at this point
Merge pull request #34 from Yolean/zookeeper-availability-zones
Show description for 4a16d4f
solsson
authored
4a16d4f
View commit details
Copy full SHA for 4a16d4f
Browse repository at this point
Forks can tweak storage classes, but here we want setup to be simple...
Show description for efb1019
solsson
committed
efb1019
View commit details
Copy full SHA for efb1019
Browse repository at this point
Commits on Jun 27, 2017
Merge pull request #33 from Yolean/zookeeper-data
Show description for 13e9818
solsson
authored
13e9818
View commit details
Copy full SHA for 13e9818
Browse repository at this point
Uses dynamically provisioned volume for Kafka too. It has matured, ...
Show description for 10543bf
solsson
committed
10543bf
View commit details
Copy full SHA for 10543bf
Browse repository at this point
Adds utility to update the kafka image, which we keep the same to minimize pull waits
solsson
committed
f45ced5
View commit details
Copy full SHA for f45ced5
Browse repository at this point
Adds a test that produces a message that you can see in the logs of 21consumer-test1
solsson
committed
49865bc
View commit details
Copy full SHA for 49865bc
Browse repository at this point
Adds tentative resource requests, based on what idle pods use (though this includes monitoring)
solsson
committed
3bc821b
View commit details
Copy full SHA for 3bc821b
Browse repository at this point
Removes volume claims documentation, as we've gone completely dynamic
solsson
committed
620c4e2
View commit details
Copy full SHA for 620c4e2
Browse repository at this point
A monitoring-only pod uses 0m / ~32Mi resources
solsson
committed
f4ac288
View commit details
Copy full SHA for f4ac288
Browse repository at this point
s
solsson
committed
1a8f2d9
View commit details
Copy full SHA for 1a8f2d9
Browse repository at this point
Got quite repeatable OOMKilled on pzoo pods, so I figured it must be...
Show description for 1344238
solsson
committed
1344238
View commit details
Copy full SHA for 1344238
Browse repository at this point
Merge pull request #35 from Yolean/resource-limits
Show description for e5ba57e
solsson
authored
e5ba57e
View commit details
Copy full SHA for e5ba57e
Browse repository at this point
Reverts to default termination period, and uses bash for "shell form"...
Show description for 411192d
solsson
committed
411192d
View commit details
Copy full SHA for 411192d
Browse repository at this point
Adds probes, but for Kafka I don't think it indicates readiness...
Show description for 2c4b6cd
solsson
committed
2c4b6cd
View commit details
Copy full SHA for 2c4b6cd
Browse repository at this point
Reduces termination grace period for zookeeper because I fail to trigger termination by signal
solsson
committed
0ab701c
View commit details
Copy full SHA for 0ab701c
Browse repository at this point
Raises memory limit for metrics; got 10 OOMKilled per pod in the last 3 hours
solsson
committed
b3c6cd2
View commit details
Copy full SHA for b3c6cd2
Browse repository at this point
Commits on Jun 28, 2017
Limiting metrics' JVM to match resource limits. Still getting OOMKilled though, but maybe half as often.
solsson
committed
53b2cb5
View commit details
Copy full SHA for 53b2cb5
Browse repository at this point
Upgrades to latest build from https://github.com/solsson/dockerfiles/pull/4, with plain logging>=INFO config
solsson
committed
bccfdfa
View commit details
Copy full SHA for bccfdfa
Browse repository at this point
shell script is now osx, but no longer gnu :)
solsson
committed
d6b870c
View commit details
Copy full SHA for d6b870c
Browse repository at this point
Applies the limit to persistent zookeeper pods too. They seem more prone to restarts than 51zoo.
solsson
committed
4481b4d
View commit details
Copy full SHA for 4481b4d
Browse repository at this point
Same startup as 51zoo
solsson
committed
07d895c
View commit details
Copy full SHA for 07d895c
Browse repository at this point
Commits on Jul 23, 2017
Fixes posix compatibility for probes
solsson
committed
ac443a9
View commit details
Copy full SHA for ac443a9
Browse repository at this point
Upgrades to current https://github.com/solsson/dockerfiles/pull/5
solsson
committed
9f47cd0
View commit details
Copy full SHA for 9f47cd0
Browse repository at this point
solsson/kafka on debian restores installation path to /opt/kafka
solsson
committed
6a934de
View commit details
Copy full SHA for 6a934de
Browse repository at this point
Default shell on debian should forward signals properly
solsson
committed
c188f43
View commit details
Copy full SHA for c188f43
Browse repository at this point
Adds yaml with the default .properties from 0.11.0.0
solsson
committed
1758478
View commit details
Copy full SHA for 1758478
Browse repository at this point
Commits on Jul 25, 2017
Use config map's config instead of image's
solsson
committed
a30b5e7
View commit details
Copy full SHA for a30b5e7
Browse repository at this point
Validates against a gotcha
solsson
committed
b3491ce
View commit details
Copy full SHA for b3491ce
Browse repository at this point
With stock config we have to change zookeeper lookup from the default localhost
solsson
committed
d8b2b41
View commit details
Copy full SHA for d8b2b41
Browse repository at this point
Merge pull request #43 from Yolean/kafka-011-config-map
Show description for b2ebc80
solsson
authored
b2ebc80
View commit details
Copy full SHA for b2ebc80
Browse repository at this point
As recommended by https://www.confluent.io/blog/apache-kafka-for-service-architectures/
solsson
committed
c86ed9c
View commit details
Copy full SHA for c86ed9c
Browse repository at this point
I think time saved by auto-creating topics will be lost ...
Show description for 0681cc5
solsson
committed
0681cc5
View commit details
Copy full SHA for 0681cc5
Browse repository at this point
New build with https://github.com/solsson/dockerfiles/pull/9
solsson
committed
480b5fa
View commit details
Copy full SHA for 480b5fa
Browse repository at this point
Commits on Jul 26, 2017
New build at commit 0314080
solsson
committed
8340b11
View commit details
Copy full SHA for 8340b11
Browse repository at this point
Clarifies a gotcha: to mount config with log4j.properties ...
Show description for 114b773
solsson
committed
114b773
View commit details
Copy full SHA for 114b773
Browse repository at this point
Tagged with the policy from https://github.com/solsson/dockerfiles/pull/11
solsson
committed
6f8f6d4
View commit details
Copy full SHA for 6f8f6d4
Browse repository at this point
With explicit log4j path we can change config mount ...
Show description for 5bb49e3
solsson
committed
5bb49e3
View commit details
Copy full SHA for 5bb49e3
Browse repository at this point
Default shell on Debian shows the same symptom ...
Show description for a2d324d
solsson
committed
a2d324d
View commit details
Copy full SHA for a2d324d
Browse repository at this point
Demonstrates how an init script can be used to ...
Show description for be5a820
solsson
committed
be5a820
View commit details
Copy full SHA for be5a820
Browse repository at this point
Moves broker.id config into init script
solsson
committed
0d534e8
View commit details
Copy full SHA for 0d534e8
Browse repository at this point
With no bash tricks in command we can use the actual bin ...
Show description for bfe7e31
solsson
committed
bfe7e31
View commit details
Copy full SHA for bfe7e31
Browse repository at this point
Employs the init script concept for zookeeper too, reducing duplcation
solsson
committed
fda7bdb
View commit details
Copy full SHA for fda7bdb
Browse repository at this point
Commits on Jul 27, 2017
Places the myid magic number where replicas are
solsson
committed
082f57a
View commit details
Copy full SHA for 082f57a
Browse repository at this point
Stops logs from growing when zookeeper is idle
solsson
committed
b848f85
View commit details
Copy full SHA for b848f85
Browse repository at this point
Merge pull request #47 from Yolean/config-init
Show description for 752cd45
solsson
authored
752cd45
View commit details
Copy full SHA for 752cd45
Browse repository at this point
Merge pull request #46 from Yolean/switch-to-debian-image
Show description for fe25fb9
solsson
authored
fe25fb9
View commit details
Copy full SHA for fe25fb9
Browse repository at this point
Unimportant
solsson
committed
829de73
View commit details
Copy full SHA for 829de73
Browse repository at this point
Belongs in the github.com/yolean/kubernetes-monitoring project
solsson
committed
51c3097
View commit details
Copy full SHA for 51c3097
Browse repository at this point
Commits on Jul 28, 2017
This project avoids scripting through addons ...
Show description for c481cba
solsson
committed
c481cba
View commit details
Copy full SHA for c481cba
Browse repository at this point
Makes /metrics export opt-in (through addon branch coming up)
solsson
committed
22a314a
View commit details
Copy full SHA for 22a314a
Browse repository at this point
Includes metrics in our prod
solsson
committed
98635cc
View commit details
Copy full SHA for 98635cc
Browse repository at this point
Uses storage classes for our prod
solsson
committed
364c9b7
View commit details
Copy full SHA for 364c9b7
Browse repository at this point
Could be a pitch
solsson
committed
ba90764
View commit details
Copy full SHA for ba90764
Browse repository at this point
Could be the motivation
solsson
committed
36a3603
View commit details
Copy full SHA for 36a3603
Browse repository at this point
Shorter
solsson
committed
dfa82bd
View commit details
Copy full SHA for dfa82bd
Browse repository at this point
Shorter still
solsson
committed
315317f
View commit details
Copy full SHA for 315317f
Browse repository at this point
If you got this far you don't need Kubernetes intro
solsson
committed
7e6df71
View commit details
Copy full SHA for 7e6df71
Browse repository at this point
Reworked in https://github.com/Yolean/kubernetes-kafka/pull/51
solsson
committed
c4f3c41
View commit details
Copy full SHA for c4f3c41
Browse repository at this point
Metrics intro belongs in ...
Show description for a880307
solsson
committed
a880307
View commit details
Copy full SHA for a880307
Browse repository at this point
Where to go from here
solsson
committed
91b4dde
View commit details
Copy full SHA for 91b4dde
Browse repository at this point
Use a lighter readme with more links, as readers' background will vary widely
solsson
committed
21a94a3
View commit details
Copy full SHA for 21a94a3
Browse repository at this point
Review after github render
solsson
committed
b581659
View commit details
Copy full SHA for b581659
Browse repository at this point
Prepares for tests to move to separate namespace
solsson
committed
8cbd671
View commit details
Copy full SHA for 8cbd671
Browse repository at this point
Suggests a structure for test cases as single yml
solsson
committed
c7eae1b
View commit details
Copy full SHA for c7eae1b
Browse repository at this point
Working boilerplate, with output to kubectl logs
solsson
committed
24c43d2
View commit details
Copy full SHA for 24c43d2
Browse repository at this point
Implements the actual test
solsson
committed
c91c42e
View commit details
Copy full SHA for c91c42e
Browse repository at this point
Introduces the test automation concept as briefly as possible
solsson
committed
754fd2f
View commit details
Copy full SHA for 754fd2f
Browse repository at this point
Makes room for a more basic basic test
solsson
committed
454bea4
View commit details
Copy full SHA for 454bea4
Browse repository at this point
Now we're on par with the old tests, but automated
solsson
committed
48bd7c3
View commit details
Copy full SHA for 48bd7c3
Browse repository at this point
Merge pull request #51 from Yolean/test-driven-kubernetes-concept
Show description for cabe9be
solsson
authored
cabe9be
View commit details
Copy full SHA for cabe9be
Browse repository at this point
Commits on Jul 29, 2017
The test concept just caught a mistake
solsson
committed
ab35705
View commit details
Copy full SHA for ab35705
Browse repository at this point
You can’t perform that action at this time.