May 29 20:44:37.156238 ip-10-0-133-224 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 29 20:44:37.156255 ip-10-0-133-224 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 29 20:44:37.156265 ip-10-0-133-224 systemd[1]: kubelet.service: Failed with result 'resources'.
May 29 20:44:37.156620 ip-10-0-133-224 systemd[1]: Failed to start Kubernetes Kubelet.
May 29 20:44:47.309082 ip-10-0-133-224 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 29 20:44:47.309100 ip-10-0-133-224 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ca28aa6328c74e759c6cc14932a036e4 --
May 29 20:47:16.826261 ip-10-0-133-224 systemd[1]: Starting Kubernetes Kubelet...
May 29 20:47:17.228789 ip-10-0-133-224 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 20:47:17.228789 ip-10-0-133-224 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 29 20:47:17.228789 ip-10-0-133-224 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 20:47:17.228789 ip-10-0-133-224 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 29 20:47:17.228789 ip-10-0-133-224 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 29 20:47:17.230422 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.230338    2563 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 29 20:47:17.233087 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233072    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 20:47:17.233087 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233087    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233090    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233093    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233096    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233099    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233102    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233110    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233113    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233116    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233118    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233121    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233123    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233125    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233128    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233132    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233136    2563 feature_gate.go:328] unrecognized feature gate: Example2
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233139    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233142    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233145    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233147    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 20:47:17.233157 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233150    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233153    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233155    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233158    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233161    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233163    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233166    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233169    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233171    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233174    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233176    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233179    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233181    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233183    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233186    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233188    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233191    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233193    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233197    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 20:47:17.233627 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233199    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233202    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233204    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233206    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233209    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233211    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233214    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233216    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233218    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233221    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233223    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233225    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233228    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233231    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233234    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233237    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233240    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233243    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233245    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233247    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 20:47:17.234152 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233250    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233253    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233255    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233257    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233260    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233262    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233265    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233270    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233274    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233276    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233279    2563 feature_gate.go:328] unrecognized feature gate: Example
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233281    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233284    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233287    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233289    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233291    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233294    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233300    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233302    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 20:47:17.234642 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233305    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233307    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233310    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233312    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233315    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233317    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.233320    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234350    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234357    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234360    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234363    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234365    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234368    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234371    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234373    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234376    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234379    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234381    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234384    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234386    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 20:47:17.235132 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234389    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234391    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234394    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234397    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234399    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234403    2563 feature_gate.go:328] unrecognized feature gate: Example2
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234405    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234408    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234410    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234413    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234415    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234418    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234421    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234423    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234426    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234429    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234431    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234433    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234436    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234438    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 20:47:17.235600 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234441    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234443    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234445    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234448    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234450    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234452    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234455    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234458    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234461    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234463    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234466    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234468    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234471    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234473    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234475    2563 feature_gate.go:328] unrecognized feature gate: Example
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234478    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234480    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234483    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234486    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 20:47:17.236101 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234488    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234490    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234493    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234495    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234497    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234500    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234502    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234505    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234508    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234510    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234512    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234515    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234517    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234520    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234522    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234524    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234528    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234532    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234534    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234537    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 20:47:17.236602 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234540    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234544    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234548    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234551    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234553    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234556    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234558    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234561    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234563    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234565    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234568    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234570    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234573    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.234576    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234647    2563 flags.go:64] FLAG: --address="0.0.0.0"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234658    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234665    2563 flags.go:64] FLAG: --anonymous-auth="true"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234670    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234674    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234677    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234681    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 29 20:47:17.237104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234686    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234689    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234692    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234696    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234699    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234702    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234705    2563 flags.go:64] FLAG: --cgroup-root=""
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234708    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234710    2563 flags.go:64] FLAG: --client-ca-file=""
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234713    2563 flags.go:64] FLAG: --cloud-config=""
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234716    2563 flags.go:64] FLAG: --cloud-provider="external"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234719    2563 flags.go:64] FLAG: --cluster-dns="[]"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234723    2563 flags.go:64] FLAG: --cluster-domain=""
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234726    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234729    2563 flags.go:64] FLAG: --config-dir=""
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234732    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234735    2563 flags.go:64] FLAG: --container-log-max-files="5"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234738    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234741    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234744    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234748    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234751    2563 flags.go:64] FLAG: --contention-profiling="false"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234753    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234756    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234760    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
May 29 20:47:17.237610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234763    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234767    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234770    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234773    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234775    2563 flags.go:64] FLAG: --enable-load-reader="false"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234778    2563 flags.go:64] FLAG: --enable-server="true"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234781    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234786    2563 flags.go:64] FLAG: --event-burst="100"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234789    2563 flags.go:64] FLAG: --event-qps="50"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234792    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234795    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234797    2563 flags.go:64] FLAG: --eviction-hard=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234801    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234804    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234807    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234810    2563 flags.go:64] FLAG: --eviction-soft=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234812    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234815    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234819    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234822    2563 flags.go:64] FLAG: --experimental-mounter-path=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234825    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234828    2563 flags.go:64] FLAG: --fail-swap-on="true"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234831    2563 flags.go:64] FLAG: --feature-gates=""
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234835    2563 flags.go:64] FLAG: --file-check-frequency="20s"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234838    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 29 20:47:17.238219 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234841    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234844    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234846    2563 flags.go:64] FLAG: --healthz-port="10248"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234849    2563 flags.go:64] FLAG: --help="false"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234852    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234855    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234858    2563 flags.go:64] FLAG: --http-check-frequency="20s"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234861    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234865    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234868    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234871    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234873    2563 flags.go:64] FLAG: --image-service-endpoint=""
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234876    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234879    2563 flags.go:64] FLAG: --kube-api-burst="100"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234882    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234885    2563 flags.go:64] FLAG: --kube-api-qps="50"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234887    2563 flags.go:64] FLAG: --kube-reserved=""
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234890    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234893    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234896    2563 flags.go:64] FLAG: --kubelet-cgroups=""
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234899    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234901    2563 flags.go:64] FLAG: --lock-file=""
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234904    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234907    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
May 29 20:47:17.238823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234910    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234918    2563 flags.go:64] FLAG: --log-json-split-stream="false"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234921    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234924    2563 flags.go:64] FLAG: --log-text-split-stream="false"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234927    2563 flags.go:64] FLAG: --logging-format="text"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234929    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234933    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234935    2563 flags.go:64] FLAG: --manifest-url=""
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234938    2563 flags.go:64] FLAG: --manifest-url-header=""
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234942    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234945    2563 flags.go:64] FLAG: --max-open-files="1000000"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234949    2563 flags.go:64] FLAG: --max-pods="110"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234952    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234955    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234958    2563 flags.go:64] FLAG: --memory-manager-policy="None"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234961    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234964    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234967    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234970    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234976    2563 flags.go:64] FLAG: --node-status-max-images="50"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234979    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234982    2563 flags.go:64] FLAG: --oom-score-adj="-999"
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.234985    2563 flags.go:64] FLAG: --pod-cidr=""
May 29 20:47:17.239447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235003    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235008    2563 flags.go:64] FLAG: --pod-manifest-path=""
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235011    2563 flags.go:64] FLAG: --pod-max-pids="-1"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235014    2563 flags.go:64] FLAG: --pods-per-core="0"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235017    2563 flags.go:64] FLAG: --port="10250"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235020    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235023    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b3823fcb5c8e40f3"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235026    2563 flags.go:64] FLAG: --qos-reserved=""
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235029    2563 flags.go:64] FLAG: --read-only-port="10255"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235031    2563 flags.go:64] FLAG: --register-node="true"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235036    2563 flags.go:64] FLAG: --register-schedulable="true"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235039    2563 flags.go:64] FLAG: --register-with-taints=""
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235043    2563 flags.go:64] FLAG: --registry-burst="10"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235045    2563 flags.go:64] FLAG: --registry-qps="5"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235048    2563 flags.go:64] FLAG: --reserved-cpus=""
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235051    2563 flags.go:64] FLAG: --reserved-memory=""
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235055    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235058    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235060    2563 flags.go:64] FLAG: --rotate-certificates="false"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235063    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235065    2563 flags.go:64] FLAG: --runonce="false"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235068    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235071    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235074    2563 flags.go:64] FLAG: --seccomp-default="false"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235076    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235079    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 29 20:47:17.240001 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235083    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235086    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235089    2563 flags.go:64] FLAG: --storage-driver-password="root"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235091    2563 flags.go:64] FLAG: --storage-driver-secure="false"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235094    2563 flags.go:64] FLAG: --storage-driver-table="stats"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235097    2563 flags.go:64] FLAG: --storage-driver-user="root"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235100    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235102    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235105    2563 flags.go:64] FLAG: --system-cgroups=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235108    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235113    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235116    2563 flags.go:64] FLAG: --tls-cert-file=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235119    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235122    2563 flags.go:64] FLAG: --tls-min-version=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235125    2563 flags.go:64] FLAG: --tls-private-key-file=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235128    2563 flags.go:64] FLAG: --topology-manager-policy="none"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235132    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235135    2563 flags.go:64] FLAG: --topology-manager-scope="container"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235138    2563 flags.go:64] FLAG: --v="2"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235142    2563 flags.go:64] FLAG: --version="false"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235146    2563 flags.go:64] FLAG: --vmodule=""
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235150    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.235153    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235245    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 20:47:17.240640 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235248    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235251    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235254    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235256    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235258    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235261    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235263    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235266    2563 feature_gate.go:328] unrecognized feature gate: Example2
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235268    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235271    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235274    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235276    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235278    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235281    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235283    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235285    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235288    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235290    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235293    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235295    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 20:47:17.241232 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235299    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235302    2563 feature_gate.go:328] unrecognized feature gate: Example
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235305    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235308    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235312    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235314    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235317    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235320    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235322    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235325    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235327    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235330    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235332    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235334    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235337    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235339    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235342    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235344    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235347    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235349    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 20:47:17.241794 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235351    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235354    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235357    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235360    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235362    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235364    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235367    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235369    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235372    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235374    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235377    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235379    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235384    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235386    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235389    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235391    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235395    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235397    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235400    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235402    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 20:47:17.242306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235405    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235407    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235410    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235412    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235414    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235417    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235419    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235421    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235424    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235426    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235429    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235431    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235434    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235436    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235438    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235441    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235444    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235446    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235450    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 20:47:17.243055 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235453    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235456    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235459    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235461    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235464    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.235467    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 20:47:17.243729 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.236025    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 20:47:17.245061 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.245043    2563 server.go:530] "Kubelet version" kubeletVersion="v1.33.12"
May 29 20:47:17.245106 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.245063    2563 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245115    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245120    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245123    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245126    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245129    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245132    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 20:47:17.245135 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245135    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245138    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245142    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245145    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245147    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245150    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245153    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245156    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245158    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245160    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245163    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245165    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245168    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245171    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245173    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245176    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245178    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245181    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245184    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245187    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 20:47:17.245302 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245190    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245192    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245196    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245201    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245204    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245207    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245211    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245215    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245218    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245220    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245223    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245225    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245227    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245230    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245234    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245236    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245239    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245242    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245245    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 20:47:17.245793 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245248    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245251    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245253    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245256    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245258    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245261    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245263    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245266    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245268    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245271    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245273    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245275    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245278    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245280    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245282    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245285    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245288    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245290    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245292    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 20:47:17.246276 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245295    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245297    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245300    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245302    2563 feature_gate.go:328] unrecognized feature gate: Example2
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245305    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245307    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245310    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245312    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245316    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245319    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245321    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245324    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245328    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245331    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245333    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245336    2563 feature_gate.go:328] unrecognized feature gate: Example
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245338    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245341    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245343    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245345    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 20:47:17.246733 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245348    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245350    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.245355    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245452    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245457    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245460    2563 feature_gate.go:328] unrecognized feature gate: Example2
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245463    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245466    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245468    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245471    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245473    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245476    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245478    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245480    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245483    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245485    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 29 20:47:17.247250 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245487    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245490    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245492    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245494    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245497    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245501    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245503    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245506    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245508    2563 feature_gate.go:328] unrecognized feature gate: Example
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245511    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245514    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245517    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245519    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245522    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245524    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245526    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245529    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245531    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245534    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245536    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 29 20:47:17.247634 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245538    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245541    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245543    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245546    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245548    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245550    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245553    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245555    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245558    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245560    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245562    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245564    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245567    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245571    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245575    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245578    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245581    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245584    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245586    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245589    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 29 20:47:17.248142 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245592    2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245594    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245597    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245600    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245602    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245605    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245607    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245609    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245612    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245614    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245617    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245619    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245621    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245625    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245628    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245630    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245633    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245635    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245637    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 29 20:47:17.248637 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245640    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245642    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245647    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245650    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245652    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245655    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245657    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245659    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245662    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245665    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245667    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245670    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:17.245672    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.245677    2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 29 20:47:17.249115 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.246591    2563 server.go:962] "Client rotation is on, will bootstrap in background"
May 29 20:47:17.251493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.251479    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 29 20:47:17.252551 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.252540    2563 server.go:1019] "Starting client certificate rotation"
May 29 20:47:17.252659 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.252643    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 20:47:17.252697 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.252684    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 29 20:47:17.278662 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.278641    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 20:47:17.280489 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.280470    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 29 20:47:17.296138 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.296116    2563 log.go:25] "Validated CRI v1 runtime API"
May 29 20:47:17.302957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.302936    2563 log.go:25] "Validated CRI v1 image API"
May 29 20:47:17.304423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.304407    2563 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 29 20:47:17.308849 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.308823    2563 fs.go:135] Filesystem UUIDs: map[454a71c0-0a9c-4cc2-af6e-af9d9cc395bd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ec8e9d7f-f4ee-4608-979c-6ec2a2e9438c:/dev/nvme0n1p3]
May 29 20:47:17.308917 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.308848    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 29 20:47:17.314341 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.314226    2563 manager.go:217] Machine: {Timestamp:2026-05-29 20:47:17.313692406 +0000 UTC m=+0.377858536 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106573 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298bca91744685a829e6965ceaffe7 SystemUUID:ec298bca-9174-4685-a829-e6965ceaffe7 BootID:ca28aa63-28c7-4e75-9c6c-c14932a036e4 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cc:a5:2c:09:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cc:a5:2c:09:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:80:b1:31:6a:89 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 29 20:47:17.314437 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.314355    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 29 20:47:17.314480 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.314437    2563 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 29 20:47:17.315382 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.315364    2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 20:47:17.316273 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.316248    2563 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 29 20:47:17.316447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.316278    2563 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 29 20:47:17.316534 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.316461    2563 topology_manager.go:138] "Creating topology manager with none policy"
May 29 20:47:17.316534 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.316475    2563 container_manager_linux.go:306] "Creating device plugin manager"
May 29 20:47:17.316534 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.316486    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 20:47:17.317306 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.317293    2563 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 29 20:47:17.318484 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.318472    2563 state_mem.go:36] "Initialized new in-memory state store"
May 29 20:47:17.318794 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.318781    2563 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 29 20:47:17.321544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.321532    2563 kubelet.go:491] "Attempting to sync node with API server"
May 29 20:47:17.321616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.321551    2563 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 29 20:47:17.321616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.321574    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 29 20:47:17.321616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.321588    2563 kubelet.go:397] "Adding apiserver pod source"
May 29 20:47:17.321616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.321600    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 29 20:47:17.323105 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.323092    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 20:47:17.323170 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.323116    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 29 20:47:17.326074 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.326053    2563 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1"
May 29 20:47:17.328467 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.328452    2563 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 29 20:47:17.330802 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330786    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 29 20:47:17.330802 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330805    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330812    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330820    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330834    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330844    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330850    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330855    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330862    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330867    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330885    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 29 20:47:17.330905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330891    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 29 20:47:17.331269 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330915    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 29 20:47:17.331269 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.330921    2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 29 20:47:17.332820 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.332794    2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 29 20:47:17.332891 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.332800    2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 29 20:47:17.333140 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.333125    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ptgv"
May 29 20:47:17.334363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.334345    2563 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 29 20:47:17.334447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.334383    2563 server.go:1295] "Started kubelet"
May 29 20:47:17.334553 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.334500    2563 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 29 20:47:17.334607 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.334574    2563 server_v1.go:47] "podresources" method="list" useActivePods=true
May 29 20:47:17.334895 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.334490    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 29 20:47:17.335375 ip-10-0-133-224 systemd[1]: Started Kubernetes Kubelet.
May 29 20:47:17.335669 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.335641    2563 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 29 20:47:17.336243 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.336230    2563 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 29 20:47:17.341318 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.341297    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ptgv"
May 29 20:47:17.341661 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.341647    2563 server.go:317] "Adding debug handlers to kubelet server"
May 29 20:47:17.348047 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.348026    2563 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 29 20:47:17.348160 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.348118    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 29 20:47:17.348685 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.348668    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 29 20:47:17.348875 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.348858    2563 volume_manager.go:295] "The desired_state_of_world populator starts"
May 29 20:47:17.348942 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.348875    2563 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 29 20:47:17.349005 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.348978    2563 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 29 20:47:17.349096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.349083    2563 reconstruct.go:97] "Volume reconstruction finished"
May 29 20:47:17.349160 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.349097    2563 reconciler.go:26] "Reconciler: start to sync state"
May 29 20:47:17.353626 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.353566    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.354067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.354045    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 29 20:47:17.354067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.354065    2563 factory.go:55] Registering systemd factory
May 29 20:47:17.354067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.354074    2563 factory.go:223] Registration of the systemd container factory successfully
May 29 20:47:17.354507 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.354481    2563 factory.go:153] Registering CRI-O factory
May 29 20:47:17.354575 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.354510    2563 factory.go:223] Registration of the crio container factory successfully
May 29 20:47:17.355560 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.355517    2563 factory.go:103] Registering Raw factory
May 29 20:47:17.355560 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.355545    2563 manager.go:1196] Started watching for new ooms in manager
May 29 20:47:17.356347 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.356330    2563 manager.go:319] Starting recovery of all containers
May 29 20:47:17.356633 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.356616    2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:17.356947 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.356621    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-224.ec2.internal\" not found" node="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.361463 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.361283    2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 29 20:47:17.366697 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.366681    2563 manager.go:324] Recovery completed
May 29 20:47:17.370701 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.370689    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.372522 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.372505    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.372594 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.372536    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.372594 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.372546    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.373002 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.372971    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 29 20:47:17.373002 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.372983    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 29 20:47:17.373108 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.373018    2563 state_mem.go:36] "Initialized new in-memory state store"
May 29 20:47:17.375733 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.375722    2563 policy_none.go:49] "None policy: Start"
May 29 20:47:17.375776 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.375738    2563 memory_manager.go:186] "Starting memorymanager" policy="None"
May 29 20:47:17.375776 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.375749    2563 state_mem.go:35] "Initializing new in-memory state store"
May 29 20:47:17.419562 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.419547    2563 manager.go:341] "Starting Device Plugin manager"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.419580    2563 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.419590    2563 server.go:85] "Starting device plugin registration server"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.419790    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.419800    2563 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.419911    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.420020    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.420034    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.420699    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 29 20:47:17.429175 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.420732    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.495750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.495690    2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 29 20:47:17.495750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.495722    2563 status_manager.go:230] "Starting to sync pod status with apiserver"
May 29 20:47:17.495750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.495747    2563 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 29 20:47:17.495902 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.495754    2563 kubelet.go:2451] "Starting kubelet main sync loop"
May 29 20:47:17.495902 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.495805    2563 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 29 20:47:17.499246 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.499227    2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:17.520449 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.520420    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.521340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.521327    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.521423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.521355    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.521423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.521368    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.521423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.521395    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.530460 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.530440    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.530521 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.530465    2563 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-224.ec2.internal\": node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.549027 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.549006    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.596868 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.596847    2563 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"]
May 29 20:47:17.596928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.596915    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.598337 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.598324    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.598389 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.598351    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.598389 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.598360    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.599633 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.599621    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.599795 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.599782    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.599844 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.599809    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.600298 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600286    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.600365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600298    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.600365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600306    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.600365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600314    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.600365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600316    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.600508 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.600324    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.601586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.601573    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.601627 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.601598    2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 29 20:47:17.602254 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.602239    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientMemory"
May 29 20:47:17.602353 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.602265    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasNoDiskPressure"
May 29 20:47:17.602353 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.602278    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeHasSufficientPID"
May 29 20:47:17.620518 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.620498    2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-224.ec2.internal\" not found" node="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.624792 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.624779    2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-224.ec2.internal\" not found" node="ip-10-0-133-224.ec2.internal"
May 29 20:47:17.649588 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.649563    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.650732 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.650716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.650784 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.650744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.650784 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.650772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0654a00223f55ad393dab77c88c66323-config\") pod \"kube-apiserver-proxy-ip-10-0-133-224.ec2.internal\" (UID: \"0654a00223f55ad393dab77c88c66323\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.750260 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.750193    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.751349 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751329    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.751398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751360    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.751398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751374    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0654a00223f55ad393dab77c88c66323-config\") pod \"kube-apiserver-proxy-ip-10-0-133-224.ec2.internal\" (UID: \"0654a00223f55ad393dab77c88c66323\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.751460 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751411    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0654a00223f55ad393dab77c88c66323-config\") pod \"kube-apiserver-proxy-ip-10-0-133-224.ec2.internal\" (UID: \"0654a00223f55ad393dab77c88c66323\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.751460 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751444    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.751524 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.751477    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/898e01c6fc15efb05acced35ccdff3e1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal\" (UID: \"898e01c6fc15efb05acced35ccdff3e1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.850836 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.850779    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:17.924061 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.924034    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.929017 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:17.929001    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:17.951394 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:17.951377    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.051904 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.051812    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.149536 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.149507    2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:18.152716 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.152698    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.252223 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.252186    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 29 20:47:18.252852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.252332    2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 20:47:18.252852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.252362    2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 20:47:18.252852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.252365    2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 29 20:47:18.253283 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.253266    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.344974 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.344889    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-28 20:42:17 +0000 UTC" deadline="2028-02-27 16:23:37.528710026 +0000 UTC"
May 29 20:47:18.344974 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.344927    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15331h36m19.183786713s"
May 29 20:47:18.349183 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.349164    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 29 20:47:18.353701 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.353680    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.357463 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.357443    2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 29 20:47:18.377434 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.377417    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5rhgz"
May 29 20:47:18.385405 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.385386    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5rhgz"
May 29 20:47:18.385497 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:18.385383    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0654a00223f55ad393dab77c88c66323.slice/crio-baf49f46e4137606709a4471ab4c8298e3ee83b6c8e60e3fbabd04d2404eac38 WatchSource:0}: Error finding container baf49f46e4137606709a4471ab4c8298e3ee83b6c8e60e3fbabd04d2404eac38: Status 404 returned error can't find the container with id baf49f46e4137606709a4471ab4c8298e3ee83b6c8e60e3fbabd04d2404eac38
May 29 20:47:18.385640 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:18.385619    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898e01c6fc15efb05acced35ccdff3e1.slice/crio-f819629a39a746b4e29c0fcba41a04195adf2e4e6a3091536708b968eca6fa5a WatchSource:0}: Error finding container f819629a39a746b4e29c0fcba41a04195adf2e4e6a3091536708b968eca6fa5a: Status 404 returned error can't find the container with id f819629a39a746b4e29c0fcba41a04195adf2e4e6a3091536708b968eca6fa5a
May 29 20:47:18.388640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.388615    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 20:47:18.454185 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.454164    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.499107 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.499063    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal" event={"ID":"0654a00223f55ad393dab77c88c66323","Type":"ContainerStarted","Data":"baf49f46e4137606709a4471ab4c8298e3ee83b6c8e60e3fbabd04d2404eac38"}
May 29 20:47:18.499976 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.499954    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal" event={"ID":"898e01c6fc15efb05acced35ccdff3e1","Type":"ContainerStarted","Data":"f819629a39a746b4e29c0fcba41a04195adf2e4e6a3091536708b968eca6fa5a"}
May 29 20:47:18.554232 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.554198    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.654668 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.654613    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.755075 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.755043    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.855791 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:18.855761    2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-133-224.ec2.internal\" not found"
May 29 20:47:18.928527 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.928451    2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:18.949775 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.949736    2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal"
May 29 20:47:18.960971 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.960919    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 20:47:18.962318 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:18.962297    2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal"
May 29 20:47:19.066726 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.066701    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 29 20:47:19.323251 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.323213    2563 apiserver.go:52] "Watching apiserver"
May 29 20:47:19.329264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.329242    2563 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 29 20:47:19.330402 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.330380    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-5h2kj","kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qmmwq","openshift-dns/node-resolver-r6q6z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal","openshift-network-diagnostics/network-check-target-cbmss","kube-system/konnectivity-agent-6l5wp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9","openshift-image-registry/node-ca-5gtqz","openshift-multus/multus-additional-cni-plugins-5cpqr","openshift-multus/multus-bltbg","openshift-multus/network-metrics-daemon-k2wzp","openshift-network-operator/iptables-alerter-4svmt"]
May 29 20:47:19.331909 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.331881    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.334639 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.334620    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 29 20:47:19.334639 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.334631    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 29 20:47:19.334828 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.334659    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zrfhv\""
May 29 20:47:19.335422 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.335401    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.335514 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.335487    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.336802 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.336782    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:19.336903 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.336870    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:19.337633 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.337612    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 29 20:47:19.337731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.337643    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 29 20:47:19.337731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.337652    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8xklf\""
May 29 20:47:19.337731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.337713    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 29 20:47:19.337932 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.337916    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gsjzr\""
May 29 20:47:19.338162 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.338146    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.338162 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.338161    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 29 20:47:19.340202 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340181    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p8jsm\""
May 29 20:47:19.340301 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340231    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 29 20:47:19.340394 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340364    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 29 20:47:19.340591 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340573    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 29 20:47:19.341101 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340874    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 29 20:47:19.341101 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340893    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 29 20:47:19.341101 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.340896    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 29 20:47:19.341310 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.341102    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.343631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.343569    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 29 20:47:19.343631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.343608    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 29 20:47:19.343806 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.343727    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 29 20:47:19.343873 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.343856    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jtr52\""
May 29 20:47:19.344855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.343971    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.344855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.344219    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.345780 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.345764    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bltbg"
May 29 20:47:19.346285 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.346267    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.346551    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.346597    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m7ntb\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.346725    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.346755    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2kj4r\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347011    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347024    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 29 20:47:19.347180 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347051    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 29 20:47:19.347622 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347482    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.347622 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.347522    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:19.348444 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347872    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 29 20:47:19.348444 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.347873    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 29 20:47:19.348444 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.348037    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 29 20:47:19.348444 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.348215    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f8bsw\""
May 29 20:47:19.348794 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.348780    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.350112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.350097    2563 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 29 20:47:19.350928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.350873    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 29 20:47:19.351035 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.350944    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 29 20:47:19.351186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.351169    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vz5k6\""
May 29 20:47:19.351294 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.351272    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 29 20:47:19.358591 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-kubelet\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-var-lib-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-bin\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.358695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-netns\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-host\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-tuned\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-cni-binary-copy\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-etc-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-node-log\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-config\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.358927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gw4x\" (UniqueName: \"kubernetes.io/projected/72e3d4a4-0a37-4f38-964b-9209a64c31e7-kube-api-access-7gw4x\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-multus-daemon-config\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.358971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-multus-certs\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4bf76c9-473b-4f17-b287-10a0a52bbd27-iptables-alerter-script\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6px4\" (UniqueName: \"kubernetes.io/projected/d4bf76c9-473b-4f17-b287-10a0a52bbd27-kube-api-access-v6px4\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d05b663-a9d3-43a2-9d2e-42aee9296f71-konnectivity-ca\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b943e2cb-8681-47db-8d8f-da6eadc7afa9-serviceca\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-hostroot\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-ovn\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-cnibin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-os-release\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-bin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-multus\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d05b663-a9d3-43a2-9d2e-42aee9296f71-agent-certs\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-hosts-file\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.359387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-script-lib\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-kubelet\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kmqm\" (UniqueName: \"kubernetes.io/projected/0873cca3-a3cf-4320-b20f-55a5704aa6c5-kube-api-access-8kmqm\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-conf\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-netns\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq5zj\" (UniqueName: \"kubernetes.io/projected/72dda411-afde-441f-9f1e-37644e2961b7-kube-api-access-gq5zj\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cnibin\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-conf-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-tmp\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-systemd-units\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhd9x\" (UniqueName: \"kubernetes.io/projected/ee238d75-217d-4f53-98ea-6c4db8935526-kube-api-access-mhd9x\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-tmp-dir\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.360171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-slash\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-netd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l869j\" (UniqueName: \"kubernetes.io/projected/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-kube-api-access-l869j\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-sys-fs\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.359975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-socket-dir-parent\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-etc-kubernetes\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4bf76c9-473b-4f17-b287-10a0a52bbd27-host-slash\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-env-overrides\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysconfig\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-systemd\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-lib-modules\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovn-node-metrics-cert\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.360852 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-device-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-os-release\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-system-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-k8s-cni-cncf-io\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-run\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-systemd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-socket-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b943e2cb-8681-47db-8d8f-da6eadc7afa9-host\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-kubernetes\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-sys\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-var-lib-kubelet\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45cjn\" (UniqueName: \"kubernetes.io/projected/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-kube-api-access-45cjn\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdr8\" (UniqueName: \"kubernetes.io/projected/b943e2cb-8681-47db-8d8f-da6eadc7afa9-kube-api-access-2tdr8\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-modprobe-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-log-socket\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.361586 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-registration-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.362396 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.360784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx8fk\" (UniqueName: \"kubernetes.io/projected/4afc07e1-37d2-4e94-99a0-65109ff55be7-kube-api-access-tx8fk\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.386050 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.386020    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 20:42:18 +0000 UTC" deadline="2027-10-28 02:51:10.047112239 +0000 UTC"
May 29 20:47:19.386050 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.386049    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12390h3m50.66106731s"
May 29 20:47:19.389017 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.388978    2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:19.461114 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461079    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-bin\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461124    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461152    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-netns\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461178    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461185    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-bin\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461203    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-host\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461227    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-tuned\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.461259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461251    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-cni-binary-copy\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461275    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461302    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-etc-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461327    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-node-log\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461349    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-config\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461373    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7gw4x\" (UniqueName: \"kubernetes.io/projected/72e3d4a4-0a37-4f38-964b-9209a64c31e7-kube-api-access-7gw4x\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461399    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-multus-daemon-config\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461422    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-multus-certs\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461445    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.461497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461469    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4bf76c9-473b-4f17-b287-10a0a52bbd27-iptables-alerter-script\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461499    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v6px4\" (UniqueName: \"kubernetes.io/projected/d4bf76c9-473b-4f17-b287-10a0a52bbd27-kube-api-access-v6px4\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461523    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d05b663-a9d3-43a2-9d2e-42aee9296f71-konnectivity-ca\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461548    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b943e2cb-8681-47db-8d8f-da6eadc7afa9-serviceca\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461572    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461598    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-hostroot\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461621    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-ovn\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461652    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-cnibin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461677    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-os-release\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461700    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-bin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461713    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461724    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-multus\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461750    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d05b663-a9d3-43a2-9d2e-42aee9296f71-agent-certs\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461786    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-multus-certs\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461808    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-hosts-file\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.461841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461832    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461856    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-script-lib\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461880    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461901    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461924    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-kubelet\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461940    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8kmqm\" (UniqueName: \"kubernetes.io/projected/0873cca3-a3cf-4320-b20f-55a5704aa6c5-kube-api-access-8kmqm\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461956    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-conf\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.461975    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-netns\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462026    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gq5zj\" (UniqueName: \"kubernetes.io/projected/72dda411-afde-441f-9f1e-37644e2961b7-kube-api-access-gq5zj\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462055    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cnibin\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462080    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-conf-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462102    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-tmp\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462118    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-systemd-units\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462116    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462137    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mhd9x\" (UniqueName: \"kubernetes.io/projected/ee238d75-217d-4f53-98ea-6c4db8935526-kube-api-access-mhd9x\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462152    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-tmp-dir\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462172    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.462450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462192    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462215    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462238    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-slash\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462260    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-netd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462280    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l869j\" (UniqueName: \"kubernetes.io/projected/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-kube-api-access-l869j\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462300    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-sys-fs\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462315    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-socket-dir-parent\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462333    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-etc-kubernetes\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462363    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4bf76c9-473b-4f17-b287-10a0a52bbd27-host-slash\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462381    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462404    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-env-overrides\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462429    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462458    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysconfig\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462482    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-systemd\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462507    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-lib-modules\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462531    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovn-node-metrics-cert\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462557    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-device-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.463059 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462579    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-os-release\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462604    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-system-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462632    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-k8s-cni-cncf-io\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462643    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-host\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462656    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-run\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462681    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-systemd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462687    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-netns\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462708    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-socket-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462733    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b943e2cb-8681-47db-8d8f-da6eadc7afa9-host\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462753    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462762    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-kubernetes\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462786    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-sys\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462812    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-var-lib-kubelet\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462840    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462873    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45cjn\" (UniqueName: \"kubernetes.io/projected/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-kube-api-access-45cjn\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462899    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdr8\" (UniqueName: \"kubernetes.io/projected/b943e2cb-8681-47db-8d8f-da6eadc7afa9-kube-api-access-2tdr8\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462928    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-modprobe-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.463556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.462962    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-log-socket\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463014    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-registration-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463042    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tx8fk\" (UniqueName: \"kubernetes.io/projected/4afc07e1-37d2-4e94-99a0-65109ff55be7-kube-api-access-tx8fk\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463072    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-kubelet\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463100    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-var-lib-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463191    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-var-lib-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463246    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-hosts-file\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463267    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4bf76c9-473b-4f17-b287-10a0a52bbd27-iptables-alerter-script\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463356    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-multus-daemon-config\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463784    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d05b663-a9d3-43a2-9d2e-42aee9296f71-konnectivity-ca\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463824    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-script-lib\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463881    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.464024 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.463932    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ee238d75-217d-4f53-98ea-6c4db8935526-cni-binary-copy\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.464046    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464062    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b943e2cb-8681-47db-8d8f-da6eadc7afa9-serviceca\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.464128    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:19.964096974 +0000 UTC m=+3.028263112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464163    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464199    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-hostroot\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464239    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-ovn\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-cnibin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464305    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464326    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-etc-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464343    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-os-release\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464365    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-kubelet\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464388    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-node-log\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464439    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-multus\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.464544 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464488    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-systemd\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464587    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysctl-conf\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464624    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-run-netns\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464665    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-lib-modules\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464754    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovnkube-config\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464842    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-cnibin\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.464884    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-conf-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465021    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-sys\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465075    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-device-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465136    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-os-release\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465187    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-system-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-systemd-units\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465237    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-run-k8s-cni-cncf-io\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-systemd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-tmp-dir\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465540    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-socket-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465570    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-run\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465587    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b943e2cb-8681-47db-8d8f-da6eadc7afa9-host\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465633    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-kubernetes\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465676    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72e3d4a4-0a37-4f38-964b-9209a64c31e7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.465709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465713    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-modprobe-d\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465763    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-var-lib-kubelet\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465805    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.465888    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-sys-fs\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466065    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-socket-dir-parent\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466064    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-tuned\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-etc-kubernetes\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466142    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-slash\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466130    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72e3d4a4-0a37-4f38-964b-9209a64c31e7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466163    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4bf76c9-473b-4f17-b287-10a0a52bbd27-host-slash\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466191    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-multus-cni-dir\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466241    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466245    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-cni-netd\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466264 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4afc07e1-37d2-4e94-99a0-65109ff55be7-etc-sysconfig\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466278    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-run-openvswitch\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466352    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72dda411-afde-441f-9f1e-37644e2961b7-registration-dir\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466366    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d05b663-a9d3-43a2-9d2e-42aee9296f71-agent-certs\") pod \"konnectivity-agent-6l5wp\" (UID: \"9d05b663-a9d3-43a2-9d2e-42aee9296f71\") " pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466387    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee238d75-217d-4f53-98ea-6c4db8935526-host-var-lib-cni-bin\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466401    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-log-socket\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466437    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-host-kubelet\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.466855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.466529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-env-overrides\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.467287 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.467262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-ovn-node-metrics-cert\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.468228 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.468185    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4afc07e1-37d2-4e94-99a0-65109ff55be7-tmp\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.469170 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.469150    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:19.469248 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.469175    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:19.469248 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.469190    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:19.469335 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.469310    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:19.969289309 +0000 UTC m=+3.033455424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:19.471570 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.471523    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6px4\" (UniqueName: \"kubernetes.io/projected/d4bf76c9-473b-4f17-b287-10a0a52bbd27-kube-api-access-v6px4\") pod \"iptables-alerter-4svmt\" (UID: \"d4bf76c9-473b-4f17-b287-10a0a52bbd27\") " pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.476488 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.476457    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kmqm\" (UniqueName: \"kubernetes.io/projected/0873cca3-a3cf-4320-b20f-55a5704aa6c5-kube-api-access-8kmqm\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.476926 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.476872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx8fk\" (UniqueName: \"kubernetes.io/projected/4afc07e1-37d2-4e94-99a0-65109ff55be7-kube-api-access-tx8fk\") pod \"tuned-qmmwq\" (UID: \"4afc07e1-37d2-4e94-99a0-65109ff55be7\") " pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.477940 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.477917    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45cjn\" (UniqueName: \"kubernetes.io/projected/56d938c5-6d8c-4583-9cec-ebcc03a9d29e-kube-api-access-45cjn\") pod \"node-resolver-r6q6z\" (UID: \"56d938c5-6d8c-4583-9cec-ebcc03a9d29e\") " pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.478043 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.478013    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdr8\" (UniqueName: \"kubernetes.io/projected/b943e2cb-8681-47db-8d8f-da6eadc7afa9-kube-api-access-2tdr8\") pod \"node-ca-5gtqz\" (UID: \"b943e2cb-8681-47db-8d8f-da6eadc7afa9\") " pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.478401 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.478381    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gw4x\" (UniqueName: \"kubernetes.io/projected/72e3d4a4-0a37-4f38-964b-9209a64c31e7-kube-api-access-7gw4x\") pod \"multus-additional-cni-plugins-5cpqr\" (UID: \"72e3d4a4-0a37-4f38-964b-9209a64c31e7\") " pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.478585 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.478559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhd9x\" (UniqueName: \"kubernetes.io/projected/ee238d75-217d-4f53-98ea-6c4db8935526-kube-api-access-mhd9x\") pod \"multus-bltbg\" (UID: \"ee238d75-217d-4f53-98ea-6c4db8935526\") " pod="openshift-multus/multus-bltbg"
May 29 20:47:19.478712 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.478688    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq5zj\" (UniqueName: \"kubernetes.io/projected/72dda411-afde-441f-9f1e-37644e2961b7-kube-api-access-gq5zj\") pod \"aws-ebs-csi-driver-node-5qmt9\" (UID: \"72dda411-afde-441f-9f1e-37644e2961b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.479488 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.479465    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l869j\" (UniqueName: \"kubernetes.io/projected/51351a92-8ecd-4bdd-8d04-7b6ea82516d8-kube-api-access-l869j\") pod \"ovnkube-node-5h2kj\" (UID: \"51351a92-8ecd-4bdd-8d04-7b6ea82516d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.523836 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.523811    2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:19.644433 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.644345    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:19.651251 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.651228    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmmwq"
May 29 20:47:19.659865 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.659847    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r6q6z"
May 29 20:47:19.664647 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.664627    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:19.674126 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.674101    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9"
May 29 20:47:19.680636 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.680607    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gtqz"
May 29 20:47:19.687395 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.687374    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5cpqr"
May 29 20:47:19.694020 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.693984    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bltbg"
May 29 20:47:19.699558 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.699537    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4svmt"
May 29 20:47:19.927961 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.927836    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb943e2cb_8681_47db_8d8f_da6eadc7afa9.slice/crio-54d04c71257478b048aeb6aaacc41ea3c4fc9b7064837d2ce96524253fe95f1f WatchSource:0}: Error finding container 54d04c71257478b048aeb6aaacc41ea3c4fc9b7064837d2ce96524253fe95f1f: Status 404 returned error can't find the container with id 54d04c71257478b048aeb6aaacc41ea3c4fc9b7064837d2ce96524253fe95f1f
May 29 20:47:19.931970 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.931946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d05b663_a9d3_43a2_9d2e_42aee9296f71.slice/crio-c6360d7e4c04f51f678a14620b909c9da307ff50c2483f92319d5e32a3739351 WatchSource:0}: Error finding container c6360d7e4c04f51f678a14620b909c9da307ff50c2483f92319d5e32a3739351: Status 404 returned error can't find the container with id c6360d7e4c04f51f678a14620b909c9da307ff50c2483f92319d5e32a3739351
May 29 20:47:19.933326 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.933206    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4bf76c9_473b_4f17_b287_10a0a52bbd27.slice/crio-0fb1af08dd89d403e68b924262aec907ca177032d6a7ff07ac8ab1d229a8c289 WatchSource:0}: Error finding container 0fb1af08dd89d403e68b924262aec907ca177032d6a7ff07ac8ab1d229a8c289: Status 404 returned error can't find the container with id 0fb1af08dd89d403e68b924262aec907ca177032d6a7ff07ac8ab1d229a8c289
May 29 20:47:19.934230 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.934206    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51351a92_8ecd_4bdd_8d04_7b6ea82516d8.slice/crio-f2d91951d5b83e5e210ab850637129ed1036a3e0d0ee3376ff27130071946a82 WatchSource:0}: Error finding container f2d91951d5b83e5e210ab850637129ed1036a3e0d0ee3376ff27130071946a82: Status 404 returned error can't find the container with id f2d91951d5b83e5e210ab850637129ed1036a3e0d0ee3376ff27130071946a82
May 29 20:47:19.935150 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.935093    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e3d4a4_0a37_4f38_964b_9209a64c31e7.slice/crio-5763e67d7453ad0aad14bdfb1e075771c5eb5dc5c97a2e061af98edca7070996 WatchSource:0}: Error finding container 5763e67d7453ad0aad14bdfb1e075771c5eb5dc5c97a2e061af98edca7070996: Status 404 returned error can't find the container with id 5763e67d7453ad0aad14bdfb1e075771c5eb5dc5c97a2e061af98edca7070996
May 29 20:47:19.935633 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.935609    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afc07e1_37d2_4e94_99a0_65109ff55be7.slice/crio-2655d2be0f29c0ad9431b8d794ce92350e8616ddcc33a62c4e0ea01497714d16 WatchSource:0}: Error finding container 2655d2be0f29c0ad9431b8d794ce92350e8616ddcc33a62c4e0ea01497714d16: Status 404 returned error can't find the container with id 2655d2be0f29c0ad9431b8d794ce92350e8616ddcc33a62c4e0ea01497714d16
May 29 20:47:19.936709 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.936675    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee238d75_217d_4f53_98ea_6c4db8935526.slice/crio-06b878992d0fdd5d9f184a9a4f138815cd04e53d7d85c2606c23fb8a27e081d4 WatchSource:0}: Error finding container 06b878992d0fdd5d9f184a9a4f138815cd04e53d7d85c2606c23fb8a27e081d4: Status 404 returned error can't find the container with id 06b878992d0fdd5d9f184a9a4f138815cd04e53d7d85c2606c23fb8a27e081d4
May 29 20:47:19.939368 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:47:19.938642    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dda411_afde_441f_9f1e_37644e2961b7.slice/crio-889a1c61e32dde842c7fefd5abf1e4e70e89b2cd85708abae487b935acdcb093 WatchSource:0}: Error finding container 889a1c61e32dde842c7fefd5abf1e4e70e89b2cd85708abae487b935acdcb093: Status 404 returned error can't find the container with id 889a1c61e32dde842c7fefd5abf1e4e70e89b2cd85708abae487b935acdcb093
May 29 20:47:19.966439 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:19.966419    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:19.966561 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.966544    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:19.966611 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:19.966600    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:20.966586285 +0000 UTC m=+4.030752400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:20.066742 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.066709    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:20.066879 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.066865    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:20.066928 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.066882    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:20.066928 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.066893    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:20.067010 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.066944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:21.06692692 +0000 UTC m=+4.131093037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:20.386838 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.386792    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-28 20:42:18 +0000 UTC" deadline="2027-11-22 10:00:53.045751919 +0000 UTC"
May 29 20:47:20.386838 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.386830    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12997h13m32.658926024s"
May 29 20:47:20.506942 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.506876    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bltbg" event={"ID":"ee238d75-217d-4f53-98ea-6c4db8935526","Type":"ContainerStarted","Data":"06b878992d0fdd5d9f184a9a4f138815cd04e53d7d85c2606c23fb8a27e081d4"}
May 29 20:47:20.516428 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.516346    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4svmt" event={"ID":"d4bf76c9-473b-4f17-b287-10a0a52bbd27","Type":"ContainerStarted","Data":"0fb1af08dd89d403e68b924262aec907ca177032d6a7ff07ac8ab1d229a8c289"}
May 29 20:47:20.522155 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.522117    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerStarted","Data":"5763e67d7453ad0aad14bdfb1e075771c5eb5dc5c97a2e061af98edca7070996"}
May 29 20:47:20.526242 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.526063    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"f2d91951d5b83e5e210ab850637129ed1036a3e0d0ee3376ff27130071946a82"}
May 29 20:47:20.530561 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.530532    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l5wp" event={"ID":"9d05b663-a9d3-43a2-9d2e-42aee9296f71","Type":"ContainerStarted","Data":"c6360d7e4c04f51f678a14620b909c9da307ff50c2483f92319d5e32a3739351"}
May 29 20:47:20.543453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.542791    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal" event={"ID":"0654a00223f55ad393dab77c88c66323","Type":"ContainerStarted","Data":"35eb22a0005231ae57bac40951f4e34314521a188da014b072e07980255a5766"}
May 29 20:47:20.552441 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.552208    2563 generic.go:358] "Generic (PLEG): container finished" podID="898e01c6fc15efb05acced35ccdff3e1" containerID="ebf08fa09069cf04ee0ef0c5036f674d09ff2c01c8fe1b9c66c4821ea6aa4f99" exitCode=0
May 29 20:47:20.552611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.552466    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal" event={"ID":"898e01c6fc15efb05acced35ccdff3e1","Type":"ContainerDied","Data":"ebf08fa09069cf04ee0ef0c5036f674d09ff2c01c8fe1b9c66c4821ea6aa4f99"}
May 29 20:47:20.561350 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.561322    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r6q6z" event={"ID":"56d938c5-6d8c-4583-9cec-ebcc03a9d29e","Type":"ContainerStarted","Data":"cfc635bcfa0f3faedafdc40ca68fdbc3dc1946cd848d89030c4ac862d09a5f5a"}
May 29 20:47:20.570678 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.570613    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmmwq" event={"ID":"4afc07e1-37d2-4e94-99a0-65109ff55be7","Type":"ContainerStarted","Data":"2655d2be0f29c0ad9431b8d794ce92350e8616ddcc33a62c4e0ea01497714d16"}
May 29 20:47:20.575347 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.575301    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-224.ec2.internal" podStartSLOduration=2.575285569 podStartE2EDuration="2.575285569s" podCreationTimestamp="2026-05-29 20:47:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:47:20.557877783 +0000 UTC m=+3.622043921" watchObservedRunningTime="2026-05-29 20:47:20.575285569 +0000 UTC m=+3.639451707"
May 29 20:47:20.579673 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.579644    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gtqz" event={"ID":"b943e2cb-8681-47db-8d8f-da6eadc7afa9","Type":"ContainerStarted","Data":"54d04c71257478b048aeb6aaacc41ea3c4fc9b7064837d2ce96524253fe95f1f"}
May 29 20:47:20.584778 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.584753    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9" event={"ID":"72dda411-afde-441f-9f1e-37644e2961b7","Type":"ContainerStarted","Data":"889a1c61e32dde842c7fefd5abf1e4e70e89b2cd85708abae487b935acdcb093"}
May 29 20:47:20.972491 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:20.972455    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:20.972697 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.972613    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:20.972697 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:20.972678    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:22.972658878 +0000 UTC m=+6.036824996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:21.073363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:21.073274    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:21.073532 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.073468    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:21.073532 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.073488    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:21.073532 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.073501    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:21.073678 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.073562    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:23.073542308 +0000 UTC m=+6.137708428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:21.186349 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:21.186320    2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 29 20:47:21.497183 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:21.497151    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:21.497663 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.497296    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:21.497790 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:21.497731    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:21.497867 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:21.497827    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:21.610753 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:21.610612    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal" event={"ID":"898e01c6fc15efb05acced35ccdff3e1","Type":"ContainerStarted","Data":"7435f61cd966403d29ad299c4579d0e12edbff354b1bda928793c467804d84aa"}
May 29 20:47:22.990875 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:22.990300    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:22.990875 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:22.990452    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:22.990875 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:22.990516    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:26.990496529 +0000 UTC m=+10.054662653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:23.092074 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:23.091429    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:23.092074 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.091590    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:23.092074 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.091608    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:23.092074 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.091621    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:23.092074 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.091680    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:27.091660181 +0000 UTC m=+10.155826300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:23.497820 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:23.497759    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:23.498010 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.497880    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:23.498242 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:23.498222    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:23.498337 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:23.498309    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:25.498635 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:25.498602    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:25.499063 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:25.498725    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:25.499135 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:25.499119    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:25.499252 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:25.499220    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:27.026113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:27.026077    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:27.026580 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.026238    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:27.026580 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.026301    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:35.026282462 +0000 UTC m=+18.090448595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:27.127041 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:27.126985    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:27.127201 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.127144    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:27.127201 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.127174    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:27.127201 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.127192    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:27.127346 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.127251    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:35.127232332 +0000 UTC m=+18.191398453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:27.498581 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:27.498109    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:27.498581 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.498238    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:27.498581 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:27.498109    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:27.498581 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:27.498346    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:29.496762 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:29.496714    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:29.497233 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:29.496847    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:29.497325 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:29.496719    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:29.497386 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:29.497351    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:31.497004 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:31.496954    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:31.497452 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:31.497096    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:31.497452 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:31.496954    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:31.497452 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:31.497226    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:33.496750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:33.496722    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:33.497103 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:33.496722    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:33.497103 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:33.496832    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:33.497103 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:33.496895    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:35.084044 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:35.083975    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:35.084437 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.084124    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:35.084437 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.084205    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:51.084183071 +0000 UTC m=+34.148349191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:35.184815 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:35.184775    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:35.185039 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.184969    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:35.185091 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.185047    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:35.185091 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.185062    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:35.185177 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.185128    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:47:51.185107003 +0000 UTC m=+34.249273137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:35.496658 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:35.496630    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:35.496853 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.496759    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:35.496853 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:35.496814    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:35.496967 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:35.496940    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:37.498726 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:37.498689    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:37.498726 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:37.498715    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:37.499226 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:37.498838    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:37.499226 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:37.498945    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:38.650931 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.650678    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmmwq" event={"ID":"4afc07e1-37d2-4e94-99a0-65109ff55be7","Type":"ContainerStarted","Data":"d00459c3f9b7b667e22b0eb5edf134fd9ecd00676e0299fb7d820b3bd10abe78"}
May 29 20:47:38.652936 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.652910    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gtqz" event={"ID":"b943e2cb-8681-47db-8d8f-da6eadc7afa9","Type":"ContainerStarted","Data":"b66cf21d15e931212412c68adcfe8b2fd74dfdf41aa46356d2875267e17f7f94"}
May 29 20:47:38.654665 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.654633    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9" event={"ID":"72dda411-afde-441f-9f1e-37644e2961b7","Type":"ContainerStarted","Data":"fcbc7bfda159ccb01953f27b2e5e53a95cc3242fdeef8280867f897b349abe2e"}
May 29 20:47:38.656131 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.656108    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bltbg" event={"ID":"ee238d75-217d-4f53-98ea-6c4db8935526","Type":"ContainerStarted","Data":"e0cafabc0171cda76a25f29648b6304c5d7d44dd81243f34eb6ae4690db270dc"}
May 29 20:47:38.661596 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.659367    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerStarted","Data":"16eea991967ffcedf8238367b3bb689e2e641ff86852f92fbad9c0831227da09"}
May 29 20:47:38.661596 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.661430    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:47:38.662358 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.662333    2563 generic.go:358] "Generic (PLEG): container finished" podID="51351a92-8ecd-4bdd-8d04-7b6ea82516d8" containerID="e68920220c6ddad42024b0cc4a3d298b57739e9bc3205ce98cefd5116504953c" exitCode=1
May 29 20:47:38.662469 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.662364    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerDied","Data":"e68920220c6ddad42024b0cc4a3d298b57739e9bc3205ce98cefd5116504953c"}
May 29 20:47:38.662469 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.662400    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"23b1abfd339cead953fe6953d731da8de15c3a4748c351e8340f832e38e98f2c"}
May 29 20:47:38.663818 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.663794    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6l5wp" event={"ID":"9d05b663-a9d3-43a2-9d2e-42aee9296f71","Type":"ContainerStarted","Data":"706f275d4f210b4a25ec3a0e7270eb4248a5d30d17decf78e37d195934eb44cf"}
May 29 20:47:38.665040 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.665017    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r6q6z" event={"ID":"56d938c5-6d8c-4583-9cec-ebcc03a9d29e","Type":"ContainerStarted","Data":"1ca44c8d0c4cd6491c62b072fa91cef2b1ca9099e93ae9c7bbd42edd4c866b26"}
May 29 20:47:38.669192 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.669145    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qmmwq" podStartSLOduration=3.391400945 podStartE2EDuration="21.669131071s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.9388837 +0000 UTC m=+3.003049832" lastFinishedPulling="2026-05-29 20:47:38.216613826 +0000 UTC m=+21.280779958" observedRunningTime="2026-05-29 20:47:38.66823575 +0000 UTC m=+21.732401887" watchObservedRunningTime="2026-05-29 20:47:38.669131071 +0000 UTC m=+21.733297209"
May 29 20:47:38.669367 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.669337    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-224.ec2.internal" podStartSLOduration=19.669330828 podStartE2EDuration="19.669330828s" podCreationTimestamp="2026-05-29 20:47:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:47:21.628338026 +0000 UTC m=+4.692504164" watchObservedRunningTime="2026-05-29 20:47:38.669330828 +0000 UTC m=+21.733497046"
May 29 20:47:38.705187 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.705145    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-bltbg" podStartSLOduration=3.423514598 podStartE2EDuration="21.705131281s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.938907458 +0000 UTC m=+3.003073575" lastFinishedPulling="2026-05-29 20:47:38.220524129 +0000 UTC m=+21.284690258" observedRunningTime="2026-05-29 20:47:38.705061029 +0000 UTC m=+21.769227167" watchObservedRunningTime="2026-05-29 20:47:38.705131281 +0000 UTC m=+21.769297417"
May 29 20:47:38.736917 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.736867    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-r6q6z" podStartSLOduration=3.465241947 podStartE2EDuration="21.736850623s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.942567785 +0000 UTC m=+3.006733904" lastFinishedPulling="2026-05-29 20:47:38.214176451 +0000 UTC m=+21.278342580" observedRunningTime="2026-05-29 20:47:38.736415732 +0000 UTC m=+21.800581868" watchObservedRunningTime="2026-05-29 20:47:38.736850623 +0000 UTC m=+21.801016757"
May 29 20:47:38.737104 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.737054    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6l5wp" podStartSLOduration=3.454747162 podStartE2EDuration="21.737044829s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.933543628 +0000 UTC m=+2.997709757" lastFinishedPulling="2026-05-29 20:47:38.215841306 +0000 UTC m=+21.280007424" observedRunningTime="2026-05-29 20:47:38.721780448 +0000 UTC m=+21.785946584" watchObservedRunningTime="2026-05-29 20:47:38.737044829 +0000 UTC m=+21.801210968"
May 29 20:47:38.749628 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:38.749581    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5gtqz" podStartSLOduration=11.547532783 podStartE2EDuration="21.749568795s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.92993805 +0000 UTC m=+2.994104165" lastFinishedPulling="2026-05-29 20:47:30.131974044 +0000 UTC m=+13.196140177" observedRunningTime="2026-05-29 20:47:38.748946247 +0000 UTC m=+21.813112386" watchObservedRunningTime="2026-05-29 20:47:38.749568795 +0000 UTC m=+21.813734933"
May 29 20:47:39.496303 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.496132    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:39.496587 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.496179    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:39.496587 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:39.496410    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:39.496587 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:39.496447    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:39.669019 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.668967    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4svmt" event={"ID":"d4bf76c9-473b-4f17-b287-10a0a52bbd27","Type":"ContainerStarted","Data":"9a29f43dd20fd8dc6b368a4b64b16c7cfe880fffdbc32bf4477874e95347de0f"}
May 29 20:47:39.670267 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.670241    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="16eea991967ffcedf8238367b3bb689e2e641ff86852f92fbad9c0831227da09" exitCode=0
May 29 20:47:39.670365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.670306    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"16eea991967ffcedf8238367b3bb689e2e641ff86852f92fbad9c0831227da09"}
May 29 20:47:39.672903 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.672887    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:47:39.673340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.673274    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"39f6ad4e8045f2c22cc6256b1dcc11a75d5e6a8a28a1f8449b04ddbad0043cb3"}
May 29 20:47:39.673340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.673305    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"ccf370e6aaf4110ccf6f91aa12d59a210e1df65ef33f50405931ac350cd0d210"}
May 29 20:47:39.673340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.673316    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"89ada819221ba3ea5e137c062324cce7ff1c58af59234427392cb2cae2f470d6"}
May 29 20:47:39.673340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.673324    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"ecf98b4cef27479c61b38c771254d12e17e582a282ab2af5a195b43f684a3943"}
May 29 20:47:39.685928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.685770    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4svmt" podStartSLOduration=4.406503551 podStartE2EDuration="22.685755092s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.935392173 +0000 UTC m=+2.999558301" lastFinishedPulling="2026-05-29 20:47:38.214643727 +0000 UTC m=+21.278809842" observedRunningTime="2026-05-29 20:47:39.685612616 +0000 UTC m=+22.749778753" watchObservedRunningTime="2026-05-29 20:47:39.685755092 +0000 UTC m=+22.749921228"
May 29 20:47:39.832091 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:39.831917    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 29 20:47:40.429695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:40.429590    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-29T20:47:39.832086143Z","UUID":"98f6b47e-dd8f-41cf-9455-b08c0f031c85","Handler":null,"Name":"","Endpoint":""}
May 29 20:47:40.433234 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:40.433201    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 29 20:47:40.433234 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:40.433238    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 29 20:47:40.678685 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:40.678638    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9" event={"ID":"72dda411-afde-441f-9f1e-37644e2961b7","Type":"ContainerStarted","Data":"4d010fc4201930fe9da150f84be2c30d44c617b313085431df75c387fa63c486"}
May 29 20:47:41.496131 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.496101    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:41.496318 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.496112    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:41.496318 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:41.496208    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:41.496318 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:41.496285    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:41.682831 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.682806    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:47:41.683293 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.683221    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"9f9d2fd675dfa341c2187aa820020dfec421259e31202be32ed5cf7d0a670fc1"}
May 29 20:47:41.685109 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.685080    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9" event={"ID":"72dda411-afde-441f-9f1e-37644e2961b7","Type":"ContainerStarted","Data":"56f5784210b941fd92b00d4ceba9dcd6660bc87b973e7c16e6128c6734be8fc1"}
May 29 20:47:41.702293 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:41.702245    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5qmt9" podStartSLOduration=3.664547357 podStartE2EDuration="24.702230073s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.940465524 +0000 UTC m=+3.004631652" lastFinishedPulling="2026-05-29 20:47:40.978148251 +0000 UTC m=+24.042314368" observedRunningTime="2026-05-29 20:47:41.701582599 +0000 UTC m=+24.765748738" watchObservedRunningTime="2026-05-29 20:47:41.702230073 +0000 UTC m=+24.766396210"
May 29 20:47:42.871471 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:42.871426    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:42.872350 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:42.872321    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:43.496835 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:43.496794    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:43.497057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:43.496833    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:43.497057 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:43.496938    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:43.497178 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:43.497072    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:44.691342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.691135    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="f9ff2f3789b701cbdcf93e11eba8d5ea4ffcbdb30c0e57463349ebe0dc837844" exitCode=0
May 29 20:47:44.691342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.691192    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"f9ff2f3789b701cbdcf93e11eba8d5ea4ffcbdb30c0e57463349ebe0dc837844"}
May 29 20:47:44.694250 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.694229    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:47:44.694547 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.694528    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"256b7564d28040124cc8bdf85510c13642f249ab5a9a947abc01af9339dcb2b9"}
May 29 20:47:44.694891 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.694873    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:44.694933 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.694901    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:44.694933 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.694913    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:44.695037 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.695029    2563 scope.go:117] "RemoveContainer" containerID="e68920220c6ddad42024b0cc4a3d298b57739e9bc3205ce98cefd5116504953c"
May 29 20:47:44.711697 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.711682    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:44.712812 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:44.712797    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:47:45.496608 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.496578    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:45.496752 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:45.496693    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:45.496827 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.496743    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:45.496886 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:45.496866    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:45.698587 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.698557    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="3e2902042658aa42bb85d852e66cc1e411e91dad6b8d4f282fba75203dbac748" exitCode=0
May 29 20:47:45.699029 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.698626    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"3e2902042658aa42bb85d852e66cc1e411e91dad6b8d4f282fba75203dbac748"}
May 29 20:47:45.702088 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.702071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:47:45.702482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.702458    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" event={"ID":"51351a92-8ecd-4bdd-8d04-7b6ea82516d8","Type":"ContainerStarted","Data":"e38b20cd5aa633b8ba17ba2bb16dbde172569437d4087ffd3b67f909cdb94fed"}
May 29 20:47:45.754182 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.754124    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj" podStartSLOduration=10.430732078 podStartE2EDuration="28.75410742s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.936493654 +0000 UTC m=+3.000659788" lastFinishedPulling="2026-05-29 20:47:38.259869 +0000 UTC m=+21.324035130" observedRunningTime="2026-05-29 20:47:45.753752165 +0000 UTC m=+28.817918327" watchObservedRunningTime="2026-05-29 20:47:45.75410742 +0000 UTC m=+28.818273577"
May 29 20:47:45.778056 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.778022    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:45.778192 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.778184    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 29 20:47:45.778756 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:45.778730    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6l5wp"
May 29 20:47:46.057527 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.057336    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cbmss"]
May 29 20:47:46.057752 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.057620    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:46.057752 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:46.057723    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:46.058027 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.058003    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k2wzp"]
May 29 20:47:46.058119 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.058105    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:46.058197 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:46.058180    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:46.705977 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.705899    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="6cb676fdb1aa8e80fa6cdd5cd15f3881cd8d20d9ed55958d4225be04d2347cdb" exitCode=0
May 29 20:47:46.705977 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:46.705955    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"6cb676fdb1aa8e80fa6cdd5cd15f3881cd8d20d9ed55958d4225be04d2347cdb"}
May 29 20:47:47.497734 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:47.497707    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:47.497853 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:47.497792    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:47.497853 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:47.497834    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:47.497948 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:47.497886    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:49.496395 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:49.496354    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:49.496395 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:49.496395    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:49.497118 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:49.496489    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cbmss" podUID="37983236-a8c3-4c96-a34d-d7f878479929"
May 29 20:47:49.497118 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:49.496633    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2wzp" podUID="0873cca3-a3cf-4320-b20f-55a5704aa6c5"
May 29 20:47:51.108378 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.108164    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:51.108885 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.108330    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:51.108885 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.108464    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:48:23.108438943 +0000 UTC m=+66.172605143 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 29 20:47:51.209496 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.209409    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:51.209647 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.209584    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 29 20:47:51.209647 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.209607    2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 29 20:47:51.209647 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.209620    2563 projected.go:194] Error preparing data for projected volume kube-api-access-qvb4g for pod openshift-network-diagnostics/network-check-target-cbmss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:51.209775 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.209677    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g podName:37983236-a8c3-4c96-a34d-d7f878479929 nodeName:}" failed. No retries permitted until 2026-05-29 20:48:23.209660135 +0000 UTC m=+66.273826250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qvb4g" (UniqueName: "kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g") pod "network-check-target-cbmss" (UID: "37983236-a8c3-4c96-a34d-d7f878479929") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 29 20:47:51.273654 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.273624    2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-133-224.ec2.internal" event="NodeReady"
May 29 20:47:51.273809 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.273758    2563 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 29 20:47:51.320222 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.320189    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jdb2d"]
May 29 20:47:51.323149 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.323124    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.323723 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.323687    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q689n"]
May 29 20:47:51.325806 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.325532    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hbwz\""
May 29 20:47:51.325806 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.325650    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 29 20:47:51.325806 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.325686    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 29 20:47:51.325806 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.325734    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:51.328178 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.328157    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 29 20:47:51.328695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.328487    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5d94t\""
May 29 20:47:51.328695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.328500    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 29 20:47:51.328695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.328537    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 29 20:47:51.333891 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.333870    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdb2d"]
May 29 20:47:51.337547 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.337507    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q689n"]
May 29 20:47:51.411095 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc78g\" (UniqueName: \"kubernetes.io/projected/f74cd5ec-fc29-41c8-8986-230ca8c7759a-kube-api-access-pc78g\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:51.411095 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbf2z\" (UniqueName: \"kubernetes.io/projected/7d207e7d-a860-433d-ba4f-67fcd022b23f-kube-api-access-nbf2z\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.411326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d207e7d-a860-433d-ba4f-67fcd022b23f-tmp-dir\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.411326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:51.411326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d207e7d-a860-433d-ba4f-67fcd022b23f-config-volume\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.411487 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.411327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.496552 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.496515    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:47:51.496552 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.496550    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:47:51.499398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.499195    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 20:47:51.499398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.499200    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 20:47:51.499398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.499260    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tklm4\""
May 29 20:47:51.499398 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.499307    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 20:47:51.499658 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.499639    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdrr8\""
May 29 20:47:51.511704 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511679    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pc78g\" (UniqueName: \"kubernetes.io/projected/f74cd5ec-fc29-41c8-8986-230ca8c7759a-kube-api-access-pc78g\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:51.511812 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511728    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbf2z\" (UniqueName: \"kubernetes.io/projected/7d207e7d-a860-433d-ba4f-67fcd022b23f-kube-api-access-nbf2z\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.511812 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511754    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d207e7d-a860-433d-ba4f-67fcd022b23f-tmp-dir\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.511812 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511805    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:51.511973 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511852    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d207e7d-a860-433d-ba4f-67fcd022b23f-config-volume\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.511973 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.511879    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.511973 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.511955    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:47:51.511973 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.511967    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:47:51.512196 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.512036    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:47:52.012017982 +0000 UTC m=+35.076184099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:47:51.512196 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:51.512052    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:47:52.012044106 +0000 UTC m=+35.076210227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:47:51.512317 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.512294    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d207e7d-a860-433d-ba4f-67fcd022b23f-tmp-dir\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.512637 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.512612    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d207e7d-a860-433d-ba4f-67fcd022b23f-config-volume\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.522605 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.522582    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbf2z\" (UniqueName: \"kubernetes.io/projected/7d207e7d-a860-433d-ba4f-67fcd022b23f-kube-api-access-nbf2z\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:51.522714 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:51.522626    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc78g\" (UniqueName: \"kubernetes.io/projected/f74cd5ec-fc29-41c8-8986-230ca8c7759a-kube-api-access-pc78g\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:52.015723 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:52.015689    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:52.015903 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:52.015745    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:52.015903 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:52.015857    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:47:52.015903 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:52.015878    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:47:52.016088 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:52.015923    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:47:53.015905572 +0000 UTC m=+36.080071704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:47:52.016088 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:52.015955    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:47:53.015934153 +0000 UTC m=+36.080100303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:47:53.021985 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:53.021957    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:53.022326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:53.022046    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:53.022326 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:53.022125    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:47:53.022326 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:53.022188    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:47:53.022326 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:53.022203    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:47:55.022182384 +0000 UTC m=+38.086348522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:47:53.022326 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:53.022239    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:47:55.02222636 +0000 UTC m=+38.086392474 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:47:53.721188 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:53.721152    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="6533c530c71e8acad410f4ff61f242e0f6310d425a7b0865df59f21cc64933db" exitCode=0
May 29 20:47:53.721338 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:53.721207    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"6533c530c71e8acad410f4ff61f242e0f6310d425a7b0865df59f21cc64933db"}
May 29 20:47:54.725538 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:54.725508    2563 generic.go:358] "Generic (PLEG): container finished" podID="72e3d4a4-0a37-4f38-964b-9209a64c31e7" containerID="6640d1d709dc4ac7d1e9f3155302281c4e3ae6658f29b88463016674d9bf59d7" exitCode=0
May 29 20:47:54.725963 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:54.725559    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerDied","Data":"6640d1d709dc4ac7d1e9f3155302281c4e3ae6658f29b88463016674d9bf59d7"}
May 29 20:47:55.037236 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:55.037152    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:55.037236 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:55.037205    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:55.037450 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:55.037295    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:47:55.037450 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:55.037300    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:47:55.037450 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:55.037364    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:47:59.037345282 +0000 UTC m=+42.101511403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:47:55.037450 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:55.037387    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:47:59.037380023 +0000 UTC m=+42.101546141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:47:55.730383 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:55.730347    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" event={"ID":"72e3d4a4-0a37-4f38-964b-9209a64c31e7","Type":"ContainerStarted","Data":"221454457e226fa7237abaab49f36eb286f627ccfd4741de5aea093ad0782b3d"}
May 29 20:47:55.763797 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:55.763742    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5cpqr" podStartSLOduration=5.9639693000000005 podStartE2EDuration="38.763727563s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:47:19.937969307 +0000 UTC m=+3.002135433" lastFinishedPulling="2026-05-29 20:47:52.737727578 +0000 UTC m=+35.801893696" observedRunningTime="2026-05-29 20:47:55.762528427 +0000 UTC m=+38.826694564" watchObservedRunningTime="2026-05-29 20:47:55.763727563 +0000 UTC m=+38.827893700"
May 29 20:47:59.062516 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:59.062480    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:47:59.062516 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:47:59.062523    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:47:59.062957 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:59.062627    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:47:59.062957 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:59.062638    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:47:59.062957 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:59.062684    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:48:07.062667628 +0000 UTC m=+50.126833754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:47:59.062957 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:47:59.062707    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:48:07.062690104 +0000 UTC m=+50.126856233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:48:07.119571 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:07.119530    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:48:07.120041 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:07.119583    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:48:07.120041 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:07.119678    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:48:07.120041 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:07.119682    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:48:07.120041 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:07.119745    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:48:23.119730429 +0000 UTC m=+66.183896545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:48:07.120041 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:07.119758    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:48:23.119752693 +0000 UTC m=+66.183918808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:48:16.717557 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:16.717530    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5h2kj"
May 29 20:48:23.132622 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.132582    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:48:23.132622 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.132623    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:48:23.133131 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.132669    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:48:23.133131 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.132752    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:48:23.133131 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.132788    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:48:23.133131 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.132817    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:48:55.132799679 +0000 UTC m=+98.196965808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:48:23.133131 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.132849    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:48:55.132835098 +0000 UTC m=+98.197001213 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:48:23.135498 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.135481    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 29 20:48:23.143606 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.143590    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 29 20:48:23.143654 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:23.143630    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs podName:0873cca3-a3cf-4320-b20f-55a5704aa6c5 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:27.143619347 +0000 UTC m=+130.207785462 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs") pod "network-metrics-daemon-k2wzp" (UID: "0873cca3-a3cf-4320-b20f-55a5704aa6c5") : secret "metrics-daemon-secret" not found
May 29 20:48:23.233324 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.233283    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:48:23.235910 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.235895    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 29 20:48:23.251532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.251512    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 29 20:48:23.258335 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.258309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvb4g\" (UniqueName: \"kubernetes.io/projected/37983236-a8c3-4c96-a34d-d7f878479929-kube-api-access-qvb4g\") pod \"network-check-target-cbmss\" (UID: \"37983236-a8c3-4c96-a34d-d7f878479929\") " pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:48:23.317171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.317140    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qdrr8\""
May 29 20:48:23.324634 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.324616    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:48:23.514739 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.514707    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cbmss"]
May 29 20:48:23.518901 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:48:23.518869    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37983236_a8c3_4c96_a34d_d7f878479929.slice/crio-cdd131646e2052147fecdde48ca96bdad1fbc9f7727c2098eb44949419a1d982 WatchSource:0}: Error finding container cdd131646e2052147fecdde48ca96bdad1fbc9f7727c2098eb44949419a1d982: Status 404 returned error can't find the container with id cdd131646e2052147fecdde48ca96bdad1fbc9f7727c2098eb44949419a1d982
May 29 20:48:23.783234 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:23.783202    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cbmss" event={"ID":"37983236-a8c3-4c96-a34d-d7f878479929","Type":"ContainerStarted","Data":"cdd131646e2052147fecdde48ca96bdad1fbc9f7727c2098eb44949419a1d982"}
May 29 20:48:26.790208 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:26.790166    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cbmss" event={"ID":"37983236-a8c3-4c96-a34d-d7f878479929","Type":"ContainerStarted","Data":"996c66a2b2fa878272959b1de21337cdd5ebeca7b9cf78c11f05175ddd41f7c0"}
May 29 20:48:26.790568 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:26.790290    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:48:26.805681 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:26.805640    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cbmss" podStartSLOduration=67.062691074 podStartE2EDuration="1m9.805629706s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:48:23.520231476 +0000 UTC m=+66.584397591" lastFinishedPulling="2026-05-29 20:48:26.263170108 +0000 UTC m=+69.327336223" observedRunningTime="2026-05-29 20:48:26.805173531 +0000 UTC m=+69.869339669" watchObservedRunningTime="2026-05-29 20:48:26.805629706 +0000 UTC m=+69.869795844"
May 29 20:48:55.156035 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:55.155869    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:48:55.156035 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:55.155936    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:48:55.156556 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:55.156050    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 29 20:48:55.156556 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:55.156050    2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 29 20:48:55.156556 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:55.156132    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls podName:7d207e7d-a860-433d-ba4f-67fcd022b23f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:59.156113431 +0000 UTC m=+162.220279546 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls") pod "dns-default-jdb2d" (UID: "7d207e7d-a860-433d-ba4f-67fcd022b23f") : secret "dns-default-metrics-tls" not found
May 29 20:48:55.156556 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:55.156151    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert podName:f74cd5ec-fc29-41c8-8986-230ca8c7759a nodeName:}" failed. No retries permitted until 2026-05-29 20:49:59.156143121 +0000 UTC m=+162.220309235 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert") pod "ingress-canary-q689n" (UID: "f74cd5ec-fc29-41c8-8986-230ca8c7759a") : secret "canary-serving-cert" not found
May 29 20:48:57.794403 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:57.794374    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cbmss"
May 29 20:48:58.357237 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.357204    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"]
May 29 20:48:58.360532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.360499    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-p2929"]
May 29 20:48:58.360698 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.360679    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.364788 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.364750    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 29 20:48:58.364788 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.364773    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 29 20:48:58.364961 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.364811    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cb7xs\""
May 29 20:48:58.364961 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.364773    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 29 20:48:58.365936 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.365917    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.368044 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.368027    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 29 20:48:58.368139 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.368062    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 29 20:48:58.368766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.368747    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 29 20:48:58.368766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.368763    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gx692\""
May 29 20:48:58.368905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.368792    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 29 20:48:58.373804 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.373785    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 29 20:48:58.375962 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.375936    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"]
May 29 20:48:58.376615 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.376596    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-p2929"]
May 29 20:48:58.460662 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.460631    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"]
May 29 20:48:58.462725 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.462705    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:48:58.462859 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.462831    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.464505 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.464476    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.466317 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.466303    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 29 20:48:58.466817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.466801    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 29 20:48:58.466905 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.466801    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gjxwq\""
May 29 20:48:58.467206 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.467189    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 29 20:48:58.467287 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.467254    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rp2kq\""
May 29 20:48:58.467676 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.467663    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 29 20:48:58.468066 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.468044    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 29 20:48:58.475005 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.474966    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 29 20:48:58.477391 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.477366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bql96\" (UniqueName: \"kubernetes.io/projected/e5a02a38-832f-40cc-8c71-4543ed7e8c78-kube-api-access-bql96\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.477507 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.477487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64dbcb87-94d3-4e99-9040-c81c0a726851-serving-cert\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.477574 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.477522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-trusted-ca\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.477729 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.477710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-config\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.477891 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.477870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.478078 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.478041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrs9\" (UniqueName: \"kubernetes.io/projected/64dbcb87-94d3-4e99-9040-c81c0a726851-kube-api-access-9nrs9\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.478493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.478474    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"]
May 29 20:48:58.482088 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.482068    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:48:58.578978 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.578949    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-config\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.578978 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.578982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579060    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.579133    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.579182    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls podName:e5a02a38-832f-40cc-8c71-4543ed7e8c78 nodeName:}" failed. No retries permitted until 2026-05-29 20:48:59.079168071 +0000 UTC m=+102.143334186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-d6s6n" (UID: "e5a02a38-832f-40cc-8c71-4543ed7e8c78") : secret "samples-operator-tls" not found
May 29 20:48:58.579195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579195    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrs9\" (UniqueName: \"kubernetes.io/projected/64dbcb87-94d3-4e99-9040-c81c0a726851-kube-api-access-9nrs9\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6767aba-99b2-4e19-be09-fe95ef1f352f-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579346    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bql96\" (UniqueName: \"kubernetes.io/projected/e5a02a38-832f-40cc-8c71-4543ed7e8c78-kube-api-access-bql96\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579412    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64dbcb87-94d3-4e99-9040-c81c0a726851-serving-cert\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.579442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579436    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-trusted-ca\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.579687 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.579687 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vg5c\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.579745 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.579711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-config\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.580208 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.580133    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/64dbcb87-94d3-4e99-9040-c81c0a726851-trusted-ca\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.581805 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.581787    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/64dbcb87-94d3-4e99-9040-c81c0a726851-serving-cert\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.587690 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.587672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrs9\" (UniqueName: \"kubernetes.io/projected/64dbcb87-94d3-4e99-9040-c81c0a726851-kube-api-access-9nrs9\") pod \"console-operator-86d7c6cc77-p2929\" (UID: \"64dbcb87-94d3-4e99-9040-c81c0a726851\") " pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.587780 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.587741    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bql96\" (UniqueName: \"kubernetes.io/projected/e5a02a38-832f-40cc-8c71-4543ed7e8c78-kube-api-access-bql96\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:58.680595 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680491    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:48:58.680595 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680584    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680613    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680661    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680694    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6767aba-99b2-4e19-be09-fe95ef1f352f-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680710    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.680720    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680733    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.680741    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d679959d9-bwsb2: secret "image-registry-tls" not found
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680759    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.680817 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680793    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.681270 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.680831    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls podName:091699e3-7e84-4277-b8ee-4dea10575e84 nodeName:}" failed. No retries permitted until 2026-05-29 20:48:59.180793728 +0000 UTC m=+102.244959863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls") pod "image-registry-d679959d9-bwsb2" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84") : secret "image-registry-tls" not found
May 29 20:48:58.681270 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.680854    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:48:58.681270 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680864    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2vg5c\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.681270 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.680903    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.681270 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:58.681048    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:48:59.18102858 +0000 UTC m=+102.245194712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:48:58.682152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.681468    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.682152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.681565    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.682152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.681840    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6767aba-99b2-4e19-be09-fe95ef1f352f-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:58.682152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.681972    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.683656 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.683637    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.683909 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.683890    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.689975 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.689949    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vg5c\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.690113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.690075    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:58.803503 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.803471    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-p2929"]
May 29 20:48:58.806840 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:48:58.806809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64dbcb87_94d3_4e99_9040_c81c0a726851.slice/crio-63580e093403260c875a15ec37e6d621f4e681530cbfc9ace81388ab2b51af29 WatchSource:0}: Error finding container 63580e093403260c875a15ec37e6d621f4e681530cbfc9ace81388ab2b51af29: Status 404 returned error can't find the container with id 63580e093403260c875a15ec37e6d621f4e681530cbfc9ace81388ab2b51af29
May 29 20:48:58.848357 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:58.848320    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" event={"ID":"64dbcb87-94d3-4e99-9040-c81c0a726851","Type":"ContainerStarted","Data":"63580e093403260c875a15ec37e6d621f4e681530cbfc9ace81388ab2b51af29"}
May 29 20:48:59.088742 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:59.088698    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:48:59.088917 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.088849    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 20:48:59.088956 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.088941    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls podName:e5a02a38-832f-40cc-8c71-4543ed7e8c78 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:00.088924995 +0000 UTC m=+103.153091110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-d6s6n" (UID: "e5a02a38-832f-40cc-8c71-4543ed7e8c78") : secret "samples-operator-tls" not found
May 29 20:48:59.189858 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:59.189816    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:48:59.190062 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:48:59.189883    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:48:59.190062 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.189962    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:48:59.190062 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.189966    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 20:48:59.190062 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.190056    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d679959d9-bwsb2: secret "image-registry-tls" not found
May 29 20:48:59.190321 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.190041    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:00.190025373 +0000 UTC m=+103.254191489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:48:59.190321 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:48:59.190106    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls podName:091699e3-7e84-4277-b8ee-4dea10575e84 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:00.190092392 +0000 UTC m=+103.254258508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls") pod "image-registry-d679959d9-bwsb2" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84") : secret "image-registry-tls" not found
May 29 20:49:00.097926 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.097887    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:00.098405 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.098083    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 20:49:00.098405 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.098178    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls podName:e5a02a38-832f-40cc-8c71-4543ed7e8c78 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:02.098156633 +0000 UTC m=+105.162322754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-d6s6n" (UID: "e5a02a38-832f-40cc-8c71-4543ed7e8c78") : secret "samples-operator-tls" not found
May 29 20:49:00.199036 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.198984    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:00.199220 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.199048    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:00.199220 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.199156    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:49:00.199220 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.199170    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 20:49:00.199220 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.199183    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d679959d9-bwsb2: secret "image-registry-tls" not found
May 29 20:49:00.199415 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.199237    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:02.199214703 +0000 UTC m=+105.263380823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:49:00.199415 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:00.199258    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls podName:091699e3-7e84-4277-b8ee-4dea10575e84 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:02.199248936 +0000 UTC m=+105.263415053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls") pod "image-registry-d679959d9-bwsb2" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84") : secret "image-registry-tls" not found
May 29 20:49:00.853607 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.853589    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/0.log"
May 29 20:49:00.853734 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.853623    2563 generic.go:358] "Generic (PLEG): container finished" podID="64dbcb87-94d3-4e99-9040-c81c0a726851" containerID="8dec784219bb9ca67617d6aa601ebcbc440139482b8ae820007a48f3f3ef43e8" exitCode=255
May 29 20:49:00.853734 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.853673    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" event={"ID":"64dbcb87-94d3-4e99-9040-c81c0a726851","Type":"ContainerDied","Data":"8dec784219bb9ca67617d6aa601ebcbc440139482b8ae820007a48f3f3ef43e8"}
May 29 20:49:00.853896 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:00.853879    2563 scope.go:117] "RemoveContainer" containerID="8dec784219bb9ca67617d6aa601ebcbc440139482b8ae820007a48f3f3ef43e8"
May 29 20:49:01.856862 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.856833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:49:01.857284 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.857219    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/0.log"
May 29 20:49:01.857284 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.857249    2563 generic.go:358] "Generic (PLEG): container finished" podID="64dbcb87-94d3-4e99-9040-c81c0a726851" containerID="0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0" exitCode=255
May 29 20:49:01.857392 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.857285    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" event={"ID":"64dbcb87-94d3-4e99-9040-c81c0a726851","Type":"ContainerDied","Data":"0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0"}
May 29 20:49:01.857392 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.857343    2563 scope.go:117] "RemoveContainer" containerID="8dec784219bb9ca67617d6aa601ebcbc440139482b8ae820007a48f3f3ef43e8"
May 29 20:49:01.857536 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:01.857522    2563 scope.go:117] "RemoveContainer" containerID="0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0"
May 29 20:49:01.857724 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:01.857706    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-p2929_openshift-console-operator(64dbcb87-94d3-4e99-9040-c81c0a726851)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" podUID="64dbcb87-94d3-4e99-9040-c81c0a726851"
May 29 20:49:02.114253 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:02.114149    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:02.114404 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.114308    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 20:49:02.114404 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.114374    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls podName:e5a02a38-832f-40cc-8c71-4543ed7e8c78 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:06.114358012 +0000 UTC m=+109.178524126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-d6s6n" (UID: "e5a02a38-832f-40cc-8c71-4543ed7e8c78") : secret "samples-operator-tls" not found
May 29 20:49:02.214957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:02.214922    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:02.215110 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:02.214969    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:02.215173 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.215110    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 20:49:02.215173 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.215116    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:49:02.215269 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.215202    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:06.21518164 +0000 UTC m=+109.279347762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:49:02.215269 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.215121    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d679959d9-bwsb2: secret "image-registry-tls" not found
May 29 20:49:02.215370 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.215276    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls podName:091699e3-7e84-4277-b8ee-4dea10575e84 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:06.215261281 +0000 UTC m=+109.279427401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls") pod "image-registry-d679959d9-bwsb2" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84") : secret "image-registry-tls" not found
May 29 20:49:02.860109 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:02.860081    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:49:02.860482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:02.860405    2563 scope.go:117] "RemoveContainer" containerID="0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0"
May 29 20:49:02.860588 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:02.860569    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-p2929_openshift-console-operator(64dbcb87-94d3-4e99-9040-c81c0a726851)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" podUID="64dbcb87-94d3-4e99-9040-c81c0a726851"
May 29 20:49:04.780893 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:04.780862    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r6q6z_56d938c5-6d8c-4583-9cec-ebcc03a9d29e/dns-node-resolver/0.log"
May 29 20:49:05.259373 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.259334    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-685df94d57-mb7vn"]
May 29 20:49:05.261304 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.261288    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.263509 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.263482    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 29 20:49:05.263642 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.263494    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vg8fg\""
May 29 20:49:05.263642 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.263585    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 29 20:49:05.264360 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.264337    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 29 20:49:05.264471 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.264441    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 29 20:49:05.270085 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.270065    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-mb7vn"]
May 29 20:49:05.340893 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.340855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-cabundle\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.341100 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.340909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zkkc\" (UniqueName: \"kubernetes.io/projected/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-kube-api-access-4zkkc\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.341100 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.340967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-key\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.380675 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.380654    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gtqz_b943e2cb-8681-47db-8d8f-da6eadc7afa9/node-ca/0.log"
May 29 20:49:05.442215 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.442183    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-cabundle\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.442287 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.442227    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4zkkc\" (UniqueName: \"kubernetes.io/projected/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-kube-api-access-4zkkc\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.442287 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.442281    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-key\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.442819 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.442802    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-cabundle\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.445356 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.445339    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-signing-key\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.454547 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.454527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zkkc\" (UniqueName: \"kubernetes.io/projected/0c8cf14c-5a59-4e19-832d-fab3f0f17c65-kube-api-access-4zkkc\") pod \"service-ca-685df94d57-mb7vn\" (UID: \"0c8cf14c-5a59-4e19-832d-fab3f0f17c65\") " pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.570468 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.570378    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-685df94d57-mb7vn"
May 29 20:49:05.687069 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.687039    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-685df94d57-mb7vn"]
May 29 20:49:05.690176 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:05.690146    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8cf14c_5a59_4e19_832d_fab3f0f17c65.slice/crio-7522147aa69812539b85c35e7c042764df48fb94b634e844d98c3d9d7eca22db WatchSource:0}: Error finding container 7522147aa69812539b85c35e7c042764df48fb94b634e844d98c3d9d7eca22db: Status 404 returned error can't find the container with id 7522147aa69812539b85c35e7c042764df48fb94b634e844d98c3d9d7eca22db
May 29 20:49:05.867056 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:05.866954    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-mb7vn" event={"ID":"0c8cf14c-5a59-4e19-832d-fab3f0f17c65","Type":"ContainerStarted","Data":"7522147aa69812539b85c35e7c042764df48fb94b634e844d98c3d9d7eca22db"}
May 29 20:49:06.148377 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:06.148191    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:06.148551 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.148392    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 29 20:49:06.148551 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.148476    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls podName:e5a02a38-832f-40cc-8c71-4543ed7e8c78 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:14.148457893 +0000 UTC m=+117.212624009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-d6s6n" (UID: "e5a02a38-832f-40cc-8c71-4543ed7e8c78") : secret "samples-operator-tls" not found
May 29 20:49:06.249123 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:06.249081    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:06.249123 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:06.249137    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:06.249358 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.249233    2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 29 20:49:06.249358 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.249235    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:49:06.249358 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.249304    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:14.249287973 +0000 UTC m=+117.313454088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:49:06.249358 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.249244    2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d679959d9-bwsb2: secret "image-registry-tls" not found
May 29 20:49:06.249501 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:06.249363    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls podName:091699e3-7e84-4277-b8ee-4dea10575e84 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:14.249350627 +0000 UTC m=+117.313516742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls") pod "image-registry-d679959d9-bwsb2" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84") : secret "image-registry-tls" not found
May 29 20:49:07.872038 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:07.871983    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-685df94d57-mb7vn" event={"ID":"0c8cf14c-5a59-4e19-832d-fab3f0f17c65","Type":"ContainerStarted","Data":"9dfb747b70943485aee3e86620c65f96e38c632c0c761bde3371420faa2fae3d"}
May 29 20:49:07.889114 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:07.889010    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-685df94d57-mb7vn" podStartSLOduration=0.95427311 podStartE2EDuration="2.888975177s" podCreationTimestamp="2026-05-29 20:49:05 +0000 UTC" firstStartedPulling="2026-05-29 20:49:05.691920644 +0000 UTC m=+108.756086758" lastFinishedPulling="2026-05-29 20:49:07.626622711 +0000 UTC m=+110.690788825" observedRunningTime="2026-05-29 20:49:07.88803138 +0000 UTC m=+110.952197516" watchObservedRunningTime="2026-05-29 20:49:07.888975177 +0000 UTC m=+110.953141313"
May 29 20:49:08.681570 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:08.681526    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:49:08.681570 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:08.681577    2563 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:49:08.681935 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:08.681923    2563 scope.go:117] "RemoveContainer" containerID="0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0"
May 29 20:49:08.682142 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:08.682124    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-p2929_openshift-console-operator(64dbcb87-94d3-4e99-9040-c81c0a726851)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" podUID="64dbcb87-94d3-4e99-9040-c81c0a726851"
May 29 20:49:14.214582 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.214539    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:14.217222 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.217196    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e5a02a38-832f-40cc-8c71-4543ed7e8c78-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-d6s6n\" (UID: \"e5a02a38-832f-40cc-8c71-4543ed7e8c78\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:14.274760 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.274735    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"
May 29 20:49:14.314904 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.314872    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:14.315070 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.314921    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:14.315070 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:14.315056    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 29 20:49:14.315154 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:14.315146    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert podName:f6767aba-99b2-4e19-be09-fe95ef1f352f nodeName:}" failed. No retries permitted until 2026-05-29 20:49:30.315123909 +0000 UTC m=+133.379290030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-j8tv4" (UID: "f6767aba-99b2-4e19-be09-fe95ef1f352f") : secret "networking-console-plugin-cert" not found
May 29 20:49:14.317330 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.317310    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"image-registry-d679959d9-bwsb2\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") " pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:14.378502 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.378464    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:14.404150 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.404124    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n"]
May 29 20:49:14.503453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.503422    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:49:14.507179 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:14.507150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091699e3_7e84_4277_b8ee_4dea10575e84.slice/crio-e2e898fc3a543d0c87696517303894fac08a2f9a8ebea0f222b08a965ecc3478 WatchSource:0}: Error finding container e2e898fc3a543d0c87696517303894fac08a2f9a8ebea0f222b08a965ecc3478: Status 404 returned error can't find the container with id e2e898fc3a543d0c87696517303894fac08a2f9a8ebea0f222b08a965ecc3478
May 29 20:49:14.885899 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.885812    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" event={"ID":"091699e3-7e84-4277-b8ee-4dea10575e84","Type":"ContainerStarted","Data":"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"}
May 29 20:49:14.885899 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.885857    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" event={"ID":"091699e3-7e84-4277-b8ee-4dea10575e84","Type":"ContainerStarted","Data":"e2e898fc3a543d0c87696517303894fac08a2f9a8ebea0f222b08a965ecc3478"}
May 29 20:49:14.886120 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.885954    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:14.886778 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.886752    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n" event={"ID":"e5a02a38-832f-40cc-8c71-4543ed7e8c78","Type":"ContainerStarted","Data":"6090a7dc4f86e03e11bdb4d6a0588038a8dce1676b6ff52bd63a59db8c153af7"}
May 29 20:49:14.905091 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:14.905039    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" podStartSLOduration=16.90502123 podStartE2EDuration="16.90502123s" podCreationTimestamp="2026-05-29 20:48:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:49:14.904562244 +0000 UTC m=+117.968728381" watchObservedRunningTime="2026-05-29 20:49:14.90502123 +0000 UTC m=+117.969187368"
May 29 20:49:16.895152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:16.895115    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n" event={"ID":"e5a02a38-832f-40cc-8c71-4543ed7e8c78","Type":"ContainerStarted","Data":"bb86962bea502195603adfc1def629c71621a14ae6205665ed74a191bb7270f2"}
May 29 20:49:16.895527 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:16.895159    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n" event={"ID":"e5a02a38-832f-40cc-8c71-4543ed7e8c78","Type":"ContainerStarted","Data":"31c30f56eeab318f2c34191b3bc622da1755e703b7165ae86aa087779aa7986d"}
May 29 20:49:16.912135 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:16.912081    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-d6s6n" podStartSLOduration=17.054031774 podStartE2EDuration="18.912063096s" podCreationTimestamp="2026-05-29 20:48:58 +0000 UTC" firstStartedPulling="2026-05-29 20:49:14.446760261 +0000 UTC m=+117.510926379" lastFinishedPulling="2026-05-29 20:49:16.304791568 +0000 UTC m=+119.368957701" observedRunningTime="2026-05-29 20:49:16.911788638 +0000 UTC m=+119.975954776" watchObservedRunningTime="2026-05-29 20:49:16.912063096 +0000 UTC m=+119.976229233"
May 29 20:49:22.496303 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:22.496274    2563 scope.go:117] "RemoveContainer" containerID="0cc90d94b6c7085bccfe52ba0d830e80c48764f71ec80016cb0506138db7e4e0"
May 29 20:49:22.911157 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:22.911080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:49:22.911157 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:22.911143    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" event={"ID":"64dbcb87-94d3-4e99-9040-c81c0a726851","Type":"ContainerStarted","Data":"1e76c0c4ce9d5a31183b908bb8715adc5c0fc7835476b495bde8d97cca6559e1"}
May 29 20:49:22.911440 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:22.911422    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:49:22.928409 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:22.928368    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929" podStartSLOduration=22.997803443 podStartE2EDuration="24.928356404s" podCreationTimestamp="2026-05-29 20:48:58 +0000 UTC" firstStartedPulling="2026-05-29 20:48:58.808594954 +0000 UTC m=+101.872761072" lastFinishedPulling="2026-05-29 20:49:00.739147911 +0000 UTC m=+103.803314033" observedRunningTime="2026-05-29 20:49:22.927805204 +0000 UTC m=+125.991971342" watchObservedRunningTime="2026-05-29 20:49:22.928356404 +0000 UTC m=+125.992522577"
May 29 20:49:23.107774 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:23.107747    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-p2929"
May 29 20:49:25.335377 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.335347    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"]
May 29 20:49:25.337997 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.337969    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.340376 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.340355    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 29 20:49:25.340604 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.340586    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jh5wp\""
May 29 20:49:25.341358 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.341337    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 29 20:49:25.341358 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.341352    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 29 20:49:25.341491 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.341456    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 29 20:49:25.348724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.348705    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"]
May 29 20:49:25.401979 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.401954    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:49:25.422603 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.422579    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vd4gs"]
May 29 20:49:25.425623 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.425605    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.427903 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.427883    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q4h98\""
May 29 20:49:25.428014 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.427910    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 29 20:49:25.428014 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.427912    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 29 20:49:25.428132 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.427883    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 29 20:49:25.428275 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.428258    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 29 20:49:25.432877 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.432854    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85f44c86b7-mmd6c"]
May 29 20:49:25.435896 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.435879    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.437606 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.437572    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vd4gs"]
May 29 20:49:25.446717 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.446688    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85f44c86b7-mmd6c"]
May 29 20:49:25.498076 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.498048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.498205 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.498114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk8th\" (UniqueName: \"kubernetes.io/projected/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-kube-api-access-bk8th\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.599092 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599021    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.599092 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-tls\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599283 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6glw\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-kube-api-access-w6glw\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599283 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-trusted-ca\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599283 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d875c44-7ad7-4dd4-97d7-13b6e3706641-crio-socket\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.599283 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.599482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-bound-sa-token\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-image-registry-private-configuration\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d875c44-7ad7-4dd4-97d7-13b6e3706641-data-volume\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.599482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-certificates\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-installation-pull-secrets\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntcgm\" (UniqueName: \"kubernetes.io/projected/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-api-access-ntcgm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.599674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599555    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bk8th\" (UniqueName: \"kubernetes.io/projected/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-kube-api-access-bk8th\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.599674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-ca-trust-extracted\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.599674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.599620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d875c44-7ad7-4dd4-97d7-13b6e3706641-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.601682 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.601664    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.615355 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.615336    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk8th\" (UniqueName: \"kubernetes.io/projected/2cb71b1e-402c-4c4f-ac9f-bbb67835e562-kube-api-access-bk8th\") pod \"managed-serviceaccount-addon-agent-58d8458fc4-r4d28\" (UID: \"2cb71b1e-402c-4c4f-ac9f-bbb67835e562\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.656319 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.656298    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700342    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d875c44-7ad7-4dd4-97d7-13b6e3706641-crio-socket\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700389    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700430    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-bound-sa-token\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700469    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-image-registry-private-configuration\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d875c44-7ad7-4dd4-97d7-13b6e3706641-crio-socket\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700498    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d875c44-7ad7-4dd4-97d7-13b6e3706641-data-volume\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700522    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-certificates\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700556    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-installation-pull-secrets\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700589    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ntcgm\" (UniqueName: \"kubernetes.io/projected/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-api-access-ntcgm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700759    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-ca-trust-extracted\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700817    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d875c44-7ad7-4dd4-97d7-13b6e3706641-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700878    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-tls\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700928    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w6glw\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-kube-api-access-w6glw\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.700967    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-trusted-ca\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.701825 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.701802    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.702144 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.702124    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-trusted-ca\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.702405 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.702361    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-ca-trust-extracted\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.702938 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.702897    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-certificates\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.703780 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.703756    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d875c44-7ad7-4dd4-97d7-13b6e3706641-data-volume\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.705346 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.705314    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-installation-pull-secrets\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.705980 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.705921    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-registry-tls\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.705980 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.705934    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-image-registry-private-configuration\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.706605 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.706583    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d875c44-7ad7-4dd4-97d7-13b6e3706641-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.710538 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.710519    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-bound-sa-token\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.718073 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.718017    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntcgm\" (UniqueName: \"kubernetes.io/projected/4d875c44-7ad7-4dd4-97d7-13b6e3706641-kube-api-access-ntcgm\") pod \"insights-runtime-extractor-vd4gs\" (UID: \"4d875c44-7ad7-4dd4-97d7-13b6e3706641\") " pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.719480 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.719456    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6glw\" (UniqueName: \"kubernetes.io/projected/ecf6cd45-33c4-4205-b1e1-ca1d8bebc220-kube-api-access-w6glw\") pod \"image-registry-85f44c86b7-mmd6c\" (UID: \"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220\") " pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.735275 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.735249    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vd4gs"
May 29 20:49:25.744791 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.744761    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:25.792148 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.791417    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28"]
May 29 20:49:25.795053 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:25.794961    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cb71b1e_402c_4c4f_ac9f_bbb67835e562.slice/crio-242edd2e37465760f61bfe9ce54a9830c32cb469898537c353d2e4e1fc4a0066 WatchSource:0}: Error finding container 242edd2e37465760f61bfe9ce54a9830c32cb469898537c353d2e4e1fc4a0066: Status 404 returned error can't find the container with id 242edd2e37465760f61bfe9ce54a9830c32cb469898537c353d2e4e1fc4a0066
May 29 20:49:25.874359 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.874277    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vd4gs"]
May 29 20:49:25.878770 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:25.878739    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d875c44_7ad7_4dd4_97d7_13b6e3706641.slice/crio-d30f59e3b4c49b72a191ab84d07a5f93fd642b6138d3e97622848d42227e6f10 WatchSource:0}: Error finding container d30f59e3b4c49b72a191ab84d07a5f93fd642b6138d3e97622848d42227e6f10: Status 404 returned error can't find the container with id d30f59e3b4c49b72a191ab84d07a5f93fd642b6138d3e97622848d42227e6f10
May 29 20:49:25.892583 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.892558    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85f44c86b7-mmd6c"]
May 29 20:49:25.895268 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:25.895238    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf6cd45_33c4_4205_b1e1_ca1d8bebc220.slice/crio-1143c5e39802c46151cc7b5ed43fd04568edfffe38041453a8ca994290935606 WatchSource:0}: Error finding container 1143c5e39802c46151cc7b5ed43fd04568edfffe38041453a8ca994290935606: Status 404 returned error can't find the container with id 1143c5e39802c46151cc7b5ed43fd04568edfffe38041453a8ca994290935606
May 29 20:49:25.919969 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.919935    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd4gs" event={"ID":"4d875c44-7ad7-4dd4-97d7-13b6e3706641","Type":"ContainerStarted","Data":"d30f59e3b4c49b72a191ab84d07a5f93fd642b6138d3e97622848d42227e6f10"}
May 29 20:49:25.921021 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.920968    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c" event={"ID":"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220","Type":"ContainerStarted","Data":"1143c5e39802c46151cc7b5ed43fd04568edfffe38041453a8ca994290935606"}
May 29 20:49:25.922008 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:25.921962    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28" event={"ID":"2cb71b1e-402c-4c4f-ac9f-bbb67835e562","Type":"ContainerStarted","Data":"242edd2e37465760f61bfe9ce54a9830c32cb469898537c353d2e4e1fc4a0066"}
May 29 20:49:26.927522 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:26.927482    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd4gs" event={"ID":"4d875c44-7ad7-4dd4-97d7-13b6e3706641","Type":"ContainerStarted","Data":"8f6254bf25dab71a9b57ee6fb29cbc20e3e9540ea1506f868c637e8371718047"}
May 29 20:49:26.927522 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:26.927524    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd4gs" event={"ID":"4d875c44-7ad7-4dd4-97d7-13b6e3706641","Type":"ContainerStarted","Data":"1d5b600a5a8587f9f5bc580014855c26dc7671b9faa85d3e45a61eb246c875a4"}
May 29 20:49:26.929182 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:26.929150    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c" event={"ID":"ecf6cd45-33c4-4205-b1e1-ca1d8bebc220","Type":"ContainerStarted","Data":"50af6467d5c061120772419763b55c2578a1ea521238f3620cd867c496f9fa5a"}
May 29 20:49:26.929879 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:26.929859    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:26.950202 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:26.949881    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c" podStartSLOduration=1.9498626460000001 podStartE2EDuration="1.949862646s" podCreationTimestamp="2026-05-29 20:49:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:49:26.948824301 +0000 UTC m=+130.012990440" watchObservedRunningTime="2026-05-29 20:49:26.949862646 +0000 UTC m=+130.014028782"
May 29 20:49:27.214502 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:27.214408    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:49:27.218468 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:27.218437    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0873cca3-a3cf-4320-b20f-55a5704aa6c5-metrics-certs\") pod \"network-metrics-daemon-k2wzp\" (UID: \"0873cca3-a3cf-4320-b20f-55a5704aa6c5\") " pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:49:27.510510 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:27.510482    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tklm4\""
May 29 20:49:27.519367 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:27.519037    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2wzp"
May 29 20:49:29.251740 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.251699    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k2wzp"]
May 29 20:49:29.255963 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:29.255936    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0873cca3_a3cf_4320_b20f_55a5704aa6c5.slice/crio-64179af4853acd5bccef91c702541b6dd90511af9065cd175d093ab85bf1e55c WatchSource:0}: Error finding container 64179af4853acd5bccef91c702541b6dd90511af9065cd175d093ab85bf1e55c: Status 404 returned error can't find the container with id 64179af4853acd5bccef91c702541b6dd90511af9065cd175d093ab85bf1e55c
May 29 20:49:29.938835 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.938795    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28" event={"ID":"2cb71b1e-402c-4c4f-ac9f-bbb67835e562","Type":"ContainerStarted","Data":"8495451c899d0d76e544cf15159f577622cea3f95ad63e5a017a16ada4031a8e"}
May 29 20:49:29.940943 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.940892    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vd4gs" event={"ID":"4d875c44-7ad7-4dd4-97d7-13b6e3706641","Type":"ContainerStarted","Data":"fe2364d1c185393c55abccebfc02699f15b81b7ed301d1b991f1c6134aa97811"}
May 29 20:49:29.942128 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.942095    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2wzp" event={"ID":"0873cca3-a3cf-4320-b20f-55a5704aa6c5","Type":"ContainerStarted","Data":"64179af4853acd5bccef91c702541b6dd90511af9065cd175d093ab85bf1e55c"}
May 29 20:49:29.954780 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.954598    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58d8458fc4-r4d28" podStartSLOduration=1.650056825 podStartE2EDuration="4.954583871s" podCreationTimestamp="2026-05-29 20:49:25 +0000 UTC" firstStartedPulling="2026-05-29 20:49:25.798833341 +0000 UTC m=+128.862999460" lastFinishedPulling="2026-05-29 20:49:29.103360388 +0000 UTC m=+132.167526506" observedRunningTime="2026-05-29 20:49:29.953914037 +0000 UTC m=+133.018080174" watchObservedRunningTime="2026-05-29 20:49:29.954583871 +0000 UTC m=+133.018750009"
May 29 20:49:29.973844 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:29.973803    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vd4gs" podStartSLOduration=1.8011899420000002 podStartE2EDuration="4.973787454s" podCreationTimestamp="2026-05-29 20:49:25 +0000 UTC" firstStartedPulling="2026-05-29 20:49:25.935758553 +0000 UTC m=+128.999924668" lastFinishedPulling="2026-05-29 20:49:29.108356064 +0000 UTC m=+132.172522180" observedRunningTime="2026-05-29 20:49:29.972526597 +0000 UTC m=+133.036692736" watchObservedRunningTime="2026-05-29 20:49:29.973787454 +0000 UTC m=+133.037953594"
May 29 20:49:30.339975 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.339938    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:30.342640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.342611    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f6767aba-99b2-4e19-be09-fe95ef1f352f-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-j8tv4\" (UID: \"f6767aba-99b2-4e19-be09-fe95ef1f352f\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:30.574908 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.574878    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gjxwq\""
May 29 20:49:30.582766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.582742    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"
May 29 20:49:30.743576 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.743546    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4"]
May 29 20:49:30.748174 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:30.748140    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6767aba_99b2_4e19_be09_fe95ef1f352f.slice/crio-8060c6d80e9eb095ea2356b6105e937c0e00973cc4668ab9b491ad923f8f5c5d WatchSource:0}: Error finding container 8060c6d80e9eb095ea2356b6105e937c0e00973cc4668ab9b491ad923f8f5c5d: Status 404 returned error can't find the container with id 8060c6d80e9eb095ea2356b6105e937c0e00973cc4668ab9b491ad923f8f5c5d
May 29 20:49:30.948876 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.948786    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2wzp" event={"ID":"0873cca3-a3cf-4320-b20f-55a5704aa6c5","Type":"ContainerStarted","Data":"dbbfae33f5522d7e9641be54f0b2152d506bb4ebc6ce6b46fe72dfb01fbf89cb"}
May 29 20:49:30.948876 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.948831    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2wzp" event={"ID":"0873cca3-a3cf-4320-b20f-55a5704aa6c5","Type":"ContainerStarted","Data":"f5d18a8e2686c7030b6b06838d96c24bb88ac3129a19085302b98c0e60cb5d98"}
May 29 20:49:30.949750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.949724    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4" event={"ID":"f6767aba-99b2-4e19-be09-fe95ef1f352f","Type":"ContainerStarted","Data":"8060c6d80e9eb095ea2356b6105e937c0e00973cc4668ab9b491ad923f8f5c5d"}
May 29 20:49:30.968412 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:30.968368    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k2wzp" podStartSLOduration=132.608138218 podStartE2EDuration="2m13.968355413s" podCreationTimestamp="2026-05-29 20:47:17 +0000 UTC" firstStartedPulling="2026-05-29 20:49:29.258783918 +0000 UTC m=+132.322950036" lastFinishedPulling="2026-05-29 20:49:30.619001102 +0000 UTC m=+133.683167231" observedRunningTime="2026-05-29 20:49:30.967135601 +0000 UTC m=+134.031301736" watchObservedRunningTime="2026-05-29 20:49:30.968355413 +0000 UTC m=+134.032521549"
May 29 20:49:32.956772 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:32.956731    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4" event={"ID":"f6767aba-99b2-4e19-be09-fe95ef1f352f","Type":"ContainerStarted","Data":"7de54c79b65fb85d0c2dc7e0c9a8b60ce9f43f8f38d8d23ad02f177d2348a58e"}
May 29 20:49:32.976175 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:32.976123    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-j8tv4" podStartSLOduration=33.735234116 podStartE2EDuration="34.976105584s" podCreationTimestamp="2026-05-29 20:48:58 +0000 UTC" firstStartedPulling="2026-05-29 20:49:30.750347736 +0000 UTC m=+133.814513866" lastFinishedPulling="2026-05-29 20:49:31.991219217 +0000 UTC m=+135.055385334" observedRunningTime="2026-05-29 20:49:32.975122043 +0000 UTC m=+136.039288193" watchObservedRunningTime="2026-05-29 20:49:32.976105584 +0000 UTC m=+136.040271722"
May 29 20:49:35.407390 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:35.407358    2563 patch_prober.go:28] interesting pod/image-registry-d679959d9-bwsb2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 29 20:49:35.407748 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:35.407409    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 29 20:49:45.406522 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:45.406490    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:45.749804 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:45.749767    2563 patch_prober.go:28] interesting pod/image-registry-85f44c86b7-mmd6c container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 29 20:49:45.749941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:45.749830    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c" podUID="ecf6cd45-33c4-4205-b1e1-ca1d8bebc220" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 29 20:49:48.938060 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:48.938032    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85f44c86b7-mmd6c"
May 29 20:49:50.421325 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.421259    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" containerName="registry" containerID="cri-o://240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee" gracePeriod=30
May 29 20:49:50.657829 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.657804    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:50.695197 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695119    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695197 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695178    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695205    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695233    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695276    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695305    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695338    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vg5c\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695596 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695373    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted\") pod \"091699e3-7e84-4277-b8ee-4dea10575e84\" (UID: \"091699e3-7e84-4277-b8ee-4dea10575e84\") "
May 29 20:49:50.695596 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.695568    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:49:50.696157 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.696016    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:49:50.697887 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.697836    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:49:50.698126 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.698101    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:49:50.698265 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.698193    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c" (OuterVolumeSpecName: "kube-api-access-2vg5c") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "kube-api-access-2vg5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:49:50.698321 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.698307    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:49:50.698365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.698350    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:49:50.716143 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.716121    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "091699e3-7e84-4277-b8ee-4dea10575e84" (UID: "091699e3-7e84-4277-b8ee-4dea10575e84"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:49:50.796493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796458    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-image-registry-private-configuration\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796487    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-registry-certificates\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796497    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vg5c\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-kube-api-access-2vg5c\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796707 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796506    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/091699e3-7e84-4277-b8ee-4dea10575e84-ca-trust-extracted\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796707 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796515    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/091699e3-7e84-4277-b8ee-4dea10575e84-trusted-ca\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796707 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796524    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-registry-tls\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796707 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796532    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/091699e3-7e84-4277-b8ee-4dea10575e84-bound-sa-token\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:50.796707 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:50.796541    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/091699e3-7e84-4277-b8ee-4dea10575e84-installation-pull-secrets\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:49:51.010479 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.010443    2563 generic.go:358] "Generic (PLEG): container finished" podID="091699e3-7e84-4277-b8ee-4dea10575e84" containerID="240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee" exitCode=0
May 29 20:49:51.010611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.010510    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d679959d9-bwsb2"
May 29 20:49:51.010611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.010527    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" event={"ID":"091699e3-7e84-4277-b8ee-4dea10575e84","Type":"ContainerDied","Data":"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"}
May 29 20:49:51.010611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.010560    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d679959d9-bwsb2" event={"ID":"091699e3-7e84-4277-b8ee-4dea10575e84","Type":"ContainerDied","Data":"e2e898fc3a543d0c87696517303894fac08a2f9a8ebea0f222b08a965ecc3478"}
May 29 20:49:51.010611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.010577    2563 scope.go:117] "RemoveContainer" containerID="240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"
May 29 20:49:51.019480 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.019447    2563 scope.go:117] "RemoveContainer" containerID="240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"
May 29 20:49:51.020027 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:51.019966    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee\": container with ID starting with 240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee not found: ID does not exist" containerID="240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"
May 29 20:49:51.020108 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.020022    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee"} err="failed to get container status \"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee\": rpc error: code = NotFound desc = could not find container \"240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee\": container with ID starting with 240a4bb3342d7e48a898f8dc5dae94c9e2ab5a0f6a40741894a2b4e2b3669aee not found: ID does not exist"
May 29 20:49:51.031901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.031879    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:49:51.035860 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.035840    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d679959d9-bwsb2"]
May 29 20:49:51.500788 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:51.500756    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" path="/var/lib/kubelet/pods/091699e3-7e84-4277-b8ee-4dea10575e84/volumes"
May 29 20:49:52.636126 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.636095    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-m45qc"]
May 29 20:49:52.636557 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.636496    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" containerName="registry"
May 29 20:49:52.636557 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.636514    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" containerName="registry"
May 29 20:49:52.636700 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.636570    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="091699e3-7e84-4277-b8ee-4dea10575e84" containerName="registry"
May 29 20:49:52.639833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.639814    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.642554 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.642313    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 29 20:49:52.642554 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.642356    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 29 20:49:52.642554 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.642356    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 29 20:49:52.642554 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.642443    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 29 20:49:52.643072 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.643054    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 29 20:49:52.643259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.643099    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zmvdl\""
May 29 20:49:52.652017 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.650879    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-m45qc"]
May 29 20:49:52.673653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.673622    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"]
May 29 20:49:52.676800 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.676775    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.678946 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.678924    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dns2q\""
May 29 20:49:52.679103 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.679081    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 29 20:49:52.679337 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.679320    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 29 20:49:52.679469 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.679451    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 29 20:49:52.681828 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.681808    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-g84rm"]
May 29 20:49:52.684272 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.684253    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.687328 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.687308    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-29fkc\""
May 29 20:49:52.687584 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.687531    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 29 20:49:52.687695 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.687650    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 29 20:49:52.687906 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.687822    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 29 20:49:52.690303 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.690192    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"]
May 29 20:49:52.709953 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.709930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-accelerators-collector-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710089 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.709971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710089 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.710208 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.710208 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-metrics-client-ca\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710208 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-sys\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxrb4\" (UniqueName: \"kubernetes.io/projected/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-kube-api-access-cxrb4\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17b20fd4-cd52-46e5-9ee4-edde38dab402-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.710364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.710364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72a67383-0522-42ab-aeb0-dbd813814032-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.710364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-wtmp\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcn9z\" (UniqueName: \"kubernetes.io/projected/17b20fd4-cd52-46e5-9ee4-edde38dab402-kube-api-access-kcn9z\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-root\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-textfile\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.710599 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.710823 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.710612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnrnm\" (UniqueName: \"kubernetes.io/projected/72a67383-0522-42ab-aeb0-dbd813814032-kube-api-access-mnrnm\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.811046 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811004    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.811046 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811053    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811256 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811077    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-wtmp\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811256 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811161    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kcn9z\" (UniqueName: \"kubernetes.io/projected/17b20fd4-cd52-46e5-9ee4-edde38dab402-kube-api-access-kcn9z\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.811256 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811202    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-root\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811256 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811212    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-wtmp\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811256 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811228    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-textfile\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811254    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-root\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811291    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811320    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mnrnm\" (UniqueName: \"kubernetes.io/projected/72a67383-0522-42ab-aeb0-dbd813814032-kube-api-access-mnrnm\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811353    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-accelerators-collector-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811394    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811422    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.811543 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811457    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.811941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811542    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-textfile\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.811941 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.811625    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 29 20:49:52.811941 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.811689    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls podName:17b20fd4-cd52-46e5-9ee4-edde38dab402 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:53.311670972 +0000 UTC m=+156.375837092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls") pod "openshift-state-metrics-85658796db-m45qc" (UID: "17b20fd4-cd52-46e5-9ee4-edde38dab402") : secret "openshift-state-metrics-tls" not found
May 29 20:49:52.811941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811766    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.812166 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.811943    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-metrics-client-ca\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812290    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.812482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812466    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-metrics-client-ca\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812557 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812511    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-sys\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812591    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-sys\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812661 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812624    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cxrb4\" (UniqueName: \"kubernetes.io/projected/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-kube-api-access-cxrb4\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812726 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812678    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-accelerators-collector-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.812846 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.812831    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 29 20:49:52.812975 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.812961    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls podName:a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e nodeName:}" failed. No retries permitted until 2026-05-29 20:49:53.312945808 +0000 UTC m=+156.377111948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls") pod "node-exporter-g84rm" (UID: "a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e") : secret "node-exporter-tls" not found
May 29 20:49:52.813078 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.812661    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17b20fd4-cd52-46e5-9ee4-edde38dab402-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.813078 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.813029    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.813078 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.813063    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72a67383-0522-42ab-aeb0-dbd813814032-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.813374 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.813293    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 29 20:49:52.813374 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:52.813351    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls podName:72a67383-0522-42ab-aeb0-dbd813814032 nodeName:}" failed. No retries permitted until 2026-05-29 20:49:53.313336311 +0000 UTC m=+156.377502429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls") pod "kube-state-metrics-8469c486bb-jvqcb" (UID: "72a67383-0522-42ab-aeb0-dbd813814032") : secret "kube-state-metrics-tls" not found
May 29 20:49:52.813479 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.813373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17b20fd4-cd52-46e5-9ee4-edde38dab402-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.813479 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.813379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72a67383-0522-42ab-aeb0-dbd813814032-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.814545 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.814521    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:52.814914 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.814892    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.815778 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.815737    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.820715 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.820691    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcn9z\" (UniqueName: \"kubernetes.io/projected/17b20fd4-cd52-46e5-9ee4-edde38dab402-kube-api-access-kcn9z\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:52.820822 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.820787    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnrnm\" (UniqueName: \"kubernetes.io/projected/72a67383-0522-42ab-aeb0-dbd813814032-kube-api-access-mnrnm\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:52.821331 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:52.821307    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxrb4\" (UniqueName: \"kubernetes.io/projected/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-kube-api-access-cxrb4\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:53.317617 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.317580    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:53.317793 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.317662    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:53.317793 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.317686    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:53.317793 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:53.317773    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 29 20:49:53.317978 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:53.317833    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls podName:a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e nodeName:}" failed. No retries permitted until 2026-05-29 20:49:54.317819428 +0000 UTC m=+157.381985543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls") pod "node-exporter-g84rm" (UID: "a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e") : secret "node-exporter-tls" not found
May 29 20:49:53.320325 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.320300    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17b20fd4-cd52-46e5-9ee4-edde38dab402-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-m45qc\" (UID: \"17b20fd4-cd52-46e5-9ee4-edde38dab402\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:53.320399 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.320307    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72a67383-0522-42ab-aeb0-dbd813814032-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-jvqcb\" (UID: \"72a67383-0522-42ab-aeb0-dbd813814032\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:53.550872 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.550837    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc"
May 29 20:49:53.589465 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.587874    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"
May 29 20:49:53.694325 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.694241    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-m45qc"]
May 29 20:49:53.698838 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:53.698810    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b20fd4_cd52_46e5_9ee4_edde38dab402.slice/crio-562a8356bc7f50270a8f367ecb804749c4746703aa22b557df7b6df4bc327cde WatchSource:0}: Error finding container 562a8356bc7f50270a8f367ecb804749c4746703aa22b557df7b6df4bc327cde: Status 404 returned error can't find the container with id 562a8356bc7f50270a8f367ecb804749c4746703aa22b557df7b6df4bc327cde
May 29 20:49:53.743890 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:53.743100    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb"]
May 29 20:49:54.020504 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.020470    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb" event={"ID":"72a67383-0522-42ab-aeb0-dbd813814032","Type":"ContainerStarted","Data":"198bf74858994ff237f152f678ea88e23bb0eb6dd8c22bf949fdd4f43ae73a15"}
May 29 20:49:54.021899 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.021862    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc" event={"ID":"17b20fd4-cd52-46e5-9ee4-edde38dab402","Type":"ContainerStarted","Data":"85733014f881bc90bef09b08643c9edff53d393bd09b125444fc0ead2684d5f8"}
May 29 20:49:54.021899 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.021898    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc" event={"ID":"17b20fd4-cd52-46e5-9ee4-edde38dab402","Type":"ContainerStarted","Data":"754c8de115c956b4bdfb0dd9339337ca6678c782e34f0f2b10239d737ee093fe"}
May 29 20:49:54.022072 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.021910    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc" event={"ID":"17b20fd4-cd52-46e5-9ee4-edde38dab402","Type":"ContainerStarted","Data":"562a8356bc7f50270a8f367ecb804749c4746703aa22b557df7b6df4bc327cde"}
May 29 20:49:54.325743 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.325659    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:54.328076 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.328053    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e-node-exporter-tls\") pod \"node-exporter-g84rm\" (UID: \"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e\") " pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:54.336140 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:54.336097    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jdb2d" podUID="7d207e7d-a860-433d-ba4f-67fcd022b23f"
May 29 20:49:54.343259 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:49:54.343233    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-q689n" podUID="f74cd5ec-fc29-41c8-8986-230ca8c7759a"
May 29 20:49:54.496365 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.496331    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-g84rm"
May 29 20:49:54.506740 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:54.506707    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda30e96d8_0364_483c_8eb1_f7b2dfa3fa6e.slice/crio-3f8e01c5201c1d27cdad44ed03efbafe9e3081a54645749decde872330a8f9f8 WatchSource:0}: Error finding container 3f8e01c5201c1d27cdad44ed03efbafe9e3081a54645749decde872330a8f9f8: Status 404 returned error can't find the container with id 3f8e01c5201c1d27cdad44ed03efbafe9e3081a54645749decde872330a8f9f8
May 29 20:49:54.623789 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.623710    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"]
May 29 20:49:54.626457 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.626433    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.629220 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629043    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 29 20:49:54.629359 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629273    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6r7327mkmqtb\""
May 29 20:49:54.629359 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629334    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 29 20:49:54.629493 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629479    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-b6xvs\""
May 29 20:49:54.629600 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629572    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 29 20:49:54.629685 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629643    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 29 20:49:54.629789 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.629765    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 29 20:49:54.635834 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.635680    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"]
May 29 20:49:54.730170 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z22rv\" (UniqueName: \"kubernetes.io/projected/d496a421-26e2-46c8-8685-4f7ddd609db6-kube-api-access-z22rv\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d496a421-26e2-46c8-8685-4f7ddd609db6-metrics-client-ca\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-grpc-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.730578 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.730533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831157 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831125    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-grpc-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831304 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831171    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831304 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831250    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z22rv\" (UniqueName: \"kubernetes.io/projected/d496a421-26e2-46c8-8685-4f7ddd609db6-kube-api-access-z22rv\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831409 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831303    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831409 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831333    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831526 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831486    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831577 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831550    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.831641 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.831596    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d496a421-26e2-46c8-8685-4f7ddd609db6-metrics-client-ca\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.832410 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.832355    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d496a421-26e2-46c8-8685-4f7ddd609db6-metrics-client-ca\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.834555 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.834527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.834674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.834592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-grpc-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.834731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.834703    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-tls\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.835272 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.835244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.835576 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.835551    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.835679 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.835623    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d496a421-26e2-46c8-8685-4f7ddd609db6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.839432 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.839410    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z22rv\" (UniqueName: \"kubernetes.io/projected/d496a421-26e2-46c8-8685-4f7ddd609db6-kube-api-access-z22rv\") pod \"thanos-querier-56c9fdb949-t5kcq\" (UID: \"d496a421-26e2-46c8-8685-4f7ddd609db6\") " pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:54.939504 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:54.939406    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:55.025766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:55.025727    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g84rm" event={"ID":"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e","Type":"ContainerStarted","Data":"3f8e01c5201c1d27cdad44ed03efbafe9e3081a54645749decde872330a8f9f8"}
May 29 20:49:55.025766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:55.025763    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:49:55.026012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:55.025779    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdb2d"
May 29 20:49:55.295949 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:55.295903    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"]
May 29 20:49:55.305764 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:55.305717    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd496a421_26e2_46c8_8685_4f7ddd609db6.slice/crio-5081a17a326094a75bd66b4a77e0761c4759671f2d3ca0c549f23ad64164b748 WatchSource:0}: Error finding container 5081a17a326094a75bd66b4a77e0761c4759671f2d3ca0c549f23ad64164b748: Status 404 returned error can't find the container with id 5081a17a326094a75bd66b4a77e0761c4759671f2d3ca0c549f23ad64164b748
May 29 20:49:56.030064 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.030031    2563 generic.go:358] "Generic (PLEG): container finished" podID="a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e" containerID="7d45eaddda050202c4ff72cb19f3650003f8f6785f2472b66526489d23e6286e" exitCode=0
May 29 20:49:56.030466 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.030101    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g84rm" event={"ID":"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e","Type":"ContainerDied","Data":"7d45eaddda050202c4ff72cb19f3650003f8f6785f2472b66526489d23e6286e"}
May 29 20:49:56.031280 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.031239    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"5081a17a326094a75bd66b4a77e0761c4759671f2d3ca0c549f23ad64164b748"}
May 29 20:49:56.032934 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.032909    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb" event={"ID":"72a67383-0522-42ab-aeb0-dbd813814032","Type":"ContainerStarted","Data":"f0694ec716d643e50527893ca148d8c945d16402d11ea7369973619b4b9aeaf6"}
May 29 20:49:56.033067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.032939    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb" event={"ID":"72a67383-0522-42ab-aeb0-dbd813814032","Type":"ContainerStarted","Data":"37a2f866ed9ccc271df5f9d5790956d15a946a5023620f8a3ad623172db1b61d"}
May 29 20:49:56.033067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.032955    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb" event={"ID":"72a67383-0522-42ab-aeb0-dbd813814032","Type":"ContainerStarted","Data":"980959f79c29dfb3d021458bc2bdad082ae4b834cef179bda24ca25d2660da94"}
May 29 20:49:56.034681 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.034663    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc" event={"ID":"17b20fd4-cd52-46e5-9ee4-edde38dab402","Type":"ContainerStarted","Data":"30edcb76dd68078f9ccf860f7e491a463053f2c1f343180c2384467e00e2692d"}
May 29 20:49:56.067249 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.067200    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-m45qc" podStartSLOduration=2.755336069 podStartE2EDuration="4.06718345s" podCreationTimestamp="2026-05-29 20:49:52 +0000 UTC" firstStartedPulling="2026-05-29 20:49:53.816223012 +0000 UTC m=+156.880389140" lastFinishedPulling="2026-05-29 20:49:55.128070406 +0000 UTC m=+158.192236521" observedRunningTime="2026-05-29 20:49:56.065432209 +0000 UTC m=+159.129598346" watchObservedRunningTime="2026-05-29 20:49:56.06718345 +0000 UTC m=+159.131349589"
May 29 20:49:56.082967 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:56.082923    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-jvqcb" podStartSLOduration=2.703234043 podStartE2EDuration="4.082904974s" podCreationTimestamp="2026-05-29 20:49:52 +0000 UTC" firstStartedPulling="2026-05-29 20:49:53.751878014 +0000 UTC m=+156.816044156" lastFinishedPulling="2026-05-29 20:49:55.131548952 +0000 UTC m=+158.195715087" observedRunningTime="2026-05-29 20:49:56.081835127 +0000 UTC m=+159.146001288" watchObservedRunningTime="2026-05-29 20:49:56.082904974 +0000 UTC m=+159.147071111"
May 29 20:49:57.040776 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.040732    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g84rm" event={"ID":"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e","Type":"ContainerStarted","Data":"a49c7c9cd1cc1042d8739b90f0720d7d0b690ad266a511f43a2e02ad51af81c4"}
May 29 20:49:57.040776 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.040779    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-g84rm" event={"ID":"a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e","Type":"ContainerStarted","Data":"1388470bcf36e9ffa3fd4807662168f3c708663f740fb226aba0473b497a91e1"}
May 29 20:49:57.062438 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.062392    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-g84rm" podStartSLOduration=3.837936177 podStartE2EDuration="5.062376765s" podCreationTimestamp="2026-05-29 20:49:52 +0000 UTC" firstStartedPulling="2026-05-29 20:49:54.509323849 +0000 UTC m=+157.573489982" lastFinishedPulling="2026-05-29 20:49:55.733764455 +0000 UTC m=+158.797930570" observedRunningTime="2026-05-29 20:49:57.060536503 +0000 UTC m=+160.124702641" watchObservedRunningTime="2026-05-29 20:49:57.062376765 +0000 UTC m=+160.126542913"
May 29 20:49:57.442767 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.441473    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"]
May 29 20:49:57.444526 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.444507    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:49:57.446935 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.446913    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-6qbpw\""
May 29 20:49:57.447077 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.446955    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 29 20:49:57.455445 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.455399    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"]
May 29 20:49:57.565750 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.565719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-w5jzg\" (UID: \"33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:49:57.667162 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.667091    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-w5jzg\" (UID: \"33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:49:57.669576 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.669556    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-w5jzg\" (UID: \"33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:49:57.768032 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.767970    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:49:57.891234 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:57.891204    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"]
May 29 20:49:57.894006 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:57.893949    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1bdf8_6ef7_49e5_92ae_c001ccbf0b3d.slice/crio-ecf0d57e396cbd496ebbe78ab3548ddf5c083468339304cbefc601caf68da60f WatchSource:0}: Error finding container ecf0d57e396cbd496ebbe78ab3548ddf5c083468339304cbefc601caf68da60f: Status 404 returned error can't find the container with id ecf0d57e396cbd496ebbe78ab3548ddf5c083468339304cbefc601caf68da60f
May 29 20:49:58.044800 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.044764    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg" event={"ID":"33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d","Type":"ContainerStarted","Data":"ecf0d57e396cbd496ebbe78ab3548ddf5c083468339304cbefc601caf68da60f"}
May 29 20:49:58.046364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.046337    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"b2daacb71b258574c4470ec4140d8f0bb248ec57ce52adcfdb4b3e158d183ed8"}
May 29 20:49:58.046459 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.046373    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"80a75e2e2a00860d0299e633b1b801117267e3ea7e2be8b7b483e116bd17b207"}
May 29 20:49:58.046459 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.046387    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"75af6bb612b2b53e5888c46775740d9d567958810d370336989db9e2b7bfa51c"}
May 29 20:49:58.913907 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.913821    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:49:58.916494 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.916475    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.919026 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.919005    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 20:49:58.919231 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.919208    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 20:49:58.919458 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.919442    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3o8cqce670upg\""
May 29 20:49:58.919523 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.919457    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 20:49:58.920631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.920353    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 20:49:58.920631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.920407    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 20:49:58.920631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.920415    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 20:49:58.920631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.920463    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 20:49:58.920631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.920611    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2jc44\""
May 29 20:49:58.921223 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.921034    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 20:49:58.921223 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.921072    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 20:49:58.921223 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.921087    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 20:49:58.921223 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.921180    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 20:49:58.922866 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.922846    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 20:49:58.929268 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.929247    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 20:49:58.934512 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.934488    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:49:58.980515 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjlgf\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.980957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.980953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.981137 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.981038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:58.981137 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:58.981072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.052114 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.052079    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"9881d71b72c4310a48ef442df5f170fa39e1f5d134a2ddf0872debb66195ace6"}
May 29 20:49:59.052114 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.052118    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"26ddb13ec147c2a1a7be1fd96c4597ec016e9c72717e17072be886fd50203d1e"}
May 29 20:49:59.052598 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.052134    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" event={"ID":"d496a421-26e2-46c8-8685-4f7ddd609db6","Type":"ContainerStarted","Data":"07f959c10ddbe882c4cf02892ec3422d3af26a5f70e8483105d60afbdaf5f978"}
May 29 20:49:59.052598 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.052272    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:49:59.079749 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.079694    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq" podStartSLOduration=1.8637598579999999 podStartE2EDuration="5.079678345s" podCreationTimestamp="2026-05-29 20:49:54 +0000 UTC" firstStartedPulling="2026-05-29 20:49:55.307551315 +0000 UTC m=+158.371717437" lastFinishedPulling="2026-05-29 20:49:58.52346968 +0000 UTC m=+161.587635924" observedRunningTime="2026-05-29 20:49:59.079023511 +0000 UTC m=+162.143189651" watchObservedRunningTime="2026-05-29 20:49:59.079678345 +0000 UTC m=+162.143844482"
May 29 20:49:59.081698 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081667    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.081833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081710    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.081833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081739    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.081833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081765    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.081833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081797    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.081833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081822    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081854    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081888    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081915    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081942    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.081968    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082024    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082122 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082073    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082130    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082165    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082198    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082246    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.082446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082271    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjlgf\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.084235 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082922    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.084235 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.082979    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.084235 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.084214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.085954 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.085269    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.085954 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.085672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.086128 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.085958    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.086192 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.086122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.086429 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.086407    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.086644 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.086592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.087238 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.087213    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.087349 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.087262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.088259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.088223    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.088613 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.088593    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.089144 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.089105    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.089284 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.089263    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.089461 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.089420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.089825 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.089802    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.094946 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.094926    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjlgf\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf\") pod \"prometheus-k8s-0\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.183437 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.183338    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:49:59.183437 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.183424    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:49:59.186045 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.186020    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d207e7d-a860-433d-ba4f-67fcd022b23f-metrics-tls\") pod \"dns-default-jdb2d\" (UID: \"7d207e7d-a860-433d-ba4f-67fcd022b23f\") " pod="openshift-dns/dns-default-jdb2d"
May 29 20:49:59.186151 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.186047    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f74cd5ec-fc29-41c8-8986-230ca8c7759a-cert\") pod \"ingress-canary-q689n\" (UID: \"f74cd5ec-fc29-41c8-8986-230ca8c7759a\") " pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:49:59.229447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.229414    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7hbwz\""
May 29 20:49:59.230155 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.230067    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5d94t\""
May 29 20:49:59.230155 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.230116    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:49:59.238061 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.238040    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdb2d"
May 29 20:49:59.238171 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.238139    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q689n"
May 29 20:49:59.493111 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.493079    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdb2d"]
May 29 20:49:59.496775 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:59.496742    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d207e7d_a860_433d_ba4f_67fcd022b23f.slice/crio-d6dfcaf5c07aa3451cd8165b938d7a8c6adf9c2cbe1b8ca8dd0d94da4ec8d8fb WatchSource:0}: Error finding container d6dfcaf5c07aa3451cd8165b938d7a8c6adf9c2cbe1b8ca8dd0d94da4ec8d8fb: Status 404 returned error can't find the container with id d6dfcaf5c07aa3451cd8165b938d7a8c6adf9c2cbe1b8ca8dd0d94da4ec8d8fb
May 29 20:49:59.720931 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.720904    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q689n"]
May 29 20:49:59.722723 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:59.722698    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74cd5ec_fc29_41c8_8986_230ca8c7759a.slice/crio-6e33ab2b2c3bb6d58c3dce6af456762a53c4391fdda1b829b9fccf1303cfd67a WatchSource:0}: Error finding container 6e33ab2b2c3bb6d58c3dce6af456762a53c4391fdda1b829b9fccf1303cfd67a: Status 404 returned error can't find the container with id 6e33ab2b2c3bb6d58c3dce6af456762a53c4391fdda1b829b9fccf1303cfd67a
May 29 20:49:59.730242 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:49:59.730217    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:49:59.733265 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:49:59.733243    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c99d52_e5a0_4ef7_bb2e_2ba9b769e915.slice/crio-7a89c970e368235359fe3fe0987676e98424c5b1e60abb819f41ba8df851f5e1 WatchSource:0}: Error finding container 7a89c970e368235359fe3fe0987676e98424c5b1e60abb819f41ba8df851f5e1: Status 404 returned error can't find the container with id 7a89c970e368235359fe3fe0987676e98424c5b1e60abb819f41ba8df851f5e1
May 29 20:50:00.056857 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.056816    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg" event={"ID":"33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d","Type":"ContainerStarted","Data":"699b8163450c679ac1b470f5c18fd9de2d0b1ea0e858f3b21343ee89c5b853ea"}
May 29 20:50:00.057346 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.057202    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:50:00.058653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.058616    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q689n" event={"ID":"f74cd5ec-fc29-41c8-8986-230ca8c7759a","Type":"ContainerStarted","Data":"6e33ab2b2c3bb6d58c3dce6af456762a53c4391fdda1b829b9fccf1303cfd67a"}
May 29 20:50:00.060065 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.060040    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdb2d" event={"ID":"7d207e7d-a860-433d-ba4f-67fcd022b23f","Type":"ContainerStarted","Data":"d6dfcaf5c07aa3451cd8165b938d7a8c6adf9c2cbe1b8ca8dd0d94da4ec8d8fb"}
May 29 20:50:00.061259 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.061233    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"7a89c970e368235359fe3fe0987676e98424c5b1e60abb819f41ba8df851f5e1"}
May 29 20:50:00.063092 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.063063    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg"
May 29 20:50:00.072220 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:00.072135    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-w5jzg" podStartSLOduration=1.61571196 podStartE2EDuration="3.072120446s" podCreationTimestamp="2026-05-29 20:49:57 +0000 UTC" firstStartedPulling="2026-05-29 20:49:57.895777029 +0000 UTC m=+160.959943143" lastFinishedPulling="2026-05-29 20:49:59.352185499 +0000 UTC m=+162.416351629" observedRunningTime="2026-05-29 20:50:00.071895219 +0000 UTC m=+163.136061361" watchObservedRunningTime="2026-05-29 20:50:00.072120446 +0000 UTC m=+163.136286585"
May 29 20:50:02.074939 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.074898    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q689n" event={"ID":"f74cd5ec-fc29-41c8-8986-230ca8c7759a","Type":"ContainerStarted","Data":"d5d813338f20db51f69f863c67f42a1352959c06265bc0e516971312e8d98e66"}
May 29 20:50:02.076495 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.076471    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdb2d" event={"ID":"7d207e7d-a860-433d-ba4f-67fcd022b23f","Type":"ContainerStarted","Data":"cd6ac641d29d64b0424dba1cd5d2aa5f9e8d81b379744ced214018bb7ca208d7"}
May 29 20:50:02.076495 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.076497    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdb2d" event={"ID":"7d207e7d-a860-433d-ba4f-67fcd022b23f","Type":"ContainerStarted","Data":"725594838aa226e7909c2005cb4f5a4c95b588f989493cc368086a51261458ce"}
May 29 20:50:02.076691 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.076583    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jdb2d"
May 29 20:50:02.077667 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.077645    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6" exitCode=0
May 29 20:50:02.077760 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.077721    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"}
May 29 20:50:02.106274 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.106221    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-jdb2d" podStartSLOduration=128.991126559 podStartE2EDuration="2m11.106207234s" podCreationTimestamp="2026-05-29 20:47:51 +0000 UTC" firstStartedPulling="2026-05-29 20:49:59.498857298 +0000 UTC m=+162.563023416" lastFinishedPulling="2026-05-29 20:50:01.613937976 +0000 UTC m=+164.678104091" observedRunningTime="2026-05-29 20:50:02.105912497 +0000 UTC m=+165.170078633" watchObservedRunningTime="2026-05-29 20:50:02.106207234 +0000 UTC m=+165.170373370"
May 29 20:50:02.107631 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:02.107582    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q689n" podStartSLOduration=129.214607019 podStartE2EDuration="2m11.107570919s" podCreationTimestamp="2026-05-29 20:47:51 +0000 UTC" firstStartedPulling="2026-05-29 20:49:59.724705301 +0000 UTC m=+162.788871419" lastFinishedPulling="2026-05-29 20:50:01.617669203 +0000 UTC m=+164.681835319" observedRunningTime="2026-05-29 20:50:02.090690396 +0000 UTC m=+165.154856538" watchObservedRunningTime="2026-05-29 20:50:02.107570919 +0000 UTC m=+165.171737107"
May 29 20:50:05.068930 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:05.068839    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56c9fdb949-t5kcq"
May 29 20:50:06.093017 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.092956    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"}
May 29 20:50:06.093017 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.093022    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"}
May 29 20:50:06.093443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.093037    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"}
May 29 20:50:06.093443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.093050    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"}
May 29 20:50:06.093443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.093063    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"}
May 29 20:50:06.093443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.093073    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerStarted","Data":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"}
May 29 20:50:06.121266 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:06.121177    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.629492107 podStartE2EDuration="8.121161315s" podCreationTimestamp="2026-05-29 20:49:58 +0000 UTC" firstStartedPulling="2026-05-29 20:49:59.735381372 +0000 UTC m=+162.799547487" lastFinishedPulling="2026-05-29 20:50:05.227050575 +0000 UTC m=+168.291216695" observedRunningTime="2026-05-29 20:50:06.120323623 +0000 UTC m=+169.184489760" watchObservedRunningTime="2026-05-29 20:50:06.121161315 +0000 UTC m=+169.185327454"
May 29 20:50:09.230513 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:09.230479    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:50:12.083129 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:12.083101    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jdb2d"
May 29 20:50:59.231226 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:59.231166    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:50:59.251282 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:59.251251    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:50:59.266337 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:50:59.266313    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:17.363535 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.363497    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364148    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-web" containerID="cri-o://bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832" gracePeriod=600
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364148    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-thanos" containerID="cri-o://57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493" gracePeriod=600
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364170    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="prometheus" containerID="cri-o://db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd" gracePeriod=600
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364219    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy" containerID="cri-o://f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4" gracePeriod=600
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364272    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="config-reloader" containerID="cri-o://739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb" gracePeriod=600
May 29 20:51:17.364450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.364170    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="thanos-sidecar" containerID="cri-o://3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852" gracePeriod=600
May 29 20:51:17.599588 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.599565    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:17.653838 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653746    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.653838 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653789    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653816    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653886    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjlgf\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653923    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.653983    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654025    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654113 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654049    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654075    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654266    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654314    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654349    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654355    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:51:17.654406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654380    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654420    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654492    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654683    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654712    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.654925 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.654736    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db\") pod \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\" (UID: \"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915\") "
May 29 20:51:17.655177 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.655028    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-metrics-client-ca\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.658796 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.658763    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:51:17.658912 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.658800    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:51:17.659618 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659373    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.659618 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659472    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:51:17.659771 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659531    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.659771 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659743    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:51:17.659771 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659745    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf" (OuterVolumeSpecName: "kube-api-access-gjlgf") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "kube-api-access-gjlgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:51:17.659910 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659832    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.659910 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659838    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:51:17.659910 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.659892    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config" (OuterVolumeSpecName: "config") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.661023 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.660445    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.661023 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.660930    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out" (OuterVolumeSpecName: "config-out") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:51:17.662148 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.662107    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.662725 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.662625    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:51:17.666154 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.664123    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.667497 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.667049    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.673038 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.673018    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config" (OuterVolumeSpecName: "web-config") pod "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" (UID: "72c99d52-e5a0-4ef7-bb2e-2ba9b769e915"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 29 20:51:17.755597 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755561    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755597 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755590    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755597 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755601    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjlgf\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-kube-api-access-gjlgf\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755613    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-metrics-client-certs\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755623    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755632    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-tls-assets\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755640    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-grpc-tls\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755648    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-kube-rbac-proxy\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755657    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-web-config\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755666    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755674    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755683    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755692    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-config-out\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755701    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755709    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755717    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:17.755821 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:17.755726    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915-prometheus-k8s-db\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298576    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493" exitCode=0
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298601    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4" exitCode=0
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298607    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832" exitCode=0
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298612    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852" exitCode=0
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298617    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb" exitCode=0
May 29 20:51:18.298610 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298623    2563 generic.go:358] "Generic (PLEG): container finished" podID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd" exitCode=0
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298681    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298699    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298709    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298721    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298730    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298740    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298750    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298760    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"72c99d52-e5a0-4ef7-bb2e-2ba9b769e915","Type":"ContainerDied","Data":"7a89c970e368235359fe3fe0987676e98424c5b1e60abb819f41ba8df851f5e1"}
May 29 20:51:18.298941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.298775    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.306402 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.306384    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.312899 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.312884    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.319421 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.319406    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.325942 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.325919    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:18.326509 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.326492    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.330144 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.330123    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:18.333897 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.333880    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.340918 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.340902    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.347451 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.347435    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.347689 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.347672    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.347753 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.347697    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.347753 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.347717    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.347939 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.347922    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.347978 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.347943    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.347978 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.347957    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.348171 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.348149    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.348206 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348167    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.348206 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348180    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.348358 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.348343    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.348396 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348359    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.348396 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348371    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.348546 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.348529    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.348584 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348549    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.348584 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348561    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.348740 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.348725    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.348784 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348743    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.348784 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348754    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.348913 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:51:18.348897    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.348952 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348915    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.348952 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.348925    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.349146 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349125    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.349194 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349150    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.349332 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349317    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.349372 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349333    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.349534 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349513    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.349572 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349535    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.349713 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349698    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.349713 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349713    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.349926 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349904    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.349971 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.349927    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.350158 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350142    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.350214 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350159    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.350320 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350306    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.350367 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350320    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.350495 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350475    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.350556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350502    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.350761 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350740    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.350808 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350763    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.350962 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350943    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.351055 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.350963    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.351281 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351262    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.351349 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351282    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.351501 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351483    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.351553 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351502    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.351709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351692    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.351775 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351711    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.351922 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351907    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.351985 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.351923    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.352132 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352112    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.352174 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352132    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.352314 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352300    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.352356 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352315    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.352509 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352492    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.352554 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352509    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.352712 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352694    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.352758 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352714    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.352895 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352880    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.352936 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.352895    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.353095 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353080    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.353139 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353096    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.353319 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353299    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.353377 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353318    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.353547 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353529    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.353600 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353548    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.353755 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353740    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.353799 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353755    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.353934 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353920    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.353981 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.353934    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.354161 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354143    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.354205 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354161    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.354377 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354360    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.354428 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354377    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.354581 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354566    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.354627 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354581    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.354773 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354758    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.354815 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354774    2563 scope.go:117] "RemoveContainer" containerID="57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"
May 29 20:51:18.354955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354939    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493"} err="failed to get container status \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": rpc error: code = NotFound desc = could not find container \"57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493\": container with ID starting with 57db6915426d4877727f9ce48688fdf8890b2ffb17a7ca7e8963956c2fad7493 not found: ID does not exist"
May 29 20:51:18.355043 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.354955    2563 scope.go:117] "RemoveContainer" containerID="f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"
May 29 20:51:18.355395 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355363    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4"} err="failed to get container status \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": rpc error: code = NotFound desc = could not find container \"f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4\": container with ID starting with f2109639f47af1be6b7f83cbdfd3e071cf30d62b27abc363e1720bf87046a8c4 not found: ID does not exist"
May 29 20:51:18.355395 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355392    2563 scope.go:117] "RemoveContainer" containerID="bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"
May 29 20:51:18.355892 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355643    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832"} err="failed to get container status \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": rpc error: code = NotFound desc = could not find container \"bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832\": container with ID starting with bdaddec22cade7e22e4c623dec788a054a3c81053a9e4f33bd7a4001173e8832 not found: ID does not exist"
May 29 20:51:18.355892 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355667    2563 scope.go:117] "RemoveContainer" containerID="3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"
May 29 20:51:18.356064 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355905    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852"} err="failed to get container status \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": rpc error: code = NotFound desc = could not find container \"3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852\": container with ID starting with 3735e2e8e64c755366cc01dabc8a512d2a7bc73381836899270eb49387db5852 not found: ID does not exist"
May 29 20:51:18.356064 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.355928    2563 scope.go:117] "RemoveContainer" containerID="739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"
May 29 20:51:18.356203 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.356176    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb"} err="failed to get container status \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": rpc error: code = NotFound desc = could not find container \"739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb\": container with ID starting with 739fa5bdc32dab6a1022c624c962a9dad27533cab0000f52fc8576e0aaa6ddcb not found: ID does not exist"
May 29 20:51:18.356203 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.356203    2563 scope.go:117] "RemoveContainer" containerID="db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"
May 29 20:51:18.356582 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.356558    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd"} err="failed to get container status \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": rpc error: code = NotFound desc = could not find container \"db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd\": container with ID starting with db2e3613c41fc03b89af909c7e2907fb0d5b6eefb63bba37f507e690bc138bdd not found: ID does not exist"
May 29 20:51:18.356656 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.356584    2563 scope.go:117] "RemoveContainer" containerID="eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"
May 29 20:51:18.356841 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.356821    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6"} err="failed to get container status \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": rpc error: code = NotFound desc = could not find container \"eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6\": container with ID starting with eb477c029b1550bce22fffe05e6b877f44cb62097b693d4d5b4eef80303760f6 not found: ID does not exist"
May 29 20:51:18.357281 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357265    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:18.357581 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357569    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="prometheus"
May 29 20:51:18.357615 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="prometheus"
May 29 20:51:18.357615 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357600    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-thanos"
May 29 20:51:18.357615 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357606    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-thanos"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357620    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="thanos-sidecar"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357629    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="thanos-sidecar"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357638    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357643    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357653    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-web"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357659    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-web"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357665    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="config-reloader"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357669    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="config-reloader"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357677    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="init-config-reloader"
May 29 20:51:18.357702 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="init-config-reloader"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357735    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357743    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="config-reloader"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357748    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-thanos"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357755    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="kube-rbac-proxy-web"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357762    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="thanos-sidecar"
May 29 20:51:18.357955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.357769    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" containerName="prometheus"
May 29 20:51:18.361348 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.361333    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.363948 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.363931    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
May 29 20:51:18.364310 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364071    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
May 29 20:51:18.364310 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364105    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
May 29 20:51:18.364310 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364147    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
May 29 20:51:18.364457 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364325    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
May 29 20:51:18.364600 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364583    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 29 20:51:18.364642 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364603    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
May 29 20:51:18.364642 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.364583    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
May 29 20:51:18.365186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.365170    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
May 29 20:51:18.365186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.365181    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2jc44\""
May 29 20:51:18.365482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.365467    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
May 29 20:51:18.365534 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.365518    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3o8cqce670upg\""
May 29 20:51:18.365754 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.365740    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
May 29 20:51:18.369052 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.369032    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
May 29 20:51:18.371337 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.371316    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
May 29 20:51:18.374267 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.374248    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:18.461318 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k4ck\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-kube-api-access-5k4ck\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461611 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.461839 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.462085 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.461887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.562956 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.562884    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.562956 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.562918    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.562956 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.562936    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.562956 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.562954    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.562977    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563035    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563073    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563107    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563137    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563166    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563227 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563210    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563230    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563249    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5k4ck\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-kube-api-access-5k4ck\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563276    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563297    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563328    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563360    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.563542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.563392    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.565868 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.565839    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-web-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.566242 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.566217    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.566559 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.566451    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.566559 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.566527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.567514 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.566892    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.567514 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.567201    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.567514 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.567244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.567514 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.567266    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.567768 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.567696    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.568035 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.568014    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.568181 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.568161    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.568390 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.568368    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2537471a-58be-44e1-b6be-0b7e5cd33fe6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.568767 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.568747    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.568968 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.568946    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.569538 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.569516    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.570131 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.570109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2537471a-58be-44e1-b6be-0b7e5cd33fe6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.570255 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.570239    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2537471a-58be-44e1-b6be-0b7e5cd33fe6-config-out\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.571964 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.571942    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k4ck\" (UniqueName: \"kubernetes.io/projected/2537471a-58be-44e1-b6be-0b7e5cd33fe6-kube-api-access-5k4ck\") pod \"prometheus-k8s-0\" (UID: \"2537471a-58be-44e1-b6be-0b7e5cd33fe6\") " pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.671330 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.671293    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:51:18.808026 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:18.807982    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
May 29 20:51:18.810400 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:51:18.810373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2537471a_58be_44e1_b6be_0b7e5cd33fe6.slice/crio-0c7b6f1862457292e2be7dec257538f9145c4113f2d8efcc3cf55288e4cd52a0 WatchSource:0}: Error finding container 0c7b6f1862457292e2be7dec257538f9145c4113f2d8efcc3cf55288e4cd52a0: Status 404 returned error can't find the container with id 0c7b6f1862457292e2be7dec257538f9145c4113f2d8efcc3cf55288e4cd52a0
May 29 20:51:19.302480 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:19.302446    2563 generic.go:358] "Generic (PLEG): container finished" podID="2537471a-58be-44e1-b6be-0b7e5cd33fe6" containerID="71936294e1cf72309fe2a55c2d1829174d88db41a6533938f599c02067363fbf" exitCode=0
May 29 20:51:19.302665 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:19.302536    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerDied","Data":"71936294e1cf72309fe2a55c2d1829174d88db41a6533938f599c02067363fbf"}
May 29 20:51:19.302665 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:19.302578    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"0c7b6f1862457292e2be7dec257538f9145c4113f2d8efcc3cf55288e4cd52a0"}
May 29 20:51:19.504388 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:19.504324    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72c99d52-e5a0-4ef7-bb2e-2ba9b769e915" path="/var/lib/kubelet/pods/72c99d52-e5a0-4ef7-bb2e-2ba9b769e915/volumes"
May 29 20:51:20.309829 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309798    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"75bba8383453538a69ba101966aa3a7d737bf6d0b2fed3ff4e602b93b8195fc1"}
May 29 20:51:20.309829 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309831    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"78b131a8260871fa39ce4196eebb491b1722829bcb3ba93b9bba970b4f7f0780"}
May 29 20:51:20.310057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309841    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"dae37e1dd770189b8ff21ec7aa26f995027719f6fe622804791f5c750746267d"}
May 29 20:51:20.310057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309850    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"aafbf3971cf8d70b1ead251b020a2a2cb84d64b5e6b16318b606f6d11d4e0e42"}
May 29 20:51:20.310057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309858    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"3141678c5e78590a6506b87669b33f11eff07e2f7b1d9db4029254c77f6e20fc"}
May 29 20:51:20.310057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.309865    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2537471a-58be-44e1-b6be-0b7e5cd33fe6","Type":"ContainerStarted","Data":"d2946a87aba917d9415c3631b33b698fc50a5cd731a54affb8d9094b7124d825"}
May 29 20:51:20.346675 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:20.346582    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.346563392 podStartE2EDuration="2.346563392s" podCreationTimestamp="2026-05-29 20:51:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:51:20.344963867 +0000 UTC m=+243.409130030" watchObservedRunningTime="2026-05-29 20:51:20.346563392 +0000 UTC m=+243.410729535"
May 29 20:51:23.672424 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:51:23.672375    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:52:17.362232 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:17.362203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:52:17.367647 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:17.367621    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:52:17.369133 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:17.369110    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:52:17.371309 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:17.371289    2563 kubelet.go:1628] "Image garbage collection succeeded"
May 29 20:52:17.375946 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:17.375930    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:52:18.671478 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:18.671430    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:52:18.687625 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:18.687598    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:52:19.499722 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:52:19.499694    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
May 29 20:53:16.058572 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.058485    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tgwnn"]
May 29 20:53:16.060800 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.060783    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.063253 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.063229    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 29 20:53:16.070055 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.070032    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tgwnn"]
May 29 20:53:16.140432 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.140392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-kubelet-config\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.140669 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.140452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f07a1d7-f5d2-4d11-971a-745b24aed88f-original-pull-secret\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.140669 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.140546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-dbus\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.241370 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.241336    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-kubelet-config\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.241532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.241390    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f07a1d7-f5d2-4d11-971a-745b24aed88f-original-pull-secret\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.241532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.241428    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-dbus\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.241532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.241449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-kubelet-config\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.241694 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.241564    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f07a1d7-f5d2-4d11-971a-745b24aed88f-dbus\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.243797 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.243779    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f07a1d7-f5d2-4d11-971a-745b24aed88f-original-pull-secret\") pod \"global-pull-secret-syncer-tgwnn\" (UID: \"3f07a1d7-f5d2-4d11-971a-745b24aed88f\") " pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.370571 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.370494    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tgwnn"
May 29 20:53:16.495775 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.495746    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tgwnn"]
May 29 20:53:16.498821 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:53:16.498793    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f07a1d7_f5d2_4d11_971a_745b24aed88f.slice/crio-0ff95117b626fd75a3b4b68fb58f609c9b6b55b92389d0933178606ae6bb80b8 WatchSource:0}: Error finding container 0ff95117b626fd75a3b4b68fb58f609c9b6b55b92389d0933178606ae6bb80b8: Status 404 returned error can't find the container with id 0ff95117b626fd75a3b4b68fb58f609c9b6b55b92389d0933178606ae6bb80b8
May 29 20:53:16.500468 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.500448    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 20:53:16.641943 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:16.641868    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tgwnn" event={"ID":"3f07a1d7-f5d2-4d11-971a-745b24aed88f","Type":"ContainerStarted","Data":"0ff95117b626fd75a3b4b68fb58f609c9b6b55b92389d0933178606ae6bb80b8"}
May 29 20:53:20.654685 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:20.654600    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tgwnn" event={"ID":"3f07a1d7-f5d2-4d11-971a-745b24aed88f","Type":"ContainerStarted","Data":"f26347cd76ec44b503b27f70b30b8ea7a664bb4418b1ff4b3a18ed8dadb1d9a7"}
May 29 20:53:20.671453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:53:20.671403    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tgwnn" podStartSLOduration=0.831907105 podStartE2EDuration="4.671381365s" podCreationTimestamp="2026-05-29 20:53:16 +0000 UTC" firstStartedPulling="2026-05-29 20:53:16.500569823 +0000 UTC m=+359.564735938" lastFinishedPulling="2026-05-29 20:53:20.340044084 +0000 UTC m=+363.404210198" observedRunningTime="2026-05-29 20:53:20.670626151 +0000 UTC m=+363.734792287" watchObservedRunningTime="2026-05-29 20:53:20.671381365 +0000 UTC m=+363.735547502"
May 29 20:54:53.343311 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.343227    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"]
May 29 20:54:53.345942 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.345920    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.348447 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.348424    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8ttjm\""
May 29 20:54:53.348614 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.348598    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 29 20:54:53.348870 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.348856    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 29 20:54:53.349342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.349327    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 29 20:54:53.349390 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.349332    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 29 20:54:53.360407 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.360383    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"]
May 29 20:54:53.452189 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.452144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlnxt\" (UniqueName: \"kubernetes.io/projected/1f599285-5f82-44e0-a230-bfcac8643330-kube-api-access-vlnxt\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.452363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.452196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-manifests\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.452363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.452282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.452363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.452316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.452363 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.452340    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-tmp\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.552833 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.552789    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vlnxt\" (UniqueName: \"kubernetes.io/projected/1f599285-5f82-44e0-a230-bfcac8643330-kube-api-access-vlnxt\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.553039 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.552842    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-manifests\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.553039 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.552892    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.553039 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.552917    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.553039 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.552954    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-tmp\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.553383 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.553363    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-manifests\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.555898 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.555872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599285-5f82-44e0-a230-bfcac8643330-tmp\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.556173 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.556152    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.556313 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.556291    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599285-5f82-44e0-a230-bfcac8643330-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.561393 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.561367    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlnxt\" (UniqueName: \"kubernetes.io/projected/1f599285-5f82-44e0-a230-bfcac8643330-kube-api-access-vlnxt\") pod \"opendatahub-operator-controller-manager-6c4c488574-q96fl\" (UID: \"1f599285-5f82-44e0-a230-bfcac8643330\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.655736 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.655646    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:53.787887 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.787857    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"]
May 29 20:54:53.791020 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:54:53.790972    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f599285_5f82_44e0_a230_bfcac8643330.slice/crio-37dfd857883ffbda972da20e9ffe26e4fb78eca9f121315210969f445aaba529 WatchSource:0}: Error finding container 37dfd857883ffbda972da20e9ffe26e4fb78eca9f121315210969f445aaba529: Status 404 returned error can't find the container with id 37dfd857883ffbda972da20e9ffe26e4fb78eca9f121315210969f445aaba529
May 29 20:54:53.927010 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:53.926889    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl" event={"ID":"1f599285-5f82-44e0-a230-bfcac8643330","Type":"ContainerStarted","Data":"37dfd857883ffbda972da20e9ffe26e4fb78eca9f121315210969f445aaba529"}
May 29 20:54:56.939195 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:56.939156    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl" event={"ID":"1f599285-5f82-44e0-a230-bfcac8643330","Type":"ContainerStarted","Data":"98bab39a40a6365863ce6cda58d935cc514bf1996f5d3b17377d683fd79cbc49"}
May 29 20:54:57.943483 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:57.943447    2563 generic.go:358] "Generic (PLEG): container finished" podID="1f599285-5f82-44e0-a230-bfcac8643330" containerID="98bab39a40a6365863ce6cda58d935cc514bf1996f5d3b17377d683fd79cbc49" exitCode=0
May 29 20:54:57.943865 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:57.943501    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl" event={"ID":"1f599285-5f82-44e0-a230-bfcac8643330","Type":"ContainerDied","Data":"98bab39a40a6365863ce6cda58d935cc514bf1996f5d3b17377d683fd79cbc49"}
May 29 20:54:58.948941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:58.948898    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl" event={"ID":"1f599285-5f82-44e0-a230-bfcac8643330","Type":"ContainerStarted","Data":"c9278d45e725bde177d3483dfad841362f8c3673cbce2c454203e326d9ac2893"}
May 29 20:54:58.949405 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:58.949032    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:54:58.973465 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:54:58.973405    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl" podStartSLOduration=1.487732189 podStartE2EDuration="5.973385242s" podCreationTimestamp="2026-05-29 20:54:53 +0000 UTC" firstStartedPulling="2026-05-29 20:54:53.792663909 +0000 UTC m=+456.856830023" lastFinishedPulling="2026-05-29 20:54:58.278316961 +0000 UTC m=+461.342483076" observedRunningTime="2026-05-29 20:54:58.971099366 +0000 UTC m=+462.035265502" watchObservedRunningTime="2026-05-29 20:54:58.973385242 +0000 UTC m=+462.037551381"
May 29 20:55:09.956379 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:09.956346    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4c488574-q96fl"
May 29 20:55:21.238212 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.238173    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"]
May 29 20:55:21.244490 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.244468    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.246820 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.246798    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 29 20:55:21.247843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247613    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zjm4x\""
May 29 20:55:21.247843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247658    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 29 20:55:21.247843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247617    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 29 20:55:21.247843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247709    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 29 20:55:21.247843 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247750    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 29 20:55:21.248107 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.247866    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"]
May 29 20:55:21.393732 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.393692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.393893 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.393758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44f11fcd-aab2-4b43-a11f-710ce1e4428d-manager-config\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.393893 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.393785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-metrics-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.394016 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.393887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv7gs\" (UniqueName: \"kubernetes.io/projected/44f11fcd-aab2-4b43-a11f-710ce1e4428d-kube-api-access-lv7gs\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.495299 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.495220    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.495299 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.495272    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44f11fcd-aab2-4b43-a11f-710ce1e4428d-manager-config\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.495299 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.495300    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-metrics-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.495542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.495409    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lv7gs\" (UniqueName: \"kubernetes.io/projected/44f11fcd-aab2-4b43-a11f-710ce1e4428d-kube-api-access-lv7gs\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.496051 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.495961    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44f11fcd-aab2-4b43-a11f-710ce1e4428d-manager-config\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.498079 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.498052    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-metrics-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.498079 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.498061    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44f11fcd-aab2-4b43-a11f-710ce1e4428d-cert\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.507790 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.507765    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv7gs\" (UniqueName: \"kubernetes.io/projected/44f11fcd-aab2-4b43-a11f-710ce1e4428d-kube-api-access-lv7gs\") pod \"lws-controller-manager-5cdf9b5b7c-z5k5s\" (UID: \"44f11fcd-aab2-4b43-a11f-710ce1e4428d\") " pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.554755 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.554722    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:21.680898 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:21.680860    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"]
May 29 20:55:21.683516 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:55:21.683489    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f11fcd_aab2_4b43_a11f_710ce1e4428d.slice/crio-26f955610c63edcfcb368a6f26610268d6c84b2b28fec57ac5d513d3c09b8bea WatchSource:0}: Error finding container 26f955610c63edcfcb368a6f26610268d6c84b2b28fec57ac5d513d3c09b8bea: Status 404 returned error can't find the container with id 26f955610c63edcfcb368a6f26610268d6c84b2b28fec57ac5d513d3c09b8bea
May 29 20:55:22.028936 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:22.028898    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s" event={"ID":"44f11fcd-aab2-4b43-a11f-710ce1e4428d","Type":"ContainerStarted","Data":"26f955610c63edcfcb368a6f26610268d6c84b2b28fec57ac5d513d3c09b8bea"}
May 29 20:55:25.041127 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:25.041089    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s" event={"ID":"44f11fcd-aab2-4b43-a11f-710ce1e4428d","Type":"ContainerStarted","Data":"4f66b8b32206641a1820df10f2af40cd319fde916c6a6434bc23ff7540942b45"}
May 29 20:55:25.041519 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:25.041215    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:25.060752 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:25.060693    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s" podStartSLOduration=1.165476088 podStartE2EDuration="4.060680638s" podCreationTimestamp="2026-05-29 20:55:21 +0000 UTC" firstStartedPulling="2026-05-29 20:55:21.685245789 +0000 UTC m=+484.749411918" lastFinishedPulling="2026-05-29 20:55:24.580450352 +0000 UTC m=+487.644616468" observedRunningTime="2026-05-29 20:55:25.059128427 +0000 UTC m=+488.123294569" watchObservedRunningTime="2026-05-29 20:55:25.060680638 +0000 UTC m=+488.124846774"
May 29 20:55:30.936785 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:30.936746    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"]
May 29 20:55:30.943432 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:30.943409    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:30.945753 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:30.945727    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9jb8w\""
May 29 20:55:30.945753 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:30.945738    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\""
May 29 20:55:30.949012 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:30.948968    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"]
May 29 20:55:31.090245 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.090204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9c7z\" (UniqueName: \"kubernetes.io/projected/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-kube-api-access-t9c7z\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.090437 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.090257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-redirect-config\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.191489 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.191402    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t9c7z\" (UniqueName: \"kubernetes.io/projected/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-kube-api-access-t9c7z\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.191489 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.191466    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-redirect-config\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.192139 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.192121    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-redirect-config\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.199684 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.199660    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9c7z\" (UniqueName: \"kubernetes.io/projected/f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2-kube-api-access-t9c7z\") pod \"dashboard-redirect-6b77dbffcd-2wbhr\" (UID: \"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2\") " pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.254614 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.254580    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"
May 29 20:55:31.410073 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:31.410038    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6b77dbffcd-2wbhr"]
May 29 20:55:31.413842 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:55:31.413802    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dd10a5_2ad6_4a46_ae06_e4eeefc3a4b2.slice/crio-1ffe47ed46e048893ceef1581e970fa5b50205714196cc79aa723e8ef452a702 WatchSource:0}: Error finding container 1ffe47ed46e048893ceef1581e970fa5b50205714196cc79aa723e8ef452a702: Status 404 returned error can't find the container with id 1ffe47ed46e048893ceef1581e970fa5b50205714196cc79aa723e8ef452a702
May 29 20:55:32.065856 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:32.065808    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr" event={"ID":"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2","Type":"ContainerStarted","Data":"1ffe47ed46e048893ceef1581e970fa5b50205714196cc79aa723e8ef452a702"}
May 29 20:55:34.075057 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:34.075016    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr" event={"ID":"f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2","Type":"ContainerStarted","Data":"4051890a86753b8c2219d12815368c524857bef4fa14fb7ee267370a99545d9a"}
May 29 20:55:34.089734 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:34.089655    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6b77dbffcd-2wbhr" podStartSLOduration=2.22575025 podStartE2EDuration="4.089637632s" podCreationTimestamp="2026-05-29 20:55:30 +0000 UTC" firstStartedPulling="2026-05-29 20:55:31.415166317 +0000 UTC m=+494.479332435" lastFinishedPulling="2026-05-29 20:55:33.279053702 +0000 UTC m=+496.343219817" observedRunningTime="2026-05-29 20:55:34.08913471 +0000 UTC m=+497.153300846" watchObservedRunningTime="2026-05-29 20:55:34.089637632 +0000 UTC m=+497.153803771"
May 29 20:55:36.046742 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:36.046711    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cdf9b5b7c-z5k5s"
May 29 20:55:52.345341 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.345307    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"]
May 29 20:55:52.352260 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.352238    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.354627 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.354602    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 29 20:55:52.354757 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.354639    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 29 20:55:52.354757 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.354608    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 29 20:55:52.354960 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.354941    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nd4wb\""
May 29 20:55:52.359443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.359406    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"]
May 29 20:55:52.470450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470450 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/471729a2-3820-4e21-9382-37e134c8acea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470654 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470654 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470654 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470654 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470851 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h988z\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-kube-api-access-h988z\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470851 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/471729a2-3820-4e21-9382-37e134c8acea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.470851 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.470761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572135 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572105    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572323 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572195    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572323 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572238    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572323 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572271    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572323 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572307    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h988z\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-kube-api-access-h988z\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572548 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572390    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/471729a2-3820-4e21-9382-37e134c8acea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572548 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572483    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572657 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572553    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572657 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572555    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572657 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572621    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/471729a2-3820-4e21-9382-37e134c8acea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.572882 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.572816    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.573228 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.573069    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.573228 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.573168    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.573371 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.573285    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/471729a2-3820-4e21-9382-37e134c8acea-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.575387 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.575359    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/471729a2-3820-4e21-9382-37e134c8acea-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.575546 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.575529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/471729a2-3820-4e21-9382-37e134c8acea-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.588300 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.588274    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h988z\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-kube-api-access-h988z\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.602509 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.602456    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/471729a2-3820-4e21-9382-37e134c8acea-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl\" (UID: \"471729a2-3820-4e21-9382-37e134c8acea\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.666250 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.666218    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:52.808095 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:52.808061    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"]
May 29 20:55:52.811957 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:55:52.811929    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471729a2_3820_4e21_9382_37e134c8acea.slice/crio-1484c8021b2db587d068b253ca354ef191cc82e3afedf31ca636cc701bea9f9b WatchSource:0}: Error finding container 1484c8021b2db587d068b253ca354ef191cc82e3afedf31ca636cc701bea9f9b: Status 404 returned error can't find the container with id 1484c8021b2db587d068b253ca354ef191cc82e3afedf31ca636cc701bea9f9b
May 29 20:55:53.136337 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:53.136301    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl" event={"ID":"471729a2-3820-4e21-9382-37e134c8acea","Type":"ContainerStarted","Data":"1484c8021b2db587d068b253ca354ef191cc82e3afedf31ca636cc701bea9f9b"}
May 29 20:55:56.454482 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:56.454431    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:55:56.454775 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:56.454532    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:55:56.454775 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:56.454564    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:55:57.152079 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:57.152041    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl" event={"ID":"471729a2-3820-4e21-9382-37e134c8acea","Type":"ContainerStarted","Data":"1422634e50994237eba651dc177f7430068e54b42b3fbdbe86312c5c3f0d81b1"}
May 29 20:55:57.174454 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:57.174402    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl" podStartSLOduration=1.534283883 podStartE2EDuration="5.174383218s" podCreationTimestamp="2026-05-29 20:55:52 +0000 UTC" firstStartedPulling="2026-05-29 20:55:52.814047669 +0000 UTC m=+515.878213784" lastFinishedPulling="2026-05-29 20:55:56.454147003 +0000 UTC m=+519.518313119" observedRunningTime="2026-05-29 20:55:57.172746523 +0000 UTC m=+520.236912661" watchObservedRunningTime="2026-05-29 20:55:57.174383218 +0000 UTC m=+520.238549355"
May 29 20:55:57.667044 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:57.667012    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:57.671300 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:57.671278    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:58.155378 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:58.155347    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:55:58.156253 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:55:58.156234    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl"
May 29 20:56:13.950567 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.950477    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:13.954423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.954397    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:13.956660 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.956632    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-g6pjf\""
May 29 20:56:13.956809 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.956633    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 29 20:56:13.957364 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.957346    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 29 20:56:13.965979 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:13.965954    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:14.064326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.064282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwwb4\" (UniqueName: \"kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4\") pod \"kuadrant-operator-catalog-fwcbh\" (UID: \"8506b8f0-0d1a-4959-b52f-7d8a65da850e\") " pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:14.165607 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.165561    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zwwb4\" (UniqueName: \"kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4\") pod \"kuadrant-operator-catalog-fwcbh\" (UID: \"8506b8f0-0d1a-4959-b52f-7d8a65da850e\") " pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:14.173724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.173692    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwwb4\" (UniqueName: \"kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4\") pod \"kuadrant-operator-catalog-fwcbh\" (UID: \"8506b8f0-0d1a-4959-b52f-7d8a65da850e\") " pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:14.266177 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.266134    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:14.325016 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.321684    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:14.401942 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.401919    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:14.404306 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:56:14.404276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8506b8f0_0d1a_4959_b52f_7d8a65da850e.slice/crio-eee2f5c96d6486c85345d7db35a68c157f2ed8fca9f641130b0388b17859489d WatchSource:0}: Error finding container eee2f5c96d6486c85345d7db35a68c157f2ed8fca9f641130b0388b17859489d: Status 404 returned error can't find the container with id eee2f5c96d6486c85345d7db35a68c157f2ed8fca9f641130b0388b17859489d
May 29 20:56:14.529486 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.529411    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m7jcr"]
May 29 20:56:14.533937 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.533921    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:14.540011 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.539967    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m7jcr"]
May 29 20:56:14.670086 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.670048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8jd\" (UniqueName: \"kubernetes.io/projected/bcec8537-e8c0-4628-a5f1-e3e47c32ca05-kube-api-access-cb8jd\") pod \"kuadrant-operator-catalog-m7jcr\" (UID: \"bcec8537-e8c0-4628-a5f1-e3e47c32ca05\") " pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:14.770708 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.770669    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8jd\" (UniqueName: \"kubernetes.io/projected/bcec8537-e8c0-4628-a5f1-e3e47c32ca05-kube-api-access-cb8jd\") pod \"kuadrant-operator-catalog-m7jcr\" (UID: \"bcec8537-e8c0-4628-a5f1-e3e47c32ca05\") " pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:14.779944 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.779869    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8jd\" (UniqueName: \"kubernetes.io/projected/bcec8537-e8c0-4628-a5f1-e3e47c32ca05-kube-api-access-cb8jd\") pod \"kuadrant-operator-catalog-m7jcr\" (UID: \"bcec8537-e8c0-4628-a5f1-e3e47c32ca05\") " pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:14.845099 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.845059    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:14.977354 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:14.977325    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m7jcr"]
May 29 20:56:15.032292 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:56:15.032221    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcec8537_e8c0_4628_a5f1_e3e47c32ca05.slice/crio-66be55d87ceebfae52e8ee85701f15f2fa009d54971f598853b7676d0998f86b WatchSource:0}: Error finding container 66be55d87ceebfae52e8ee85701f15f2fa009d54971f598853b7676d0998f86b: Status 404 returned error can't find the container with id 66be55d87ceebfae52e8ee85701f15f2fa009d54971f598853b7676d0998f86b
May 29 20:56:15.212741 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:15.212696    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr" event={"ID":"bcec8537-e8c0-4628-a5f1-e3e47c32ca05","Type":"ContainerStarted","Data":"66be55d87ceebfae52e8ee85701f15f2fa009d54971f598853b7676d0998f86b"}
May 29 20:56:15.213762 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:15.213737    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" event={"ID":"8506b8f0-0d1a-4959-b52f-7d8a65da850e","Type":"ContainerStarted","Data":"eee2f5c96d6486c85345d7db35a68c157f2ed8fca9f641130b0388b17859489d"}
May 29 20:56:17.222732 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.222691    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr" event={"ID":"bcec8537-e8c0-4628-a5f1-e3e47c32ca05","Type":"ContainerStarted","Data":"161ee83f9432d0e6c3419e20562a2e9e2cf69e7f18989d6ffe64b5c067da6402"}
May 29 20:56:17.224160 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.224131    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" event={"ID":"8506b8f0-0d1a-4959-b52f-7d8a65da850e","Type":"ContainerStarted","Data":"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"}
May 29 20:56:17.224261 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.224195    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" podUID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" containerName="registry-server" containerID="cri-o://5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f" gracePeriod=2
May 29 20:56:17.239376 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.239313    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr" podStartSLOduration=1.56450735 podStartE2EDuration="3.239294096s" podCreationTimestamp="2026-05-29 20:56:14 +0000 UTC" firstStartedPulling="2026-05-29 20:56:15.03362518 +0000 UTC m=+538.097791295" lastFinishedPulling="2026-05-29 20:56:16.708411912 +0000 UTC m=+539.772578041" observedRunningTime="2026-05-29 20:56:17.237513205 +0000 UTC m=+540.301679342" watchObservedRunningTime="2026-05-29 20:56:17.239294096 +0000 UTC m=+540.303460234"
May 29 20:56:17.255065 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.255013    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" podStartSLOduration=1.955694874 podStartE2EDuration="4.254971888s" podCreationTimestamp="2026-05-29 20:56:13 +0000 UTC" firstStartedPulling="2026-05-29 20:56:14.405759058 +0000 UTC m=+537.469925188" lastFinishedPulling="2026-05-29 20:56:16.705036084 +0000 UTC m=+539.769202202" observedRunningTime="2026-05-29 20:56:17.252859998 +0000 UTC m=+540.317026137" watchObservedRunningTime="2026-05-29 20:56:17.254971888 +0000 UTC m=+540.319138062"
May 29 20:56:17.468574 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.468545    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:17.598212 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.598127    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwwb4\" (UniqueName: \"kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4\") pod \"8506b8f0-0d1a-4959-b52f-7d8a65da850e\" (UID: \"8506b8f0-0d1a-4959-b52f-7d8a65da850e\") "
May 29 20:56:17.600542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.600514    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4" (OuterVolumeSpecName: "kube-api-access-zwwb4") pod "8506b8f0-0d1a-4959-b52f-7d8a65da850e" (UID: "8506b8f0-0d1a-4959-b52f-7d8a65da850e"). InnerVolumeSpecName "kube-api-access-zwwb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:56:17.698818 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:17.698776    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwwb4\" (UniqueName: \"kubernetes.io/projected/8506b8f0-0d1a-4959-b52f-7d8a65da850e-kube-api-access-zwwb4\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:56:18.228850 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.228813    2563 generic.go:358] "Generic (PLEG): container finished" podID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" containerID="5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f" exitCode=0
May 29 20:56:18.229280 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.228873    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh"
May 29 20:56:18.229280 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.228897    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" event={"ID":"8506b8f0-0d1a-4959-b52f-7d8a65da850e","Type":"ContainerDied","Data":"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"}
May 29 20:56:18.229280 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.228935    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fwcbh" event={"ID":"8506b8f0-0d1a-4959-b52f-7d8a65da850e","Type":"ContainerDied","Data":"eee2f5c96d6486c85345d7db35a68c157f2ed8fca9f641130b0388b17859489d"}
May 29 20:56:18.229280 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.228954    2563 scope.go:117] "RemoveContainer" containerID="5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"
May 29 20:56:18.240535 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.240510    2563 scope.go:117] "RemoveContainer" containerID="5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"
May 29 20:56:18.240832 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:56:18.240809    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f\": container with ID starting with 5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f not found: ID does not exist" containerID="5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"
May 29 20:56:18.240911 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.240845    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f"} err="failed to get container status \"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f\": rpc error: code = NotFound desc = could not find container \"5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f\": container with ID starting with 5f6e38ec59f381fac5e60fce27c5d87cde29f1eac75ae2bb4cafd3dfbae1104f not found: ID does not exist"
May 29 20:56:18.255888 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.255856    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:18.259268 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:18.259237    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fwcbh"]
May 29 20:56:19.501959 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:19.501925    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" path="/var/lib/kubelet/pods/8506b8f0-0d1a-4959-b52f-7d8a65da850e/volumes"
May 29 20:56:24.845383 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:24.845349    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:24.845783 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:24.845402    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:24.868301 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:24.868275    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:25.275343 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:25.275315    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-m7jcr"
May 29 20:56:47.361469 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.361434    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"]
May 29 20:56:47.361927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.361806    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" containerName="registry-server"
May 29 20:56:47.361927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.361818    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" containerName="registry-server"
May 29 20:56:47.361927 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.361890    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="8506b8f0-0d1a-4959-b52f-7d8a65da850e" containerName="registry-server"
May 29 20:56:47.368652 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.368630    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.371009 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.370973    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6wpxd\""
May 29 20:56:47.374176 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.374154    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"]
May 29 20:56:47.456642 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.456596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.456845 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.456740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlfpk\" (UniqueName: \"kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.557913 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.557880    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wlfpk\" (UniqueName: \"kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.558116 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.557942    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.558382 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.558362    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.566434 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.566407    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlfpk\" (UniqueName: \"kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.680548 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.680459    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:47.814253 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:47.814217    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"]
May 29 20:56:47.817094 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:56:47.817066    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd41ce0_a3c7_4a57_81e1_9a7e15032b17.slice/crio-15816024b08b85ae6f9366f016108992a31b750c1efd1d584fe69a5603a35b90 WatchSource:0}: Error finding container 15816024b08b85ae6f9366f016108992a31b750c1efd1d584fe69a5603a35b90: Status 404 returned error can't find the container with id 15816024b08b85ae6f9366f016108992a31b750c1efd1d584fe69a5603a35b90
May 29 20:56:48.328957 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:48.328899    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" event={"ID":"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17","Type":"ContainerStarted","Data":"15816024b08b85ae6f9366f016108992a31b750c1efd1d584fe69a5603a35b90"}
May 29 20:56:49.369108 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.369073    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"]
May 29 20:56:49.372290 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.372266    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:49.374628 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.374595    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 29 20:56:49.374766 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.374655    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6xfwr\""
May 29 20:56:49.384268 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.384243    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"]
May 29 20:56:49.474439 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.474400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk5kh\" (UniqueName: \"kubernetes.io/projected/773d544e-b8f5-4667-a1a2-ed3cd963cfe1-kube-api-access-fk5kh\") pod \"dns-operator-controller-manager-648d5c98bc-5bj4d\" (UID: \"773d544e-b8f5-4667-a1a2-ed3cd963cfe1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:49.574919 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.574866    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fk5kh\" (UniqueName: \"kubernetes.io/projected/773d544e-b8f5-4667-a1a2-ed3cd963cfe1-kube-api-access-fk5kh\") pod \"dns-operator-controller-manager-648d5c98bc-5bj4d\" (UID: \"773d544e-b8f5-4667-a1a2-ed3cd963cfe1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:49.583697 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.583651    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk5kh\" (UniqueName: \"kubernetes.io/projected/773d544e-b8f5-4667-a1a2-ed3cd963cfe1-kube-api-access-fk5kh\") pod \"dns-operator-controller-manager-648d5c98bc-5bj4d\" (UID: \"773d544e-b8f5-4667-a1a2-ed3cd963cfe1\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:49.687887 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.687790    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:49.830637 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:49.830610    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"]
May 29 20:56:49.832922 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:56:49.832890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773d544e_b8f5_4667_a1a2_ed3cd963cfe1.slice/crio-d9b43df06ccca4338564c8830e8ee9d6948fb3bd63e42883ac98401e591d8eae WatchSource:0}: Error finding container d9b43df06ccca4338564c8830e8ee9d6948fb3bd63e42883ac98401e591d8eae: Status 404 returned error can't find the container with id d9b43df06ccca4338564c8830e8ee9d6948fb3bd63e42883ac98401e591d8eae
May 29 20:56:50.338708 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:50.338670    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d" event={"ID":"773d544e-b8f5-4667-a1a2-ed3cd963cfe1","Type":"ContainerStarted","Data":"d9b43df06ccca4338564c8830e8ee9d6948fb3bd63e42883ac98401e591d8eae"}
May 29 20:56:52.049040 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.048985    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"]
May 29 20:56:52.052089 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.052058    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:52.055307 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.054264    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ntz8p\""
May 29 20:56:52.063105 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.063076    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"]
May 29 20:56:52.100784 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.100705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp929\" (UniqueName: \"kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929\") pod \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" (UID: \"13855fb6-0b70-4d4e-9fb8-f7034163350b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:52.202124 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.202085    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hp929\" (UniqueName: \"kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929\") pod \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" (UID: \"13855fb6-0b70-4d4e-9fb8-f7034163350b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:52.210366 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.210337    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp929\" (UniqueName: \"kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929\") pod \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" (UID: \"13855fb6-0b70-4d4e-9fb8-f7034163350b\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:52.368033 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:52.367927    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:53.272005 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.271965    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"]
May 29 20:56:53.274451 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:56:53.274419    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13855fb6_0b70_4d4e_9fb8_f7034163350b.slice/crio-d2e953b0827f0d42bca026e1441a1468169d60d118da07c5762f3383c37fc5b8 WatchSource:0}: Error finding container d2e953b0827f0d42bca026e1441a1468169d60d118da07c5762f3383c37fc5b8: Status 404 returned error can't find the container with id d2e953b0827f0d42bca026e1441a1468169d60d118da07c5762f3383c37fc5b8
May 29 20:56:53.352598 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.352562    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" event={"ID":"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17","Type":"ContainerStarted","Data":"953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec"}
May 29 20:56:53.352824 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.352803    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:56:53.353801 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.353776    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" event={"ID":"13855fb6-0b70-4d4e-9fb8-f7034163350b","Type":"ContainerStarted","Data":"d2e953b0827f0d42bca026e1441a1468169d60d118da07c5762f3383c37fc5b8"}
May 29 20:56:53.354955 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.354937    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d" event={"ID":"773d544e-b8f5-4667-a1a2-ed3cd963cfe1","Type":"ContainerStarted","Data":"ef78e906a4d7c1e29449d5122f2ff41a80298071824b8a5dcdf92fe569f1e558"}
May 29 20:56:53.355110 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.355095    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:56:53.375496 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.375444    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" podStartSLOduration=1.058783256 podStartE2EDuration="6.375427931s" podCreationTimestamp="2026-05-29 20:56:47 +0000 UTC" firstStartedPulling="2026-05-29 20:56:47.81926546 +0000 UTC m=+570.883431575" lastFinishedPulling="2026-05-29 20:56:53.135910133 +0000 UTC m=+576.200076250" observedRunningTime="2026-05-29 20:56:53.373784374 +0000 UTC m=+576.437950512" watchObservedRunningTime="2026-05-29 20:56:53.375427931 +0000 UTC m=+576.439594070"
May 29 20:56:53.392361 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:53.392279    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d" podStartSLOduration=1.096357604 podStartE2EDuration="4.392267794s" podCreationTimestamp="2026-05-29 20:56:49 +0000 UTC" firstStartedPulling="2026-05-29 20:56:49.834774554 +0000 UTC m=+572.898940673" lastFinishedPulling="2026-05-29 20:56:53.130684747 +0000 UTC m=+576.194850863" observedRunningTime="2026-05-29 20:56:53.391180609 +0000 UTC m=+576.455346747" watchObservedRunningTime="2026-05-29 20:56:53.392267794 +0000 UTC m=+576.456433928"
May 29 20:56:55.363640 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:55.363605    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" event={"ID":"13855fb6-0b70-4d4e-9fb8-f7034163350b","Type":"ContainerStarted","Data":"63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b"}
May 29 20:56:55.364087 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:55.363683    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:56:55.389653 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:56:55.389596    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" podStartSLOduration=1.429102298 podStartE2EDuration="3.389581554s" podCreationTimestamp="2026-05-29 20:56:52 +0000 UTC" firstStartedPulling="2026-05-29 20:56:53.276410628 +0000 UTC m=+576.340576746" lastFinishedPulling="2026-05-29 20:56:55.236889888 +0000 UTC m=+578.301056002" observedRunningTime="2026-05-29 20:56:55.387194964 +0000 UTC m=+578.451361104" watchObservedRunningTime="2026-05-29 20:56:55.389581554 +0000 UTC m=+578.453747690"
May 29 20:57:04.361050 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:04.361017    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:57:04.361562 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:04.361070    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-5bj4d"
May 29 20:57:06.021474 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.021435    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"]
May 29 20:57:06.024174 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.024144    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.034966 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.034939    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"]
May 29 20:57:06.095491 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.095457    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"]
May 29 20:57:06.095751 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:06.095729    2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-7lh5h], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2"
May 29 20:57:06.099850 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.099823    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"]
May 29 20:57:06.107004 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.106967    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"]
May 29 20:57:06.107210 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.107184    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" containerName="manager" containerID="cri-o://953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec" gracePeriod=2
May 29 20:57:06.120748 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.119618    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"]
May 29 20:57:06.123257 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.123237    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"]
May 29 20:57:06.123539 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.123499    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.128555 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.128525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lh5h\" (UniqueName: \"kubernetes.io/projected/8509b6e0-e8c5-4541-93bf-2364ffea02d2-kube-api-access-7lh5h\") pod \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.128667 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.128635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.135055 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.135033    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"]
May 29 20:57:06.135309 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.135290    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" containerName="manager" containerID="cri-o://63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b" gracePeriod=2
May 29 20:57:06.136984 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.136958    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:57:06.137103 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.137036    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:06.137583 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.137564    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" containerName="manager"
May 29 20:57:06.137583 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.137584    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" containerName="manager"
May 29 20:57:06.137716 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.137673    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" containerName="manager"
May 29 20:57:06.140579 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.140512    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"]
May 29 20:57:06.140685 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.140649    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.144207 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.144174    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"]
May 29 20:57:06.147267 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.147242    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:06.149319 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.149046    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.155052 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.155027    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"]
May 29 20:57:06.155471 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.155455    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" containerName="manager"
May 29 20:57:06.155531 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.155474    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" containerName="manager"
May 29 20:57:06.155571 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.155549    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" containerName="manager"
May 29 20:57:06.163392 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.162836    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:06.167632 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.166800    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"]
May 29 20:57:06.190273 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.190229    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.229585 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229556    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.229718 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78d4c292-41ef-4383-9d60-5775cdcda22b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.229718 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.229718 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54p54\" (UniqueName: \"kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.229718 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229689    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lh5h\" (UniqueName: \"kubernetes.io/projected/8509b6e0-e8c5-4541-93bf-2364ffea02d2-kube-api-access-7lh5h\") pod \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.229898 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj9gd\" (UniqueName: \"kubernetes.io/projected/78d4c292-41ef-4383-9d60-5775cdcda22b-kube-api-access-nj9gd\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.229898 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.229871    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.234305 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:06.234290    2563 projected.go:194] Error preparing data for projected volume kube-api-access-7lh5h for pod kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9: failed to fetch token: pod "kuadrant-operator-controller-manager-55c7f4c975-6q2s9" not found
May 29 20:57:06.234365 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:06.234355    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8509b6e0-e8c5-4541-93bf-2364ffea02d2-kube-api-access-7lh5h podName:8509b6e0-e8c5-4541-93bf-2364ffea02d2 nodeName:}" failed. No retries permitted until 2026-05-29 20:57:06.734333627 +0000 UTC m=+589.798499741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7lh5h" (UniqueName: "kubernetes.io/projected/8509b6e0-e8c5-4541-93bf-2364ffea02d2-kube-api-access-7lh5h") pod "kuadrant-operator-controller-manager-55c7f4c975-6q2s9" (UID: "8509b6e0-e8c5-4541-93bf-2364ffea02d2") : failed to fetch token: pod "kuadrant-operator-controller-manager-55c7f4c975-6q2s9" not found
May 29 20:57:06.330752 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.330722    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nj9gd\" (UniqueName: \"kubernetes.io/projected/78d4c292-41ef-4383-9d60-5775cdcda22b-kube-api-access-nj9gd\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.330901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.330763    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78d4c292-41ef-4383-9d60-5775cdcda22b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.330901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.330798    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.330901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.330844    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-54p54\" (UniqueName: \"kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.330901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.330884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r64vk\" (UniqueName: \"kubernetes.io/projected/7b7800dd-b5b8-451f-89b0-e1ff9599aa0e-kube-api-access-r64vk\") pod \"limitador-operator-controller-manager-85c4996f8c-xkrgg\" (UID: \"7b7800dd-b5b8-451f-89b0-e1ff9599aa0e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:06.331332 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.331308    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/78d4c292-41ef-4383-9d60-5775cdcda22b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.331449 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.331315    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.342311 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.342286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj9gd\" (UniqueName: \"kubernetes.io/projected/78d4c292-41ef-4383-9d60-5775cdcda22b-kube-api-access-nj9gd\") pod \"kuadrant-operator-controller-manager-55c7f4c975-z2vzd\" (UID: \"78d4c292-41ef-4383-9d60-5775cdcda22b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.342585 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.342563    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54p54\" (UniqueName: \"kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-skszr\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.387369 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.387343    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:57:06.389866 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.389838    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.390686 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.390671    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:57:06.391739 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.391716    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.393346 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.393326    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.395053 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.395034    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.402851 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.402829    2563 generic.go:358] "Generic (PLEG): container finished" podID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" containerID="953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec" exitCode=0
May 29 20:57:06.402973 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.402869    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54"
May 29 20:57:06.402973 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.402916    2563 scope.go:117] "RemoveContainer" containerID="953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec"
May 29 20:57:06.404053 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.404033    2563 generic.go:358] "Generic (PLEG): container finished" podID="13855fb6-0b70-4d4e-9fb8-f7034163350b" containerID="63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b" exitCode=0
May 29 20:57:06.404157 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.404078    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp"
May 29 20:57:06.404206 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.404198    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.405082 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.405057    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.407089 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.407056    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.408436 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.408416    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:06.408897 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.408876    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.410430 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.410411    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.411968 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.411951    2563 scope.go:117] "RemoveContainer" containerID="953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec"
May 29 20:57:06.412143 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.412123    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.412227 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:06.412205    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec\": container with ID starting with 953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec not found: ID does not exist" containerID="953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec"
May 29 20:57:06.412275 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.412236    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec"} err="failed to get container status \"953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec\": rpc error: code = NotFound desc = could not find container \"953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec\": container with ID starting with 953a891da86411379515c56dda4c8e40af36d979bcdef81fcc246527a76828ec not found: ID does not exist"
May 29 20:57:06.412275 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.412258    2563 scope.go:117] "RemoveContainer" containerID="63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b"
May 29 20:57:06.413826 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.413802    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.415461 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.415441    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.417138 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.417113    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.419751 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.419724    2563 scope.go:117] "RemoveContainer" containerID="63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b"
May 29 20:57:06.420030 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:06.419984    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b\": container with ID starting with 63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b not found: ID does not exist" containerID="63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b"
May 29 20:57:06.420093 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.420036    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b"} err="failed to get container status \"63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b\": rpc error: code = NotFound desc = could not find container \"63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b\": container with ID starting with 63acd0942a309294870d518ba13c862cb52a6fb45b1ff89c3545bb9927d8aa0b not found: ID does not exist"
May 29 20:57:06.431467 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.431439    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r64vk\" (UniqueName: \"kubernetes.io/projected/7b7800dd-b5b8-451f-89b0-e1ff9599aa0e-kube-api-access-r64vk\") pod \"limitador-operator-controller-manager-85c4996f8c-xkrgg\" (UID: \"7b7800dd-b5b8-451f-89b0-e1ff9599aa0e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:06.440453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.440430    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r64vk\" (UniqueName: \"kubernetes.io/projected/7b7800dd-b5b8-451f-89b0-e1ff9599aa0e-kube-api-access-r64vk\") pod \"limitador-operator-controller-manager-85c4996f8c-xkrgg\" (UID: \"7b7800dd-b5b8-451f-89b0-e1ff9599aa0e\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:06.532301 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532268    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume\") pod \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") "
May 29 20:57:06.532453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532317    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp929\" (UniqueName: \"kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929\") pod \"13855fb6-0b70-4d4e-9fb8-f7034163350b\" (UID: \"13855fb6-0b70-4d4e-9fb8-f7034163350b\") "
May 29 20:57:06.532453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532348    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume\") pod \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\" (UID: \"8509b6e0-e8c5-4541-93bf-2364ffea02d2\") "
May 29 20:57:06.532453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532420    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlfpk\" (UniqueName: \"kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk\") pod \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\" (UID: \"6dd41ce0-a3c7-4a57-81e1-9a7e15032b17\") "
May 29 20:57:06.532658 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532630    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8509b6e0-e8c5-4541-93bf-2364ffea02d2" (UID: "8509b6e0-e8c5-4541-93bf-2364ffea02d2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:57:06.532718 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532700    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lh5h\" (UniqueName: \"kubernetes.io/projected/8509b6e0-e8c5-4541-93bf-2364ffea02d2-kube-api-access-7lh5h\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:06.532757 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.532717    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" (UID: "6dd41ce0-a3c7-4a57-81e1-9a7e15032b17"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:57:06.534615 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.534588    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk" (OuterVolumeSpecName: "kube-api-access-wlfpk") pod "6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" (UID: "6dd41ce0-a3c7-4a57-81e1-9a7e15032b17"). InnerVolumeSpecName "kube-api-access-wlfpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:57:06.534721 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.534676    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929" (OuterVolumeSpecName: "kube-api-access-hp929") pod "13855fb6-0b70-4d4e-9fb8-f7034163350b" (UID: "13855fb6-0b70-4d4e-9fb8-f7034163350b"). InnerVolumeSpecName "kube-api-access-hp929". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:57:06.557871 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.557819    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:06.566067 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.566037    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:06.571777 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.571756    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:06.634634 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.634595    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-extensions-socket-volume\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:06.634769 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.634637    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp929\" (UniqueName: \"kubernetes.io/projected/13855fb6-0b70-4d4e-9fb8-f7034163350b-kube-api-access-hp929\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:06.634769 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.634659    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8509b6e0-e8c5-4541-93bf-2364ffea02d2-extensions-socket-volume\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:06.634769 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.634674    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlfpk\" (UniqueName: \"kubernetes.io/projected/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17-kube-api-access-wlfpk\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:06.711644 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.711608    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"]
May 29 20:57:06.716259 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:06.716228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d4c292_41ef_4383_9d60_5775cdcda22b.slice/crio-6b643cbb3a55810b218244f38acbdc2fcbac908f3561c35f203bec6960e4bc4a WatchSource:0}: Error finding container 6b643cbb3a55810b218244f38acbdc2fcbac908f3561c35f203bec6960e4bc4a: Status 404 returned error can't find the container with id 6b643cbb3a55810b218244f38acbdc2fcbac908f3561c35f203bec6960e4bc4a
May 29 20:57:06.716650 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.716537    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.718439 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.718404    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.720867 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.720101    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.722109 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.722068    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.723827 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.723802    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.725555 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.725527    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:06.946837 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.946774    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"]
May 29 20:57:06.949021 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:06.948964    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:06.949188 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:06.949166    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7800dd_b5b8_451f_89b0_e1ff9599aa0e.slice/crio-d2ebed605e51cd90f380ce54bb90e6a08def7277f7e5609b6621ea39bdb3fe5d WatchSource:0}: Error finding container d2ebed605e51cd90f380ce54bb90e6a08def7277f7e5609b6621ea39bdb3fe5d: Status 404 returned error can't find the container with id d2ebed605e51cd90f380ce54bb90e6a08def7277f7e5609b6621ea39bdb3fe5d
May 29 20:57:06.951217 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:06.951193    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ff782b0_a3c7_4f1a_a36d_bbcd354fcfd9.slice/crio-fb139d869c83cfd40204ef2a01d030d86d44e80dcab8b64d9a026790b7a3984b WatchSource:0}: Error finding container fb139d869c83cfd40204ef2a01d030d86d44e80dcab8b64d9a026790b7a3984b: Status 404 returned error can't find the container with id fb139d869c83cfd40204ef2a01d030d86d44e80dcab8b64d9a026790b7a3984b
May 29 20:57:07.411202 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.411165    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" event={"ID":"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9","Type":"ContainerStarted","Data":"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"}
May 29 20:57:07.412432 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.412405    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" event={"ID":"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9","Type":"ContainerStarted","Data":"fb139d869c83cfd40204ef2a01d030d86d44e80dcab8b64d9a026790b7a3984b"}
May 29 20:57:07.413036 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.413018    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:07.413616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.413593    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg" event={"ID":"7b7800dd-b5b8-451f-89b0-e1ff9599aa0e","Type":"ContainerStarted","Data":"a14273e6428da17bc223a251ac430249dc710a8b5f972bc34df97a197b8e8461"}
May 29 20:57:07.413616 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.413621    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg" event={"ID":"7b7800dd-b5b8-451f-89b0-e1ff9599aa0e","Type":"ContainerStarted","Data":"d2ebed605e51cd90f380ce54bb90e6a08def7277f7e5609b6621ea39bdb3fe5d"}
May 29 20:57:07.414165 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.414145    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:07.414323 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.414295    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.415542 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.415521    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9"
May 29 20:57:07.415674 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.415517    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd" event={"ID":"78d4c292-41ef-4383-9d60-5775cdcda22b","Type":"ContainerStarted","Data":"7f90836efe962c60c2fe400240ea49e4dffee8d24ce1e19202090c6a9ae02958"}
May 29 20:57:07.415731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.415685    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd" event={"ID":"78d4c292-41ef-4383-9d60-5775cdcda22b","Type":"ContainerStarted","Data":"6b643cbb3a55810b218244f38acbdc2fcbac908f3561c35f203bec6960e4bc4a"}
May 29 20:57:07.415731 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.415705    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:07.415971 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.415951    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.417733 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.417710    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.437023 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.436964    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" podStartSLOduration=1.436953734 podStartE2EDuration="1.436953734s" podCreationTimestamp="2026-05-29 20:57:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:57:07.436013123 +0000 UTC m=+590.500179254" watchObservedRunningTime="2026-05-29 20:57:07.436953734 +0000 UTC m=+590.501119865"
May 29 20:57:07.438091 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.438066    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.439671 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.439647    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.441304 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.441268    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.456112 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.456064    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg" podStartSLOduration=1.456050602 podStartE2EDuration="1.456050602s" podCreationTimestamp="2026-05-29 20:57:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:57:07.45506013 +0000 UTC m=+590.519226268" watchObservedRunningTime="2026-05-29 20:57:07.456050602 +0000 UTC m=+590.520216740"
May 29 20:57:07.471328 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.471292    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd" podStartSLOduration=1.471279149 podStartE2EDuration="1.471279149s" podCreationTimestamp="2026-05-29 20:57:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:57:07.470980114 +0000 UTC m=+590.535146251" watchObservedRunningTime="2026-05-29 20:57:07.471279149 +0000 UTC m=+590.535445286"
May 29 20:57:07.499973 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.499946    2563 status_manager.go:895] "Failed to get status for pod" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mwc54" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mwc54\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.500868 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.500849    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" path="/var/lib/kubelet/pods/13855fb6-0b70-4d4e-9fb8-f7034163350b/volumes"
May 29 20:57:07.501211 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.501198    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dd41ce0-a3c7-4a57-81e1-9a7e15032b17" path="/var/lib/kubelet/pods/6dd41ce0-a3c7-4a57-81e1-9a7e15032b17/volumes"
May 29 20:57:07.501474 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.501464    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" path="/var/lib/kubelet/pods/8509b6e0-e8c5-4541-93bf-2364ffea02d2/volumes"
May 29 20:57:07.501844 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.501824    2563 status_manager.go:895] "Failed to get status for pod" podUID="13855fb6-0b70-4d4e-9fb8-f7034163350b" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-wk4dp" err="pods \"limitador-operator-controller-manager-85c4996f8c-wk4dp\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:07.504453 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:07.504434    2563 status_manager.go:895] "Failed to get status for pod" podUID="8509b6e0-e8c5-4541-93bf-2364ffea02d2" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-6q2s9" err="pods \"kuadrant-operator-controller-manager-55c7f4c975-6q2s9\" is forbidden: User \"system:node:ip-10-0-133-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-224.ec2.internal' and this object"
May 29 20:57:17.398152 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:17.398125    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:57:17.398593 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:17.398429    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 20:57:17.405373 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:17.405352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:57:17.405373 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:17.405367    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 20:57:18.421016 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.420963    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-z2vzd"
May 29 20:57:18.465617 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.465585    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:18.465855 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.465823    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" podUID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" containerName="manager" containerID="cri-o://e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4" gracePeriod=10
May 29 20:57:18.467864 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.467840    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:18.719700 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.719677    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:18.847623 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.847572    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54p54\" (UniqueName: \"kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54\") pod \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") "
May 29 20:57:18.847802 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.847663    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume\") pod \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\" (UID: \"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9\") "
May 29 20:57:18.848071 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.848040    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" (UID: "0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 29 20:57:18.849802 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.849779    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54" (OuterVolumeSpecName: "kube-api-access-54p54") pod "0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" (UID: "0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9"). InnerVolumeSpecName "kube-api-access-54p54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:57:18.949138 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.949105    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54p54\" (UniqueName: \"kubernetes.io/projected/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-kube-api-access-54p54\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:18.949138 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:18.949131    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9-extensions-socket-volume\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:19.430351 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.430317    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-xkrgg"
May 29 20:57:19.464769 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.464739    2563 generic.go:358] "Generic (PLEG): container finished" podID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" containerID="e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4" exitCode=0
May 29 20:57:19.464941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.464858    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" event={"ID":"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9","Type":"ContainerDied","Data":"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"}
May 29 20:57:19.464941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.464891    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"
May 29 20:57:19.464941 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.464906    2563 scope.go:117] "RemoveContainer" containerID="e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"
May 29 20:57:19.465249 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.464894    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr" event={"ID":"0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9","Type":"ContainerDied","Data":"fb139d869c83cfd40204ef2a01d030d86d44e80dcab8b64d9a026790b7a3984b"}
May 29 20:57:19.474290 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.474067    2563 scope.go:117] "RemoveContainer" containerID="e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"
May 29 20:57:19.474759 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:57:19.474738    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4\": container with ID starting with e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4 not found: ID does not exist" containerID="e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"
May 29 20:57:19.474820 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.474767    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4"} err="failed to get container status \"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4\": rpc error: code = NotFound desc = could not find container \"e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4\": container with ID starting with e343d5a175856f4eeea50f2d5b8136164b1a3b59861514e2df64ef65d9c140f4 not found: ID does not exist"
May 29 20:57:19.493744 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.493715    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:19.496330 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.496305    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-skszr"]
May 29 20:57:19.502041 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:19.501979    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" path="/var/lib/kubelet/pods/0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9/volumes"
May 29 20:57:34.753406 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.753326    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"]
May 29 20:57:34.753915 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.753846    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" containerName="manager"
May 29 20:57:34.753915 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.753887    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" containerName="manager"
May 29 20:57:34.754100 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.753966    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="0ff782b0-a3c7-4f1a-a36d-bbcd354fcfd9" containerName="manager"
May 29 20:57:34.757532 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.757514    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.760110 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.760082    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gkvmf\""
May 29 20:57:34.774724 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.774695    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"]
May 29 20:57:34.794759 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.794928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.794928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd6b2\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-kube-api-access-fd6b2\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.794928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.794928 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.795186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.795186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.795186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.794972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.795186 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.795031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.895819 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.895782    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.895819 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.895823    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.895851    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.895881    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.895967    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896036    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896096 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896073    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fd6b2\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-kube-api-access-fd6b2\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896131    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896290    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896342 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896315    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896488 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896465    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896533 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896516    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896606 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896589    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.896757 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.896739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.898824 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.898795    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.899038 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.899019    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.903976 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.903950    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:34.904158 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:34.904139    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd6b2\" (UniqueName: \"kubernetes.io/projected/23bfeffb-f0c8-4c55-8f4b-60fc0312ee73-kube-api-access-fd6b2\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-446s5\" (UID: \"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:35.069037 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.068930    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:35.199296 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.199270    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"]
May 29 20:57:35.201342 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:35.201314    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23bfeffb_f0c8_4c55_8f4b_60fc0312ee73.slice/crio-6c2b0d7cbb8a4bebec90f67c90c7bb92402af1bec120a4c242d3ac8ca64c3f3b WatchSource:0}: Error finding container 6c2b0d7cbb8a4bebec90f67c90c7bb92402af1bec120a4c242d3ac8ca64c3f3b: Status 404 returned error can't find the container with id 6c2b0d7cbb8a4bebec90f67c90c7bb92402af1bec120a4c242d3ac8ca64c3f3b
May 29 20:57:35.203277 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.203246    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:57:35.203356 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.203314    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:57:35.203356 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.203343    2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
May 29 20:57:35.523934 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.523890    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5" event={"ID":"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73","Type":"ContainerStarted","Data":"b0f94d3cc858df7c4c30979b422ba3f0db31a6ee3b322b52b479665451a27080"}
May 29 20:57:35.523934 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.523933    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5" event={"ID":"23bfeffb-f0c8-4c55-8f4b-60fc0312ee73","Type":"ContainerStarted","Data":"6c2b0d7cbb8a4bebec90f67c90c7bb92402af1bec120a4c242d3ac8ca64c3f3b"}
May 29 20:57:35.546850 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:35.546801    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5" podStartSLOduration=1.546786189 podStartE2EDuration="1.546786189s" podCreationTimestamp="2026-05-29 20:57:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 20:57:35.545215314 +0000 UTC m=+618.609381451" watchObservedRunningTime="2026-05-29 20:57:35.546786189 +0000 UTC m=+618.610952326"
May 29 20:57:36.069798 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:36.069759    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:36.071439 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:36.071414    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-7d6f447cf5-446s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
May 29 20:57:36.071556 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:36.071476    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5" podUID="23bfeffb-f0c8-4c55-8f4b-60fc0312ee73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
May 29 20:57:37.074282 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:37.074246    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:37.530830 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:37.530797    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:37.531906 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:37.531890    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-446s5"
May 29 20:57:38.567279 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.567244    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:38.569898 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.569876    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.572087 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.572066    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 29 20:57:38.572225 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.572200    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s9vfb\""
May 29 20:57:38.578476 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.578454    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:38.629985 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.629947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjxt\" (UniqueName: \"kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.630245 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.630219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.665386 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.665351    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:38.731116 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.731075    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.731273 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.731145    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjxt\" (UniqueName: \"kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.731776 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.731757    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.740097 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.740074    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjxt\" (UniqueName: \"kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt\") pod \"limitador-limitador-7d549b5b-q76dp\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") " pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:38.881031 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:38.880926    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:39.023555 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:39.023373    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:39.026254 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:39.026226    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49201d5_bbcd_4f13_bb02_37e2dca44604.slice/crio-6a5a4d5bf48c0595759338339d1a611efcf26bdaf654af00462d82a3756d8c8d WatchSource:0}: Error finding container 6a5a4d5bf48c0595759338339d1a611efcf26bdaf654af00462d82a3756d8c8d: Status 404 returned error can't find the container with id 6a5a4d5bf48c0595759338339d1a611efcf26bdaf654af00462d82a3756d8c8d
May 29 20:57:39.539131 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:39.539092    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" event={"ID":"a49201d5-bbcd-4f13-bb02-37e2dca44604","Type":"ContainerStarted","Data":"6a5a4d5bf48c0595759338339d1a611efcf26bdaf654af00462d82a3756d8c8d"}
May 29 20:57:42.550931 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:42.550896    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" event={"ID":"a49201d5-bbcd-4f13-bb02-37e2dca44604","Type":"ContainerStarted","Data":"56b41639d550ee7b0df5e5de07b87e36e6407bc64d5b569fdc728ff591198b62"}
May 29 20:57:42.551344 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:42.550953    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:42.567725 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:42.567678    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" podStartSLOduration=2.002711654 podStartE2EDuration="4.5676629s" podCreationTimestamp="2026-05-29 20:57:38 +0000 UTC" firstStartedPulling="2026-05-29 20:57:39.028135898 +0000 UTC m=+622.092302016" lastFinishedPulling="2026-05-29 20:57:41.593087136 +0000 UTC m=+624.657253262" observedRunningTime="2026-05-29 20:57:42.56531001 +0000 UTC m=+625.629476147" watchObservedRunningTime="2026-05-29 20:57:42.5676629 +0000 UTC m=+625.631829037"
May 29 20:57:53.101849 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.101781    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:53.102521 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.102230    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" podUID="a49201d5-bbcd-4f13-bb02-37e2dca44604" containerName="limitador" containerID="cri-o://56b41639d550ee7b0df5e5de07b87e36e6407bc64d5b569fdc728ff591198b62" gracePeriod=30
May 29 20:57:53.106538 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.106513    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:53.595741 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.595709    2563 generic.go:358] "Generic (PLEG): container finished" podID="a49201d5-bbcd-4f13-bb02-37e2dca44604" containerID="56b41639d550ee7b0df5e5de07b87e36e6407bc64d5b569fdc728ff591198b62" exitCode=0
May 29 20:57:53.595915 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.595807    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" event={"ID":"a49201d5-bbcd-4f13-bb02-37e2dca44604","Type":"ContainerDied","Data":"56b41639d550ee7b0df5e5de07b87e36e6407bc64d5b569fdc728ff591198b62"}
May 29 20:57:53.650727 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.650699    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:53.766681 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.766654    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcjxt\" (UniqueName: \"kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt\") pod \"a49201d5-bbcd-4f13-bb02-37e2dca44604\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") "
May 29 20:57:53.766858 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.766795    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file\") pod \"a49201d5-bbcd-4f13-bb02-37e2dca44604\" (UID: \"a49201d5-bbcd-4f13-bb02-37e2dca44604\") "
May 29 20:57:53.767165 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.767138    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file" (OuterVolumeSpecName: "config-file") pod "a49201d5-bbcd-4f13-bb02-37e2dca44604" (UID: "a49201d5-bbcd-4f13-bb02-37e2dca44604"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 29 20:57:53.768909 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.768883    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt" (OuterVolumeSpecName: "kube-api-access-jcjxt") pod "a49201d5-bbcd-4f13-bb02-37e2dca44604" (UID: "a49201d5-bbcd-4f13-bb02-37e2dca44604"). InnerVolumeSpecName "kube-api-access-jcjxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:57:53.868165 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.868130    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a49201d5-bbcd-4f13-bb02-37e2dca44604-config-file\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:53.868165 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:53.868159    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcjxt\" (UniqueName: \"kubernetes.io/projected/a49201d5-bbcd-4f13-bb02-37e2dca44604-kube-api-access-jcjxt\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:57:54.600063 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.600020    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp" event={"ID":"a49201d5-bbcd-4f13-bb02-37e2dca44604","Type":"ContainerDied","Data":"6a5a4d5bf48c0595759338339d1a611efcf26bdaf654af00462d82a3756d8c8d"}
May 29 20:57:54.600063 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.600057    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-q76dp"
May 29 20:57:54.600558 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.600078    2563 scope.go:117] "RemoveContainer" containerID="56b41639d550ee7b0df5e5de07b87e36e6407bc64d5b569fdc728ff591198b62"
May 29 20:57:54.625032 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.625003    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:54.627901 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.627875    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-q76dp"]
May 29 20:57:54.717145 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.717101    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qck4p"]
May 29 20:57:54.717736 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.717703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49201d5-bbcd-4f13-bb02-37e2dca44604" containerName="limitador"
May 29 20:57:54.717736 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.717726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49201d5-bbcd-4f13-bb02-37e2dca44604" containerName="limitador"
May 29 20:57:54.718294 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.717821    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="a49201d5-bbcd-4f13-bb02-37e2dca44604" containerName="limitador"
May 29 20:57:54.720944 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.720920    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.723638 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.723423    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 29 20:57:54.729220 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.729191    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qck4p"]
May 29 20:57:54.876489 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.876399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-data\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.876489 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.876445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxl4x\" (UniqueName: \"kubernetes.io/projected/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-kube-api-access-cxl4x\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.977457 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.977422    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-data\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.977457 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.977464    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cxl4x\" (UniqueName: \"kubernetes.io/projected/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-kube-api-access-cxl4x\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.977808 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.977789    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-data\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:54.986403 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:54.986373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxl4x\" (UniqueName: \"kubernetes.io/projected/7fb1608a-692d-417e-8ecb-6a1eaa6a76bd-kube-api-access-cxl4x\") pod \"postgres-68c4fbbd6f-qck4p\" (UID: \"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd\") " pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:55.034443 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:55.034406    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:55.165444 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:55.165369    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-qck4p"]
May 29 20:57:55.168195 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:57:55.168163    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb1608a_692d_417e_8ecb_6a1eaa6a76bd.slice/crio-364511a64736843e5005c57af881ce1c1e61253f3bf76c11b79d6125a43ef112 WatchSource:0}: Error finding container 364511a64736843e5005c57af881ce1c1e61253f3bf76c11b79d6125a43ef112: Status 404 returned error can't find the container with id 364511a64736843e5005c57af881ce1c1e61253f3bf76c11b79d6125a43ef112
May 29 20:57:55.501351 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:55.501316    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49201d5-bbcd-4f13-bb02-37e2dca44604" path="/var/lib/kubelet/pods/a49201d5-bbcd-4f13-bb02-37e2dca44604/volumes"
May 29 20:57:55.604851 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:55.604813    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-qck4p" event={"ID":"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd","Type":"ContainerStarted","Data":"364511a64736843e5005c57af881ce1c1e61253f3bf76c11b79d6125a43ef112"}
May 29 20:57:57.614691 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:57.614600    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-qck4p" event={"ID":"7fb1608a-692d-417e-8ecb-6a1eaa6a76bd","Type":"ContainerStarted","Data":"ced4d287966c19810956f126460903363b60062e9b6574e9d1f991028425c191"}
May 29 20:57:57.615147 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:57.614720    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:57:57.630451 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:57:57.630397    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-qck4p" podStartSLOduration=1.461993912 podStartE2EDuration="3.630380655s" podCreationTimestamp="2026-05-29 20:57:54 +0000 UTC" firstStartedPulling="2026-05-29 20:57:55.169521825 +0000 UTC m=+638.233687940" lastFinishedPulling="2026-05-29 20:57:57.337908568 +0000 UTC m=+640.402074683" observedRunningTime="2026-05-29 20:57:57.630064691 +0000 UTC m=+640.694230832" watchObservedRunningTime="2026-05-29 20:57:57.630380655 +0000 UTC m=+640.694546794"
May 29 20:58:03.649457 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:03.649426    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-qck4p"
May 29 20:58:27.502958 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.502929    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:58:27.509966 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.509937    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:58:27.510133 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.510059    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:58:27.512423 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.512400    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
May 29 20:58:27.512744 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.512720    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
May 29 20:58:27.512873 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.512747    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-cq44j\""
May 29 20:58:27.512873 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.512845    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
May 29 20:58:27.577545 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.577503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kfq\" (UniqueName: \"kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq\") pod \"maas-keycloak-0\" (UID: \"8031000e-b7ed-4da2-92a4-eaed15b4136b\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:58:27.678528 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.678493    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kfq\" (UniqueName: \"kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq\") pod \"maas-keycloak-0\" (UID: \"8031000e-b7ed-4da2-92a4-eaed15b4136b\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:58:27.686075 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.686047    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kfq\" (UniqueName: \"kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq\") pod \"maas-keycloak-0\" (UID: \"8031000e-b7ed-4da2-92a4-eaed15b4136b\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:58:27.819417 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.819334    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:58:27.948827 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.948800    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:58:27.950839 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:58:27.950804    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8031000e_b7ed_4da2_92a4_eaed15b4136b.slice/crio-8b0a26f5bf1758a652ca12a9fecf22ab1ff40bf8f1d3150705e258872e83e74d WatchSource:0}: Error finding container 8b0a26f5bf1758a652ca12a9fecf22ab1ff40bf8f1d3150705e258872e83e74d: Status 404 returned error can't find the container with id 8b0a26f5bf1758a652ca12a9fecf22ab1ff40bf8f1d3150705e258872e83e74d
May 29 20:58:27.952240 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:27.952223    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 20:58:28.723824 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:28.723768    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8031000e-b7ed-4da2-92a4-eaed15b4136b","Type":"ContainerStarted","Data":"8b0a26f5bf1758a652ca12a9fecf22ab1ff40bf8f1d3150705e258872e83e74d"}
May 29 20:58:32.742421 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:32.742380    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8031000e-b7ed-4da2-92a4-eaed15b4136b","Type":"ContainerStarted","Data":"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"}
May 29 20:58:32.759475 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:32.759413    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.3334225530000001 podStartE2EDuration="5.759398007s" podCreationTimestamp="2026-05-29 20:58:27 +0000 UTC" firstStartedPulling="2026-05-29 20:58:27.952347193 +0000 UTC m=+671.016513312" lastFinishedPulling="2026-05-29 20:58:32.37832265 +0000 UTC m=+675.442488766" observedRunningTime="2026-05-29 20:58:32.758216969 +0000 UTC m=+675.822383149" watchObservedRunningTime="2026-05-29 20:58:32.759398007 +0000 UTC m=+675.823564144"
May 29 20:58:32.820048 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:32.820008    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
May 29 20:58:32.821326 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:32.821288    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:33.820339 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:33.820282    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:34.820247 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:34.820199    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:35.820520 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:35.820468    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:36.820478 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:36.820422    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:37.819500 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:37.819463    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
May 29 20:58:37.821080 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:37.821047    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:38.820053 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:38.819971    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:39.820196 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:39.820138    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:40.820576 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:40.820526    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:41.820388 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:41.820351    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:42.819911 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:42.819854    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:43.820373 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:43.820318    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:44.820107 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:44.820062    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused"
May 29 20:58:45.945837 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:45.945788    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
May 29 20:58:45.963601 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:45.963560    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 29 20:58:55.952739 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:58:55.952651    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:36.558715 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.558627    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5ffdc8c99c-8sqzr"]
May 29 20:59:36.561582 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.561561    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.564156 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.564134    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-7g5gp\""
May 29 20:59:36.564266 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.564202    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 29 20:59:36.570146 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.570123    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5ffdc8c99c-8sqzr"]
May 29 20:59:36.660425 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.660387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbc7m\" (UniqueName: \"kubernetes.io/projected/4b162209-972d-4c7c-9f62-1303a6be6460-kube-api-access-gbc7m\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.660621 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.660453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b162209-972d-4c7c-9f62-1303a6be6460-maas-api-tls\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.761966 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.761930    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gbc7m\" (UniqueName: \"kubernetes.io/projected/4b162209-972d-4c7c-9f62-1303a6be6460-kube-api-access-gbc7m\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.762167 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.762032    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b162209-972d-4c7c-9f62-1303a6be6460-maas-api-tls\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.764875 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.764847    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b162209-972d-4c7c-9f62-1303a6be6460-maas-api-tls\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.770709 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.770681    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbc7m\" (UniqueName: \"kubernetes.io/projected/4b162209-972d-4c7c-9f62-1303a6be6460-kube-api-access-gbc7m\") pod \"maas-api-5ffdc8c99c-8sqzr\" (UID: \"4b162209-972d-4c7c-9f62-1303a6be6460\") " pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:36.874634 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:36.874550    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:37.023312 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:37.023286    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5ffdc8c99c-8sqzr"]
May 29 20:59:37.025877 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:59:37.025848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b162209_972d_4c7c_9f62_1303a6be6460.slice/crio-34bd669ab6e08b83e95fa71b9064ca6e77709cfc8806385674d8b6f54919c9b1 WatchSource:0}: Error finding container 34bd669ab6e08b83e95fa71b9064ca6e77709cfc8806385674d8b6f54919c9b1: Status 404 returned error can't find the container with id 34bd669ab6e08b83e95fa71b9064ca6e77709cfc8806385674d8b6f54919c9b1
May 29 20:59:37.992763 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:37.992724    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr" event={"ID":"4b162209-972d-4c7c-9f62-1303a6be6460","Type":"ContainerStarted","Data":"34bd669ab6e08b83e95fa71b9064ca6e77709cfc8806385674d8b6f54919c9b1"}
May 29 20:59:40.001269 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:40.001234    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr" event={"ID":"4b162209-972d-4c7c-9f62-1303a6be6460","Type":"ContainerStarted","Data":"98eda9d3dc677fcaf6f55ca569aef2039e6c2f22e4993a57260588d2f3c99e99"}
May 29 20:59:40.001657 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:40.001301    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:40.020649 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:40.020576    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr" podStartSLOduration=1.246085858 podStartE2EDuration="4.020561433s" podCreationTimestamp="2026-05-29 20:59:36 +0000 UTC" firstStartedPulling="2026-05-29 20:59:37.027782587 +0000 UTC m=+740.091948702" lastFinishedPulling="2026-05-29 20:59:39.802258148 +0000 UTC m=+742.866424277" observedRunningTime="2026-05-29 20:59:40.018751707 +0000 UTC m=+743.082917845" watchObservedRunningTime="2026-05-29 20:59:40.020561433 +0000 UTC m=+743.084727569"
May 29 20:59:46.010394 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:46.010366    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5ffdc8c99c-8sqzr"
May 29 20:59:47.890327 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:47.890290    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:47.890771 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:47.890511    2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak" containerID="cri-o://65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e" gracePeriod=30
May 29 20:59:49.447340 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:49.447312    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:49.584324 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:49.584242    2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2kfq\" (UniqueName: \"kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq\") pod \"8031000e-b7ed-4da2-92a4-eaed15b4136b\" (UID: \"8031000e-b7ed-4da2-92a4-eaed15b4136b\") "
May 29 20:59:49.586908 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:49.586874    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq" (OuterVolumeSpecName: "kube-api-access-s2kfq") pod "8031000e-b7ed-4da2-92a4-eaed15b4136b" (UID: "8031000e-b7ed-4da2-92a4-eaed15b4136b"). InnerVolumeSpecName "kube-api-access-s2kfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 29 20:59:49.685430 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:49.685393    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2kfq\" (UniqueName: \"kubernetes.io/projected/8031000e-b7ed-4da2-92a4-eaed15b4136b-kube-api-access-s2kfq\") on node \"ip-10-0-133-224.ec2.internal\" DevicePath \"\""
May 29 20:59:50.038252 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.038214    2563 generic.go:358] "Generic (PLEG): container finished" podID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerID="65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e" exitCode=143
May 29 20:59:50.038446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.038272    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8031000e-b7ed-4da2-92a4-eaed15b4136b","Type":"ContainerDied","Data":"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"}
May 29 20:59:50.038446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.038319    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8031000e-b7ed-4da2-92a4-eaed15b4136b","Type":"ContainerDied","Data":"8b0a26f5bf1758a652ca12a9fecf22ab1ff40bf8f1d3150705e258872e83e74d"}
May 29 20:59:50.038446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.038334    2563 scope.go:117] "RemoveContainer" containerID="65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"
May 29 20:59:50.038446 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.038284    2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.047891 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.047872    2563 scope.go:117] "RemoveContainer" containerID="65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"
May 29 20:59:50.048183 ip-10-0-133-224 kubenswrapper[2563]: E0529 20:59:50.048159    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e\": container with ID starting with 65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e not found: ID does not exist" containerID="65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"
May 29 20:59:50.048250 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.048191    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e"} err="failed to get container status \"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e\": rpc error: code = NotFound desc = could not find container \"65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e\": container with ID starting with 65f403fc1f9917d11886142b211b57251653022dc146904b8e9825279ed5b95e not found: ID does not exist"
May 29 20:59:50.060095 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.060069    2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:50.065622 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.065597    2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:50.085896 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.085856    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:50.086350 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.086331    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak"
May 29 20:59:50.086350 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.086352    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak"
May 29 20:59:50.086539 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.086470    2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" containerName="keycloak"
May 29 20:59:50.091308 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.091287    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.093850 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.093826    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\""
May 29 20:59:50.093850 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.093857    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-cq44j\""
May 29 20:59:50.094098 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.093889    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\""
May 29 20:59:50.094098 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.093857    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\""
May 29 20:59:50.094229 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.094124    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\""
May 29 20:59:50.097757 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.097732    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:50.190277 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.190235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f682fadc-c4f5-4017-ac8b-7bce2928c208-test-realms\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.190462 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.190410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lccjj\" (UniqueName: \"kubernetes.io/projected/f682fadc-c4f5-4017-ac8b-7bce2928c208-kube-api-access-lccjj\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.291097 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.290970    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lccjj\" (UniqueName: \"kubernetes.io/projected/f682fadc-c4f5-4017-ac8b-7bce2928c208-kube-api-access-lccjj\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.291097 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.291057    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f682fadc-c4f5-4017-ac8b-7bce2928c208-test-realms\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.291831 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.291811    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f682fadc-c4f5-4017-ac8b-7bce2928c208-test-realms\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.298975 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.298939    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lccjj\" (UniqueName: \"kubernetes.io/projected/f682fadc-c4f5-4017-ac8b-7bce2928c208-kube-api-access-lccjj\") pod \"maas-keycloak-0\" (UID: \"f682fadc-c4f5-4017-ac8b-7bce2928c208\") " pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.401953 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.401920    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:50.537260 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:50.537233    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"]
May 29 20:59:50.539740 ip-10-0-133-224 kubenswrapper[2563]: W0529 20:59:50.539714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf682fadc_c4f5_4017_ac8b_7bce2928c208.slice/crio-8b6659521a60d74e8761bcde49a92a1e708838c27dca2b58395428c4d42791fe WatchSource:0}: Error finding container 8b6659521a60d74e8761bcde49a92a1e708838c27dca2b58395428c4d42791fe: Status 404 returned error can't find the container with id 8b6659521a60d74e8761bcde49a92a1e708838c27dca2b58395428c4d42791fe
May 29 20:59:51.044559 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.044516    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f682fadc-c4f5-4017-ac8b-7bce2928c208","Type":"ContainerStarted","Data":"9907663468b9cd1cf3214991a097a143ef4be73c21a6f2be741491bf04ab4e77"}
May 29 20:59:51.044559 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.044566    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f682fadc-c4f5-4017-ac8b-7bce2928c208","Type":"ContainerStarted","Data":"8b6659521a60d74e8761bcde49a92a1e708838c27dca2b58395428c4d42791fe"}
May 29 20:59:51.062442 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.062386    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.780677172 podStartE2EDuration="1.062367237s" podCreationTimestamp="2026-05-29 20:59:50 +0000 UTC" firstStartedPulling="2026-05-29 20:59:50.541427998 +0000 UTC m=+753.605594113" lastFinishedPulling="2026-05-29 20:59:50.823118059 +0000 UTC m=+753.887284178" observedRunningTime="2026-05-29 20:59:51.061676875 +0000 UTC m=+754.125843003" watchObservedRunningTime="2026-05-29 20:59:51.062367237 +0000 UTC m=+754.126533377"
May 29 20:59:51.402857 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.402815    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0"
May 29 20:59:51.404785 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.404755    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:51.502798 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:51.502759    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8031000e-b7ed-4da2-92a4-eaed15b4136b" path="/var/lib/kubelet/pods/8031000e-b7ed-4da2-92a4-eaed15b4136b/volumes"
May 29 20:59:52.403178 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:52.403122    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:53.402440 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:53.402390    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:54.402476 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:54.402416    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:55.402915 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:55.402865    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:56.403120 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:56.403066    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:57.403026 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:57.402953    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:58.403128 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:58.403079    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 20:59:59.402953 ip-10-0-133-224 kubenswrapper[2563]: I0529 20:59:59.402898    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 21:00:00.402431 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:00.402386    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0"
May 29 21:00:00.402945 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:00.402744    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 21:00:01.402964 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:01.402913    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 21:00:02.402503 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:02.402453    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 21:00:03.403051 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:03.402971    2563 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.40:9000/health/started\": dial tcp 10.134.0.40:9000: connect: connection refused"
May 29 21:00:04.539562 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:04.538864    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0"
May 29 21:00:04.559853 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:04.559810    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f682fadc-c4f5-4017-ac8b-7bce2928c208" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 29 21:00:14.545808 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:14.545765    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0"
May 29 21:00:37.594304 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.594231    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"]
May 29 21:00:37.596973 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.596958    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.601176 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.601153    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-trsqn\""
May 29 21:00:37.601280 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.601180    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 29 21:00:37.601773 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.601755    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 29 21:00:37.601881 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.601868    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 29 21:00:37.623624 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.623592    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"]
May 29 21:00:37.764653 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.764837 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.764837 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f470001-4955-46ed-87ad-94e0aa3d195c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.764837 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v96n\" (UniqueName: \"kubernetes.io/projected/4f470001-4955-46ed-87ad-94e0aa3d195c-kube-api-access-4v96n\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.764837 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.764837 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.764827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866352 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866265    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866352 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866305    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866581 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866352    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866581 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866402    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866581 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866421    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f470001-4955-46ed-87ad-94e0aa3d195c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866581 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866480    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4v96n\" (UniqueName: \"kubernetes.io/projected/4f470001-4955-46ed-87ad-94e0aa3d195c-kube-api-access-4v96n\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866805 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866771    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.866935 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.866903    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.867069 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.867052    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.868910 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.868894    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f470001-4955-46ed-87ad-94e0aa3d195c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.869238 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.869216    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f470001-4955-46ed-87ad-94e0aa3d195c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.878732 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.878708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v96n\" (UniqueName: \"kubernetes.io/projected/4f470001-4955-46ed-87ad-94e0aa3d195c-kube-api-access-4v96n\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652\" (UID: \"4f470001-4955-46ed-87ad-94e0aa3d195c\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:37.908258 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:37.908218    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:38.054826 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:38.054802    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"]
May 29 21:00:38.057119 ip-10-0-133-224 kubenswrapper[2563]: W0529 21:00:38.057090    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f470001_4955_46ed_87ad_94e0aa3d195c.slice/crio-dac8ff53d27953ab1c4326fad537fa3badbabbd915664caa3397ef6317b2b69d WatchSource:0}: Error finding container dac8ff53d27953ab1c4326fad537fa3badbabbd915664caa3397ef6317b2b69d: Status 404 returned error can't find the container with id dac8ff53d27953ab1c4326fad537fa3badbabbd915664caa3397ef6317b2b69d
May 29 21:00:38.240929 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:38.240890    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652" event={"ID":"4f470001-4955-46ed-87ad-94e0aa3d195c","Type":"ContainerStarted","Data":"dac8ff53d27953ab1c4326fad537fa3badbabbd915664caa3397ef6317b2b69d"}
May 29 21:00:42.260047 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:42.259981    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652" event={"ID":"4f470001-4955-46ed-87ad-94e0aa3d195c","Type":"ContainerStarted","Data":"fd5e28ad6cbe4a303538647e983ad4a590f40c9b9e341f011ccf610d5c485d7b"}
May 29 21:00:44.268673 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:44.268627    2563 generic.go:358] "Generic (PLEG): container finished" podID="4f470001-4955-46ed-87ad-94e0aa3d195c" containerID="fd5e28ad6cbe4a303538647e983ad4a590f40c9b9e341f011ccf610d5c485d7b" exitCode=0
May 29 21:00:44.269126 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:44.268705    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652" event={"ID":"4f470001-4955-46ed-87ad-94e0aa3d195c","Type":"ContainerDied","Data":"fd5e28ad6cbe4a303538647e983ad4a590f40c9b9e341f011ccf610d5c485d7b"}
May 29 21:00:56.316444 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:56.316406    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652" event={"ID":"4f470001-4955-46ed-87ad-94e0aa3d195c","Type":"ContainerStarted","Data":"1cb052d736ec59a35cce0f574ce92f1454dcfbe393394d8bd102540c182f0625"}
May 29 21:00:56.352422 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:56.352369    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652" podStartSLOduration=1.70797374 podStartE2EDuration="19.352355608s" podCreationTimestamp="2026-05-29 21:00:37 +0000 UTC" firstStartedPulling="2026-05-29 21:00:38.05882188 +0000 UTC m=+801.122987997" lastFinishedPulling="2026-05-29 21:00:55.703203745 +0000 UTC m=+818.767369865" observedRunningTime="2026-05-29 21:00:56.350252508 +0000 UTC m=+819.414418659" watchObservedRunningTime="2026-05-29 21:00:56.352355608 +0000 UTC m=+819.416521744"
May 29 21:00:57.908866 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:57.908822    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:57.908866 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:57.908867    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:57.924543 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:57.924513    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:58.337029 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:58.336981    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652"
May 29 21:00:59.979585 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:59.979553    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"]
May 29 21:00:59.983259 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:59.983240    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:00:59.987517 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:00:59.987497    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
May 29 21:01:00.005008 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.004957    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"]
May 29 21:01:00.062623 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.062623 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.062875 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b116909-ab50-467f-b97c-ff0a49708142-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.062875 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.062875 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.063068 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.062882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lkr6\" (UniqueName: \"kubernetes.io/projected/1b116909-ab50-467f-b97c-ff0a49708142-kube-api-access-6lkr6\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164156 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164118    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b116909-ab50-467f-b97c-ff0a49708142-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164364 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164270    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164364 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164333    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164486 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164385    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lkr6\" (UniqueName: \"kubernetes.io/projected/1b116909-ab50-467f-b97c-ff0a49708142-kube-api-access-6lkr6\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164486 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164469    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.164585 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.164524    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.165744 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.165655    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.167043 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.166213    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.167043 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.166534    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.171943 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.171843    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b116909-ab50-467f-b97c-ff0a49708142-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.172203 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.172184    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b116909-ab50-467f-b97c-ff0a49708142-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.183633 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.183605    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lkr6\" (UniqueName: \"kubernetes.io/projected/1b116909-ab50-467f-b97c-ff0a49708142-kube-api-access-6lkr6\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v\" (UID: \"1b116909-ab50-467f-b97c-ff0a49708142\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.293666 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.293554    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:01:00.474112 ip-10-0-133-224 kubenswrapper[2563]: W0529 21:01:00.474072    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b116909_ab50_467f_b97c_ff0a49708142.slice/crio-47e0aa92cd6c9ed02bd165b9fbe56c907dda024f43951aa7d1f7cd4804ca77f2 WatchSource:0}: Error finding container 47e0aa92cd6c9ed02bd165b9fbe56c907dda024f43951aa7d1f7cd4804ca77f2: Status 404 returned error can't find the container with id 47e0aa92cd6c9ed02bd165b9fbe56c907dda024f43951aa7d1f7cd4804ca77f2
May 29 21:01:00.478896 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:00.478870    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"]
May 29 21:01:01.336815 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:01.336777    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v" event={"ID":"1b116909-ab50-467f-b97c-ff0a49708142","Type":"ContainerStarted","Data":"d44b2632ac2c892de99f34a69681a9129657c5e3ea3c759820843a5327d65dd3"}
May 29 21:01:01.336815 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:01.336815    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v" event={"ID":"1b116909-ab50-467f-b97c-ff0a49708142","Type":"ContainerStarted","Data":"47e0aa92cd6c9ed02bd165b9fbe56c907dda024f43951aa7d1f7cd4804ca77f2"}
May 29 21:01:02.688149 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.688112    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"]
May 29 21:01:02.692372 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.692351    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.696101 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.696070    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
May 29 21:01:02.713268 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.713233    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"]
May 29 21:01:02.789554 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be247ac-93c5-4b5f-bd97-a057a67b5118-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.789725 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhnnf\" (UniqueName: \"kubernetes.io/projected/1be247ac-93c5-4b5f-bd97-a057a67b5118-kube-api-access-lhnnf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.789725 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.789725 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.789725 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.789725 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.789692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.890909 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.890866    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891103 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.890979    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be247ac-93c5-4b5f-bd97-a057a67b5118-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891103 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891055    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lhnnf\" (UniqueName: \"kubernetes.io/projected/1be247ac-93c5-4b5f-bd97-a057a67b5118-kube-api-access-lhnnf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891178 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891132    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891178 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891163    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891269 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891189    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891454 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891429    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891532 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891469    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.891594 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.891571    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.893528 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.893509    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1be247ac-93c5-4b5f-bd97-a057a67b5118-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.893703 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.893685    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1be247ac-93c5-4b5f-bd97-a057a67b5118-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:02.901884 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:02.901858    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhnnf\" (UniqueName: \"kubernetes.io/projected/1be247ac-93c5-4b5f-bd97-a057a67b5118-kube-api-access-lhnnf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8\" (UID: \"1be247ac-93c5-4b5f-bd97-a057a67b5118\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:03.004043 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:03.004009    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:03.154021 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:03.153975    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"]
May 29 21:01:03.156022 ip-10-0-133-224 kubenswrapper[2563]: W0529 21:01:03.155967    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be247ac_93c5_4b5f_bd97_a057a67b5118.slice/crio-7b6fa54c8b2b860a8068d527292379393757f25718ce0a7e3e6055400972a613 WatchSource:0}: Error finding container 7b6fa54c8b2b860a8068d527292379393757f25718ce0a7e3e6055400972a613: Status 404 returned error can't find the container with id 7b6fa54c8b2b860a8068d527292379393757f25718ce0a7e3e6055400972a613
May 29 21:01:03.346635 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:03.346537    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8" event={"ID":"1be247ac-93c5-4b5f-bd97-a057a67b5118","Type":"ContainerStarted","Data":"430e59836f032ec4741074e53bb19672e2304193333809e92b71fc434697d09f"}
May 29 21:01:03.346635 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:03.346580    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8" event={"ID":"1be247ac-93c5-4b5f-bd97-a057a67b5118","Type":"ContainerStarted","Data":"7b6fa54c8b2b860a8068d527292379393757f25718ce0a7e3e6055400972a613"}
May 29 21:01:05.355796 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:05.355766    2563 generic.go:358] "Generic (PLEG): container finished" podID="1be247ac-93c5-4b5f-bd97-a057a67b5118" containerID="430e59836f032ec4741074e53bb19672e2304193333809e92b71fc434697d09f" exitCode=0
May 29 21:01:05.356193 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:05.355806    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8" event={"ID":"1be247ac-93c5-4b5f-bd97-a057a67b5118","Type":"ContainerDied","Data":"430e59836f032ec4741074e53bb19672e2304193333809e92b71fc434697d09f"}
May 29 21:01:21.426760 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:21.426725    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8" event={"ID":"1be247ac-93c5-4b5f-bd97-a057a67b5118","Type":"ContainerStarted","Data":"b069b721b77f99ba416dca7d50a20bd24ab00fc1b7456da66c6ce1bc40c9ff4c"}
May 29 21:01:21.464327 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:21.464271    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8" podStartSLOduration=3.49996741 podStartE2EDuration="19.464257957s" podCreationTimestamp="2026-05-29 21:01:02 +0000 UTC" firstStartedPulling="2026-05-29 21:01:05.356517737 +0000 UTC m=+828.420683852" lastFinishedPulling="2026-05-29 21:01:21.320808273 +0000 UTC m=+844.384974399" observedRunningTime="2026-05-29 21:01:21.463136963 +0000 UTC m=+844.527303101" watchObservedRunningTime="2026-05-29 21:01:21.464257957 +0000 UTC m=+844.528424094"
May 29 21:01:23.004341 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:23.004302    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:23.004760 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:23.004630    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:23.017670 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:23.017641    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:23.445738 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:23.445659    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8"
May 29 21:01:54.555914 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:54.555881    2563 generic.go:358] "Generic (PLEG): container finished" podID="1b116909-ab50-467f-b97c-ff0a49708142" containerID="d44b2632ac2c892de99f34a69681a9129657c5e3ea3c759820843a5327d65dd3" exitCode=0
May 29 21:01:54.556355 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:54.555953    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v" event={"ID":"1b116909-ab50-467f-b97c-ff0a49708142","Type":"ContainerDied","Data":"d44b2632ac2c892de99f34a69681a9129657c5e3ea3c759820843a5327d65dd3"}
May 29 21:01:55.561029 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:55.560911    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v" event={"ID":"1b116909-ab50-467f-b97c-ff0a49708142","Type":"ContainerStarted","Data":"9994060b07c91e855ce596bfb050e85b1d9ada26f027c3e4ba6548a7eaa20ec4"}
May 29 21:01:55.587969 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:01:55.587916    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v" podStartSLOduration=55.890359541 podStartE2EDuration="56.587902213s" podCreationTimestamp="2026-05-29 21:00:59 +0000 UTC" firstStartedPulling="2026-05-29 21:01:54.556677622 +0000 UTC m=+877.620843740" lastFinishedPulling="2026-05-29 21:01:55.254220298 +0000 UTC m=+878.318386412" observedRunningTime="2026-05-29 21:01:55.587716318 +0000 UTC m=+878.651882457" watchObservedRunningTime="2026-05-29 21:01:55.587902213 +0000 UTC m=+878.652068351"
May 29 21:02:00.293767 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:00.293732    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:02:00.293767 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:00.293773    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:02:00.306179 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:00.306155    2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:02:00.590522 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:00.590444    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v"
May 29 21:02:17.430670 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:17.430633    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 21:02:17.432042 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:17.432017    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 21:02:17.436930 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:17.436907    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 21:02:17.437822 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:02:17.437802    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 21:07:17.459942 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:17.459901    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 21:07:17.462612 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:17.462590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 21:07:17.465967 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:17.465944    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 21:07:17.468920 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:17.468898    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5h2kj_51351a92-8ecd-4bdd-8d04-7b6ea82516d8/ovn-acl-logging/0.log"
May 29 21:07:25.471904 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:25.471871    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6b77dbffcd-2wbhr_f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2/nginx/0.log"
May 29 21:07:25.953846 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:25.953755    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5ffdc8c99c-8sqzr_4b162209-972d-4c7c-9f62-1303a6be6460/maas-api/0.log"
May 29 21:07:26.819477 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:26.819444    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4c488574-q96fl_1f599285-5f82-44e0-a230-bfcac8643330/copy-manifests/0.log"
May 29 21:07:26.831813 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:26.831785    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4c488574-q96fl_1f599285-5f82-44e0-a230-bfcac8643330/manager/0.log"
May 29 21:07:26.953414 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:26.953364    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-qck4p_7fb1608a-692d-417e-8ecb-6a1eaa6a76bd/postgres/0.log"
May 29 21:07:28.524979 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:28.524914    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-5bj4d_773d544e-b8f5-4667-a1a2-ed3cd963cfe1/manager/0.log"
May 29 21:07:28.751384 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:28.751351    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-m7jcr_bcec8537-e8c0-4628-a5f1-e3e47c32ca05/registry-server/0.log"
May 29 21:07:28.870718 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:28.870644    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-z2vzd_78d4c292-41ef-4383-9d60-5775cdcda22b/manager/0.log"
May 29 21:07:29.099439 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:29.099406    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-xkrgg_7b7800dd-b5b8-451f-89b0-e1ff9599aa0e/manager/0.log"
May 29 21:07:29.454809 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:29.454782    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl_471729a2-3820-4e21-9382-37e134c8acea/istio-proxy/0.log"
May 29 21:07:29.926891 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:29.926864    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-446s5_23bfeffb-f0c8-4c55-8f4b-60fc0312ee73/istio-proxy/0.log"
May 29 21:07:30.517435 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.517410    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652_4f470001-4955-46ed-87ad-94e0aa3d195c/storage-initializer/0.log"
May 29 21:07:30.526219 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.526197    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-c6652_4f470001-4955-46ed-87ad-94e0aa3d195c/main/0.log"
May 29 21:07:30.771346 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.771264    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v_1b116909-ab50-467f-b97c-ff0a49708142/main/0.log"
May 29 21:07:30.778596 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.778571    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-4fj9v_1b116909-ab50-467f-b97c-ff0a49708142/storage-initializer/0.log"
May 29 21:07:30.893869 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.893844    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8_1be247ac-93c5-4b5f-bd97-a057a67b5118/storage-initializer/0.log"
May 29 21:07:30.901455 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:30.901434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9f2bs8_1be247ac-93c5-4b5f-bd97-a057a67b5118/main/0.log"
May 29 21:07:38.468737 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:38.468705    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tgwnn_3f07a1d7-f5d2-4d11-971a-745b24aed88f/global-pull-secret-syncer/0.log"
May 29 21:07:38.549865 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:38.549835    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6l5wp_9d05b663-a9d3-43a2-9d2e-42aee9296f71/konnectivity-agent/0.log"
May 29 21:07:38.641415 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:38.641372    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-224.ec2.internal_0654a00223f55ad393dab77c88c66323/haproxy/0.log"
May 29 21:07:42.787168 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:42.787138    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-5bj4d_773d544e-b8f5-4667-a1a2-ed3cd963cfe1/manager/0.log"
May 29 21:07:42.867058 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:42.866943    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-m7jcr_bcec8537-e8c0-4628-a5f1-e3e47c32ca05/registry-server/0.log"
May 29 21:07:42.911697 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:42.911660    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-z2vzd_78d4c292-41ef-4383-9d60-5775cdcda22b/manager/0.log"
May 29 21:07:42.964538 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:42.964512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-xkrgg_7b7800dd-b5b8-451f-89b0-e1ff9599aa0e/manager/0.log"
May 29 21:07:44.439547 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.439461    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jvqcb_72a67383-0522-42ab-aeb0-dbd813814032/kube-state-metrics/0.log"
May 29 21:07:44.464385 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.464359    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jvqcb_72a67383-0522-42ab-aeb0-dbd813814032/kube-rbac-proxy-main/0.log"
May 29 21:07:44.492703 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.492678    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-jvqcb_72a67383-0522-42ab-aeb0-dbd813814032/kube-rbac-proxy-self/0.log"
May 29 21:07:44.556082 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.556060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-w5jzg_33c1bdf8-6ef7-49e5-92ae-c001ccbf0b3d/monitoring-plugin/0.log"
May 29 21:07:44.711064 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.710971    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g84rm_a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e/node-exporter/0.log"
May 29 21:07:44.733915 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.733876    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g84rm_a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e/kube-rbac-proxy/0.log"
May 29 21:07:44.755206 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.755178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-g84rm_a30e96d8-0364-483c-8eb1-f7b2dfa3fa6e/init-textfile/0.log"
May 29 21:07:44.866359 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.866330    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-m45qc_17b20fd4-cd52-46e5-9ee4-edde38dab402/kube-rbac-proxy-main/0.log"
May 29 21:07:44.887720 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.887691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-m45qc_17b20fd4-cd52-46e5-9ee4-edde38dab402/kube-rbac-proxy-self/0.log"
May 29 21:07:44.910623 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.910600    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-m45qc_17b20fd4-cd52-46e5-9ee4-edde38dab402/openshift-state-metrics/0.log"
May 29 21:07:44.953085 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.953051    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/prometheus/0.log"
May 29 21:07:44.976676 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.976654    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/config-reloader/0.log"
May 29 21:07:44.999059 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:44.999034    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/thanos-sidecar/0.log"
May 29 21:07:45.028000 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.027967    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/kube-rbac-proxy-web/0.log"
May 29 21:07:45.049528 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.049508    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/kube-rbac-proxy/0.log"
May 29 21:07:45.073283 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.073265    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/kube-rbac-proxy-thanos/0.log"
May 29 21:07:45.095800 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.095784    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2537471a-58be-44e1-b6be-0b7e5cd33fe6/init-config-reloader/0.log"
May 29 21:07:45.301539 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.301469    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/thanos-query/0.log"
May 29 21:07:45.325620 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.325592    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/kube-rbac-proxy-web/0.log"
May 29 21:07:45.367360 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.367338    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/kube-rbac-proxy/0.log"
May 29 21:07:45.397956 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.397934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/prom-label-proxy/0.log"
May 29 21:07:45.425773 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.425750    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/kube-rbac-proxy-rules/0.log"
May 29 21:07:45.449871 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:45.449844    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56c9fdb949-t5kcq_d496a421-26e2-46c8-8685-4f7ddd609db6/kube-rbac-proxy-metrics/0.log"
May 29 21:07:46.597128 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:46.597094    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-j8tv4_f6767aba-99b2-4e19-be09-fe95ef1f352f/networking-console-plugin/0.log"
May 29 21:07:47.197802 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:47.197771    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/1.log"
May 29 21:07:47.203563 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:47.203536    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-p2929_64dbcb87-94d3-4e99-9040-c81c0a726851/console-operator/2.log"
May 29 21:07:48.092394 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.092359    2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"]
May 29 21:07:48.095909 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.095888    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.102381 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.102347    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg47z\"/\"openshift-service-ca.crt\""
May 29 21:07:48.103093 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.103064    2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gg47z\"/\"default-dockercfg-4sgqx\""
May 29 21:07:48.103406 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.103336    2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg47z\"/\"kube-root-ca.crt\""
May 29 21:07:48.104400 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.104379    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"]
May 29 21:07:48.203017 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.202961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-proc\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.203186 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.203039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfsq2\" (UniqueName: \"kubernetes.io/projected/de24ea5c-cc22-48b0-a8ce-59266bb2794a-kube-api-access-cfsq2\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.203186 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.203066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-sys\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.203186 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.203095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-lib-modules\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.203291 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.203235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-podres\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.303976 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.303932    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-sys\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304018    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-lib-modules\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304074    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-sys\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304083    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-podres\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304165    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-proc\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304193    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-podres\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304208    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-lib-modules\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304229    2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cfsq2\" (UniqueName: \"kubernetes.io/projected/de24ea5c-cc22-48b0-a8ce-59266bb2794a-kube-api-access-cfsq2\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.304289 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.304270    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de24ea5c-cc22-48b0-a8ce-59266bb2794a-proc\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.312303 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.312269    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfsq2\" (UniqueName: \"kubernetes.io/projected/de24ea5c-cc22-48b0-a8ce-59266bb2794a-kube-api-access-cfsq2\") pod \"perf-node-gather-daemonset-r4ms5\" (UID: \"de24ea5c-cc22-48b0-a8ce-59266bb2794a\") " pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.406927 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.406835    2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.543549 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.543527    2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"]
May 29 21:07:48.546049 ip-10-0-133-224 kubenswrapper[2563]: W0529 21:07:48.546018    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podde24ea5c_cc22_48b0_a8ce_59266bb2794a.slice/crio-e7e7157797232a1229e1e7b24fc72208b183341205eeca3715ec6b340b3981a8 WatchSource:0}: Error finding container e7e7157797232a1229e1e7b24fc72208b183341205eeca3715ec6b340b3981a8: Status 404 returned error can't find the container with id e7e7157797232a1229e1e7b24fc72208b183341205eeca3715ec6b340b3981a8
May 29 21:07:48.548013 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.547973    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 29 21:07:48.892321 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.892284    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5" event={"ID":"de24ea5c-cc22-48b0-a8ce-59266bb2794a","Type":"ContainerStarted","Data":"336c8003a53eef0478a1de4e239ccf51eec36d17d8ce0b95cb2c1ab5213bc29d"}
May 29 21:07:48.892321 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.892321    2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5" event={"ID":"de24ea5c-cc22-48b0-a8ce-59266bb2794a","Type":"ContainerStarted","Data":"e7e7157797232a1229e1e7b24fc72208b183341205eeca3715ec6b340b3981a8"}
May 29 21:07:48.892546 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.892352    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:48.921796 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:48.921744    2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5" podStartSLOduration=0.921728232 podStartE2EDuration="921.728232ms" podCreationTimestamp="2026-05-29 21:07:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-29 21:07:48.9196881 +0000 UTC m=+1231.983854238" watchObservedRunningTime="2026-05-29 21:07:48.921728232 +0000 UTC m=+1231.985894373"
May 29 21:07:49.396723 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:49.396690    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jdb2d_7d207e7d-a860-433d-ba4f-67fcd022b23f/dns/0.log"
May 29 21:07:49.442609 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:49.442582    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jdb2d_7d207e7d-a860-433d-ba4f-67fcd022b23f/kube-rbac-proxy/0.log"
May 29 21:07:49.609755 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:49.609726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r6q6z_56d938c5-6d8c-4583-9cec-ebcc03a9d29e/dns-node-resolver/0.log"
May 29 21:07:50.136388 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:50.136356    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85f44c86b7-mmd6c_ecf6cd45-33c4-4205-b1e1-ca1d8bebc220/registry/0.log"
May 29 21:07:50.181183 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:50.181152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gtqz_b943e2cb-8681-47db-8d8f-da6eadc7afa9/node-ca/0.log"
May 29 21:07:51.225261 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:51.225232    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bjqjjl_471729a2-3820-4e21-9382-37e134c8acea/istio-proxy/0.log"
May 29 21:07:51.432487 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:51.432457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-446s5_23bfeffb-f0c8-4c55-8f4b-60fc0312ee73/istio-proxy/0.log"
May 29 21:07:52.260523 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:52.260482    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q689n_f74cd5ec-fc29-41c8-8986-230ca8c7759a/serve-healthcheck-canary/0.log"
May 29 21:07:52.981773 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:52.981747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd4gs_4d875c44-7ad7-4dd4-97d7-13b6e3706641/kube-rbac-proxy/0.log"
May 29 21:07:53.006293 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:53.006273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd4gs_4d875c44-7ad7-4dd4-97d7-13b6e3706641/exporter/0.log"
May 29 21:07:53.033650 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:53.033627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vd4gs_4d875c44-7ad7-4dd4-97d7-13b6e3706641/extractor/0.log"
May 29 21:07:54.907425 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:54.907397    2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gg47z/perf-node-gather-daemonset-r4ms5"
May 29 21:07:55.337134 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:55.337102    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6b77dbffcd-2wbhr_f6dd10a5-2ad6-4a46-ae06-e4eeefc3a4b2/nginx/0.log"
May 29 21:07:55.458668 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:55.458638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5ffdc8c99c-8sqzr_4b162209-972d-4c7c-9f62-1303a6be6460/maas-api/0.log"
May 29 21:07:55.801231 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:55.801200    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4c488574-q96fl_1f599285-5f82-44e0-a230-bfcac8643330/manager/0.log"
May 29 21:07:55.821529 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:55.821495    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4c488574-q96fl_1f599285-5f82-44e0-a230-bfcac8643330/copy-manifests/0.log"
May 29 21:07:55.848305 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:55.848265    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-qck4p_7fb1608a-692d-417e-8ecb-6a1eaa6a76bd/postgres/0.log"
May 29 21:07:57.404055 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:07:57.403945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cdf9b5b7c-z5k5s_44f11fcd-aab2-4b43-a11f-710ce1e4428d/manager/0.log"
May 29 21:08:03.954521 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:03.954485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/kube-multus-additional-cni-plugins/0.log"
May 29 21:08:03.981141 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:03.981116    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/egress-router-binary-copy/0.log"
May 29 21:08:04.009276 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.009255    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/cni-plugins/0.log"
May 29 21:08:04.040396 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.040369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/bond-cni-plugin/0.log"
May 29 21:08:04.068262 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.068238    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/routeoverride-cni/0.log"
May 29 21:08:04.093738 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.093716    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/whereabouts-cni-bincopy/0.log"
May 29 21:08:04.118663 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.118635    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5cpqr_72e3d4a4-0a37-4f38-964b-9209a64c31e7/whereabouts-cni/0.log"
May 29 21:08:04.561376 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.561299    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bltbg_ee238d75-217d-4f53-98ea-6c4db8935526/kube-multus/0.log"
May 29 21:08:04.799532 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.799510    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k2wzp_0873cca3-a3cf-4320-b20f-55a5704aa6c5/network-metrics-daemon/0.log"
May 29 21:08:04.831517 ip-10-0-133-224 kubenswrapper[2563]: I0529 21:08:04.831420    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k2wzp_0873cca3-a3cf-4320-b20f-55a5704aa6c5/kube-rbac-proxy/0.log"