Jul 16 22:11:42.166414 ip-10-0-133-196 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 22:11:42.166425 ip-10-0-133-196 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 22:11:42.166432 ip-10-0-133-196 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 22:11:42.166655 ip-10-0-133-196 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 22:11:52.343945 ip-10-0-133-196 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 22:11:52.343964 ip-10-0-133-196 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b15c51cb413040518a9514e5c8c7c0a9 --
Jul 16 22:13:48.737562 ip-10-0-133-196 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 22:13:49.129478 ip-10-0-133-196 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:13:49.129478 ip-10-0-133-196 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 22:13:49.129478 ip-10-0-133-196 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:13:49.129478 ip-10-0-133-196 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 22:13:49.129478 ip-10-0-133-196 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 22:13:49.130284 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.129525    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 22:13:49.133912 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133890    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.133912 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133909    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.133912 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133916    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133923    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133928    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133932    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133936    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133940    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133943    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133947    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133951    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133955    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133958    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133962    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133966    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133970    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133974    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133978    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133982    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133986    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133991    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.134121 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133994    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.133998    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134009    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134013    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134017    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134022    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134026    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134030    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134034    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134038    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134043    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134049    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134053    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134058    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134062    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134067    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134071    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134076    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134080    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134084    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.134903 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134088    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134093    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134099    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134105    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134110    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134115    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134120    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134125    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134130    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134135    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134139    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134143    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134148    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134152    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134156    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134160    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134165    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134169    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134173    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134177    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.135768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134181    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134186    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134189    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134194    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134198    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134203    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134207    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134213    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134217    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134221    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134225    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134229    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134233    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134238    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134244    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134251    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134259    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134264    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134268    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134273    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.136615 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134278    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134282    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134309    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134313    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134318    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134322    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134326    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134330    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134334    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134339    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134343    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134347    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134351    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134355    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134359    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134364    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134368    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.134373    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135537    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135548    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.137485 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135553    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135558    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135562    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135567    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135571    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135575    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135579    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135583    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135588    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135594    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135598    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135602    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135606    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135611    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135615    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135620    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135625    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135630    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135635    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.138339 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135639    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135643    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135647    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135652    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135656    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135662    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135668    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135673    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135677    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135681    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135685    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135690    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135696    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135700    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135705    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135709    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135713    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135718    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135722    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.139027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135725    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135730    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135734    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135738    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135743    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135747    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135751    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135756    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135760    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135763    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135767    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135771    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135775    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135779    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135786    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135792    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135797    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135802    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135808    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135812    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.139648 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135817    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135822    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135826    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135831    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135835    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135841    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135845    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135850    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135855    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135859    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135863    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135869    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135873    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135877    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135881    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135885    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135890    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135896    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135900    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135904    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.140273 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135911    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135916    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135920    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135925    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135929    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135934    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135937    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135942    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135946    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135950    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135955    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135959    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135963    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135967    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135972    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135976    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135980    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135985    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.135989    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136612    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.140837 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136621    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136626    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136630    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136634    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136639    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136643    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136647    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136651    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136656    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136661    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136665    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136669    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136677    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136681    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136686    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136691    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136695    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136700    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136704    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136708    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.141372 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136712    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136716    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136720    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136724    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136728    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136733    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136737    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136741    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136746    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136750    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136754    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136758    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136762    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136766    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136770    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136775    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136779    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136784    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136788    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136792    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.141872 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136796    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136800    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136805    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136809    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136816    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136820    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136825    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136829    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136833    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136837    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136841    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136846    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136851    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136855    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136859    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136863    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136867    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136871    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136875    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.142359 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136879    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136883    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136891    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136897    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136903    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136908    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136913    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136918    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136923    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136928    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136932    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136937    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136941    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136946    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136951    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136957    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136963    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136976    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136981    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.142800 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136985    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136991    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.136997    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137001    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137006    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137010    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137014    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137018    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137022    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137027    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137031    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137035    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137039    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137044    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137049    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137055    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137059    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137063    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137067    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.137071    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.143232 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137171    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137182    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137194    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137201    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137207    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137212    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137218    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137225    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137230    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137238    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137249    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137255    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137260    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137265    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137270    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137275    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137279    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137300    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137306    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137311    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137317    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137322    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137327    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137333    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 22:13:49.143802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137338    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137343    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137349    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137355    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137360    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137365    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137370    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137376    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137382    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137387    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137391    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137396    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137401    2556 flags.go:64] FLAG: --enable-server="true"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137406    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137414    2556 flags.go:64] FLAG: --event-burst="100"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137419    2556 flags.go:64] FLAG: --event-qps="50"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137424    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137429    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137435    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137442    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137447    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137451    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137456    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137461    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137466    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 22:13:49.144827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137471    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137475    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137480    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137484    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137488    2556 flags.go:64] FLAG: --feature-gates=""
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137494    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137499    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137503    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137509    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137514    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137519    2556 flags.go:64] FLAG: --help="false"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137524    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-133-196.ec2.internal"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137529    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137534    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137538    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137544    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137549    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137553    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137558    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137563    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137567    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137572    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137577    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137581    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 16 22:13:49.145942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137586    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137591    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137596    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137601    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137605    2556 flags.go:64] FLAG: --lock-file=""
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137610    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137615    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137619    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137628    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137633    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137637    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137642    2556 flags.go:64] FLAG: --logging-format="text"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137647    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137652    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137657    2556 flags.go:64] FLAG: --manifest-url=""
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137661    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137668    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137673    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137679    2556 flags.go:64] FLAG: --max-pods="110"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137684    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137689    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137694    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137699    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137704    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137708    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 22:13:49.147125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137713    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137740    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137745    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137750    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137755    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137759    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137767    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137772    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137778    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137783    2556 flags.go:64] FLAG: --port="10250"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137788    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137794    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02ada1ecd45c784ad"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137799    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137803    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137808    2556 flags.go:64] FLAG: --register-node="true"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137813    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137819    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137824    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137829    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137833    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137840    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137845    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137850    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137854    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137859    2556 flags.go:64] FLAG: --runonce="false"
Jul 16 22:13:49.148147 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137864    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137869    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137874    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137879    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137884    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137889    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137893    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137898    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137903    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137908    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137912    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137917    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137922    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137927    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137932    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137941    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137945    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137950    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137956    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137961    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137966    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137970    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137975    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137980    2556 flags.go:64] FLAG: --v="2"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137986    2556 flags.go:64] FLAG: --version="false"
Jul 16 22:13:49.149102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137992    2556 flags.go:64] FLAG: --vmodule=""
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.137998    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.138003    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138210    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138217    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138223    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138227    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138232    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138237    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138242    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138246    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138251    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138256    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138261    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138265    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138269    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138274    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138278    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138282    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138304    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.150009 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138308    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138312    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138316    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138321    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138327    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138332    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138336    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138341    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138346    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138350    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138354    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138359    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138363    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138368    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138374    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138380    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138384    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138389    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138393    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.150644 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138397    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138404    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138408    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138412    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138416    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138420    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138425    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138430    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138434    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138438    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138442    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138446    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138450    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138456    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138463    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138470    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138474    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138479    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138483    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.151141 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138487    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138491    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138503    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138507    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138511    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138516    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138520    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138524    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138528    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138532    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138536    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138540    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138545    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138549    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138555    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138559    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138563    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138567    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138572    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138576    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.151751 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138580    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138585    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138589    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138594    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138598    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138602    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138606    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138610    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138616    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138621    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138625    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138629    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138633    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138637    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138642    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138646    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138651    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138655    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138659    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138664    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.152262 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138668    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138672    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138676    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.138680    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.138688    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.146384    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.146402    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.146421    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.146430    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146525    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146531    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146536    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146541    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146546    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146550    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.152833 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146554    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146558    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146562    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146566    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146570    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146575    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146580    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146584    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146588    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146592    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146596    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146601    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146605    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146609    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146613    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146617    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146621    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146626    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146630    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146635    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.153235 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146639    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146643    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146648    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146657    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146662    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146666    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146671    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146676    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146680    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146684    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146688    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146693    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146697    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146701    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146708    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146714    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146719    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146724    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146729    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.153819 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146734    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146738    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146742    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146746    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146751    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146755    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146759    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146763    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146767    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146771    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146775    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146779    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146784    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146788    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146792    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146797    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146801    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146808    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146813    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.154418 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146817    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146821    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146825    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146829    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146834    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146838    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146842    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146846    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146850    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146855    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146859    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146863    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146867    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146872    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146876    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146880    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146884    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146888    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146892    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146897    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.155065 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146901    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146905    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146909    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146913    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146917    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146921    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146927    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146931    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146935    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146939    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146944    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146948    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146955    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146960    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.146965    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.155596 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.146973    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147166    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147176    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147180    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147185    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147190    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147195    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147199    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147204    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147208    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147213    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147217    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147221    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147227    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147231    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147235    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147239    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147243    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147247    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147251    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 22:13:49.156030 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147258    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147263    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147267    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147272    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147277    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147281    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147313    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147317    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147321    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147326    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147330    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147333    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147337    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147342    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147346    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147351    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147355    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147359    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147362    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147366    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 22:13:49.156523 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147370    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147374    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147377    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147381    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147385    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147389    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147393    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147397    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147401    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147405    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147409    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147413    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147417    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147421    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147425    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147432    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147436    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147440    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147444    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 22:13:49.157233 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147449    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147455    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147459    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147465    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147470    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147474    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147478    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147483    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147487    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147491    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147495    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147502    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147508    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147513    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147518    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147523    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147528    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147533    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147538    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 22:13:49.158042 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147542    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147546    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147551    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147555    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147559    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147563    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147567    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147571    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147575    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147581    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147585    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147589    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147594    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147598    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147603    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147607    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147612    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147617    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147621    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147626    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 22:13:49.158689 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147630    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:49.147634    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.147642    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.147811    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.150261    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.151598    2556 server.go:1019] "Starting client certificate rotation"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.151698    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.151733    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.154878    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:13:49.159318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:49.158317    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 22:13:50.630931 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.630911    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 16 22:13:50.633563 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.633544    2556 log.go:25] "Validated CRI v1 image API"
Jul 16 22:13:50.635812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.635794    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 22:13:50.637988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.637969    2556 fs.go:135] Filesystem UUIDs: map[3d7a0af2-79fc-4ef0-a4b8-bfe169fcd67c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7f964cc1-7a9e-434f-bfd5-ce72e3c20f54:/dev/nvme0n1p4]
Jul 16 22:13:50.638053 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.637987    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 22:13:50.641227 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.641210    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:13:50.643234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.643123    2556 manager.go:216] Machine: {Timestamp:2026-07-16 22:13:50.641839533 +0000 UTC m=+0.336421377 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3062411 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec242e0f430b652b446783b526bb5150 SystemUUID:ec242e0f-430b-652b-4467-83b526bb5150 BootID:b15c51cb-4130-4051-8a95-14e5c8c7c0a9 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:07:0d:47:d7:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:07:0d:47:d7:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:85:55:d9:a3:53 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 22:13:50.643234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.643233    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 22:13:50.643344 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.643317    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 22:13:50.646434 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.646407    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 22:13:50.646560 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.646436    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-196.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 22:13:50.647153 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.647143    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 22:13:50.647192 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.647155    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 22:13:50.647192 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.647163    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:13:50.647962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.647952    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 22:13:50.649143 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.649133    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 22:13:50.652470 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652454    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:13:50.652767 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652756    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 22:13:50.652803 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652790    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 22:13:50.652803 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652799    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 22:13:50.652864 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652815    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 22:13:50.652864 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652826    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 16 22:13:50.652864 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.652836    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 22:13:50.653216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.653204    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:13:50.653286 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.653225    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 22:13:50.655269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.655246    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 22:13:50.656952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.656938    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 22:13:50.657006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.656969    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 22:13:50.658417 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658404    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 22:13:50.658451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658428    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 22:13:50.658451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658438    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 22:13:50.658451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658446    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658459    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658468    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658476    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658481    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658488    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658494    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658502    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 22:13:50.658536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.658507    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 22:13:50.659542 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.659522    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 22:13:50.659579 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.659556    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 22:13:50.659829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.659818    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 22:13:50.659865 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.659831    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 22:13:50.662947 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.662935    2556 server.go:1268] "Started kubelet"
Jul 16 22:13:50.663050 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.663018    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 22:13:50.663163 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.663069    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 22:13:50.663163 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.663057    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 22:13:50.666851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.666832    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 22:13:50.666956 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.666909    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-196.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 22:13:50.668404 ip-10-0-133-196 systemd[1]: Started Kubernetes Kubelet.
Jul 16 22:13:50.669402 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.669376    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 22:13:50.669937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.669911    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 22:13:50.674009 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.673044    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-196.ec2.internal.18c2e4dac5b1d87c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-196.ec2.internal,UID:ip-10-0-133-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-196.ec2.internal,},FirstTimestamp:2026-07-16 22:13:50.662908028 +0000 UTC m=+0.357489872,LastTimestamp:2026-07-16 22:13:50.662908028 +0000 UTC m=+0.357489872,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-196.ec2.internal,}"
Jul 16 22:13:50.674531 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.674516    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 22:13:50.674593 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.674514    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 22:13:50.674648 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.674622    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:50.675055 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675016    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 22:13:50.675055 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675022    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 22:13:50.675055 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675039    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 22:13:50.675238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675158    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 22:13:50.675238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675166    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 22:13:50.675366 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675350    2556 factory.go:55] Registering systemd factory
Jul 16 22:13:50.675426 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675393    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 16 22:13:50.675616 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675602    2556 factory.go:153] Registering CRI-O factory
Jul 16 22:13:50.675675 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675621    2556 factory.go:223] Registration of the crio container factory successfully
Jul 16 22:13:50.675710 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675675    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 22:13:50.675710 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675699    2556 factory.go:103] Registering Raw factory
Jul 16 22:13:50.675776 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.675725    2556 manager.go:1196] Started watching for new ooms in manager
Jul 16 22:13:50.676454 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.676436    2556 manager.go:349] Starting recovery of all containers
Jul 16 22:13:50.678616 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.678588    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 22:13:50.678712 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.678694    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-196.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 22:13:50.680288 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.680265    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msf46"
Jul 16 22:13:50.687270 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.687012    2556 manager.go:354] Recovery completed
Jul 16 22:13:50.688396 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.688377    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-msf46"
Jul 16 22:13:50.691802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.691786    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.695797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.695779    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.695874 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.695812    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.695874 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.695824    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.696437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.696424    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 22:13:50.696497 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.696454    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.696838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.696827    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 22:13:50.696871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.696846    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 22:13:50.697310 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.697294    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.697368 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.697329    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.697368 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.697345    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.698003 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.697990    2556 policy_none.go:49] "None policy: Start"
Jul 16 22:13:50.698054 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.698018    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 22:13:50.698054 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.698030    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 22:13:50.698129 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.698017    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-196.ec2.internal.18c2e4dac7a7b341  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-196.ec2.internal,UID:ip-10-0-133-196.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-196.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-196.ec2.internal,},FirstTimestamp:2026-07-16 22:13:50.695797569 +0000 UTC m=+0.390379418,LastTimestamp:2026-07-16 22:13:50.695797569 +0000 UTC m=+0.390379418,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-196.ec2.internal,}"
Jul 16 22:13:50.699060 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.699032    2556 policy_none.go:47] "Start"
Jul 16 22:13:50.742687 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.742665    2556 manager.go:337] "Starting Device Plugin manager"
Jul 16 22:13:50.742776 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.742708    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 22:13:50.742776 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.742721    2556 server.go:87] "Starting device plugin registration server"
Jul 16 22:13:50.742998 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.742985    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 22:13:50.743032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.743000    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 22:13:50.743157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.743131    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 22:13:50.743263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.743228    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 22:13:50.743263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.743238    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 22:13:50.743742 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.743721    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 22:13:50.743821 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.743785    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:50.821538 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.821504    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 22:13:50.822748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.822729    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 22:13:50.822837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.822767    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 22:13:50.822837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.822791    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 22:13:50.822837 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.822828    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 22:13:50.825466 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.825450    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:13:50.843449 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.843366    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.844402 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.844383    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.844485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.844416    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.844485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.844427    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.844485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.844451    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.850944 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.850922    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.851058 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.850951    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-196.ec2.internal\": node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:50.861296 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.861273    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:50.923918 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.923876    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"]
Jul 16 22:13:50.924081 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.923973    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.925053 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.925034    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.925160 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.925069    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.925160 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.925084    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.926238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.926225    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.926348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.926333    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.926383 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.926366    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.927052 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927039    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.927128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927054    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.927128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927067    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.927128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927077    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.927216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927078    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.927216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.927188    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.928163 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.928151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.928216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.928176    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 22:13:50.929880 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.929865    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 22:13:50.929942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.929894    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 22:13:50.929942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:50.929908    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeHasSufficientPID"
Jul 16 22:13:50.956845 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.956820    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-196.ec2.internal\" not found" node="ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.961148 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.961130    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-196.ec2.internal\" not found" node="ip-10-0-133-196.ec2.internal"
Jul 16 22:13:50.962120 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:50.962105    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.062414 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.062384    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.076878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.076849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.077042 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.076883    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.077042 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.076909    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94f9424feb20d8f7b53d0fbc4a5441ee-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"94f9424feb20d8f7b53d0fbc4a5441ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.163379 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.163288    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.177631 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177604    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.177735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.177735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94f9424feb20d8f7b53d0fbc4a5441ee-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"94f9424feb20d8f7b53d0fbc4a5441ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.177735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/94f9424feb20d8f7b53d0fbc4a5441ee-config\") pod \"kube-apiserver-proxy-ip-10-0-133-196.ec2.internal\" (UID: \"94f9424feb20d8f7b53d0fbc4a5441ee\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.177833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177724    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.177833 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.177716    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc92cae0202c6293dc0e89c9e7409eef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal\" (UID: \"dc92cae0202c6293dc0e89c9e7409eef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.259820 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.259778    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.264085 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.264009    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.264085 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.264037    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:51.364718 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.364668    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.465322 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.465238    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.565751 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.565706    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.610269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.610236    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 22:13:51.610872 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.610437    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 22:13:51.666728 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.666689    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.692681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.692626    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 22:08:50 +0000 UTC" deadline="2028-04-14 11:15:17.480051328 +0000 UTC"
Jul 16 22:13:51.692681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.692675    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15301h1m25.787380303s"
Jul 16 22:13:51.766964 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.766898    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.794784 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:51.794753    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f9424feb20d8f7b53d0fbc4a5441ee.slice/crio-756b33e857934c5a0d7a2495f955bebc4c3093c04787631acda299935fee9e05 WatchSource:0}: Error finding container 756b33e857934c5a0d7a2495f955bebc4c3093c04787631acda299935fee9e05: Status 404 returned error can't find the container with id 756b33e857934c5a0d7a2495f955bebc4c3093c04787631acda299935fee9e05
Jul 16 22:13:51.795138 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:51.795118    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc92cae0202c6293dc0e89c9e7409eef.slice/crio-5b204e562761499025e6ad3bbf8ba65219e9375d3851a4e2c3c204d91e890578 WatchSource:0}: Error finding container 5b204e562761499025e6ad3bbf8ba65219e9375d3851a4e2c3c204d91e890578: Status 404 returned error can't find the container with id 5b204e562761499025e6ad3bbf8ba65219e9375d3851a4e2c3c204d91e890578
Jul 16 22:13:51.800618 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.800055    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:13:51.825885 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.825839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" event={"ID":"94f9424feb20d8f7b53d0fbc4a5441ee","Type":"ContainerStarted","Data":"756b33e857934c5a0d7a2495f955bebc4c3093c04787631acda299935fee9e05"}
Jul 16 22:13:51.826786 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:51.826765    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"dc92cae0202c6293dc0e89c9e7409eef","Type":"ContainerStarted","Data":"5b204e562761499025e6ad3bbf8ba65219e9375d3851a4e2c3c204d91e890578"}
Jul 16 22:13:51.867953 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.867911    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:51.968425 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:51.968390    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:52.068941 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.068867    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-196.ec2.internal\" not found"
Jul 16 22:13:52.132604 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.132576    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:13:52.162265 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.162235    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:13:52.175628 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.175601    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:52.187196 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.187169    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:13:52.188177 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.188160    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal"
Jul 16 22:13:52.196947 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.196931    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 22:13:52.204821 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.204798    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:13:52.654180 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.654152    2556 apiserver.go:52] "Watching apiserver"
Jul 16 22:13:52.662440 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.662410    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 22:13:52.664279 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.664239    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-rjzn6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal","openshift-multus/multus-8zhn4","openshift-multus/multus-additional-cni-plugins-4zxq5","openshift-multus/network-metrics-daemon-vxtv5","openshift-network-operator/iptables-alerter-6z96p","kube-system/global-pull-secret-syncer-p4d5g","kube-system/konnectivity-agent-fszkn","openshift-dns/node-resolver-db84x","openshift-image-registry/node-ca-7bngx","openshift-network-diagnostics/network-check-target-zfg42","openshift-ovn-kubernetes/ovnkube-node-gsf85","kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"]
Jul 16 22:13:52.665794 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.665775    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.667578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.667556    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 22:13:52.667672 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.667595    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.667672 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.667608    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.667794 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.667782    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:52.667845 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.667794    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-25dc4\""
Jul 16 22:13:52.667901 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.667851    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:13:52.669369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.669350    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.670109 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.670077    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.671151 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.670983    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.671151 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671005    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.671151 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671005    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 22:13:52.671333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671157    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xmdp5\""
Jul 16 22:13:52.671952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671580    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 22:13:52.671952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671682    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 22:13:52.671952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.671837    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 22:13:52.672400 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.672244    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rlxgn\""
Jul 16 22:13:52.674049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.674031    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.674155 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.674110    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:13:52.675706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.675504    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 22:13:52.676442 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.676172    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.677462 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.677433    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.678035 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.678013    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dpdxv\""
Jul 16 22:13:52.678298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.678282    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.678770 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.678746    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 22:13:52.679006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.678987    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.679619 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.679600    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.680283 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.680262    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.680486 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.680469    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cwl7z\""
Jul 16 22:13:52.680680 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.680663    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.680919 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.680901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.681505 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.681488    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.681906 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.681886    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.682653 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.682426    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mvgh7\""
Jul 16 22:13:52.682653 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.682436    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 22:13:52.682795 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.682676    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.683475 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.683211    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5jmd6\""
Jul 16 22:13:52.683475 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.683323    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 22:13:52.683620 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.683515    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 22:13:52.684413 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.684138    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.684413 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.684302    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.685627 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.685426    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.685712 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.685655    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-snqzb\""
Jul 16 22:13:52.686004 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.685988    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.686170 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.686150    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:13:52.686250 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686163    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2pl7\" (UniqueName: \"kubernetes.io/projected/a017371e-d257-4773-bf5d-4af39c11965f-kube-api-access-v2pl7\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686250 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686215    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-system-cni-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.686346 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cnibin\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.686346 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686283    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.686346 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686322    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vprb\" (UniqueName: \"kubernetes.io/projected/6ad72e05-81f9-449a-8f6f-a552cdbe9748-kube-api-access-8vprb\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.686467 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686366    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-iptables-alerter-script\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.686467 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686467 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686432    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-bin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686462    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-cnibin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686501    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-k8s-cni-cncf-io\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686524    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-multus\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686588    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-multus-daemon-config\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686612    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-etc-kubernetes\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686644    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 22:13:52.686780 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686750    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 22:13:52.686820 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686646    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkp86\" (UniqueName: \"kubernetes.io/projected/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-kube-api-access-mkp86\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.686820 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686805    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-os-release\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.686900 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686844    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.686900 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686869    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ad72e05-81f9-449a-8f6f-a552cdbe9748-hosts-file\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.686900 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686892    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:52.687000 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-cni-binary-copy\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687000 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686961    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.687000 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686980    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 22:13:52.687158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687023    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 22:13:52.687158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.686983    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-host-slash\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.687256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687195    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 22:13:52.687306 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-socket-dir-parent\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687306 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687282    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-conf-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687306 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687287    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wkd68\""
Jul 16 22:13:52.687446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687330    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.687446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hkc\" (UniqueName: \"kubernetes.io/projected/889fca95-1e44-443e-87f4-dbf482146ac6-kube-api-access-p8hkc\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.687446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3065d8c-65a8-4b78-b859-3fd67a18a667-serviceca\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.687446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687423    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59nc2\" (UniqueName: \"kubernetes.io/projected/f3065d8c-65a8-4b78-b859-3fd67a18a667-kube-api-access-59nc2\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687446    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-netns\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687470    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-binary-copy\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687496    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szz7w\" (UniqueName: \"kubernetes.io/projected/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-kube-api-access-szz7w\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687518    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ad72e05-81f9-449a-8f6f-a552cdbe9748-tmp-dir\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687584    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3065d8c-65a8-4b78-b859-3fd67a18a667-host\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687605    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-system-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687628    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-hostroot\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-os-release\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.687965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-kubelet\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.687965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.687752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-multus-certs\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.688209 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.688189    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 22:13:52.688428 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.688409    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 22:13:52.707921 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.707896    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wdhxw"
Jul 16 22:13:52.730268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.730244    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wdhxw"
Jul 16 22:13:52.776843 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.776815    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 22:13:52.779175 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.779149    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 22:13:52.788705 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788675    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-etc-kubernetes\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.788815 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkp86\" (UniqueName: \"kubernetes.io/projected/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-kube-api-access-mkp86\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.788815 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788751    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-node-log\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.788815 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.788815 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788807    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ad72e05-81f9-449a-8f6f-a552cdbe9748-hosts-file\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.788938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.788938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788857    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.788938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-dbus\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.788938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788909    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-etc-kubernetes\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788912    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-cni-binary-copy\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.788975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.789067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-kubelet\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789034    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-slash\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-netns\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789080    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-var-lib-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-socket-dir-parent\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789177    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hkc\" (UniqueName: \"kubernetes.io/projected/889fca95-1e44-443e-87f4-dbf482146ac6-kube-api-access-p8hkc\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-log-socket\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.789213    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789222    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-netd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59nc2\" (UniqueName: \"kubernetes.io/projected/f3065d8c-65a8-4b78-b859-3fd67a18a667-kube-api-access-59nc2\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789277    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-netns\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789301    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-binary-copy\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789326    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szz7w\" (UniqueName: \"kubernetes.io/projected/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-kube-api-access-szz7w\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.789336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6ad72e05-81f9-449a-8f6f-a552cdbe9748-hosts-file\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789350    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ad72e05-81f9-449a-8f6f-a552cdbe9748-tmp-dir\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-os-release\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789400    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-etc-selinux\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789422    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-systemd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-bin\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789472    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2pl7\" (UniqueName: \"kubernetes.io/projected/a017371e-d257-4773-bf5d-4af39c11965f-kube-api-access-v2pl7\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-iptables-alerter-script\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqf4\" (UniqueName: \"kubernetes.io/projected/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kube-api-access-dtqf4\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789567    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-ovn\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789581    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-cni-binary-copy\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789600    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-config\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789649    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-bin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789672    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-systemd\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789686    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-netns\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.789971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.789697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-socket-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.789725    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:53.289703797 +0000 UTC m=+2.984285633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790004    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-os-release\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ad72e05-81f9-449a-8f6f-a552cdbe9748-tmp-dir\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790279    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-socket-dir-parent\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-sys-fs\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790416    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-cnibin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790441    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-k8s-cni-cncf-io\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790465    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-multus\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-multus-daemon-config\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790518    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-agent-certs\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790526    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-iptables-alerter-script\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.790549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-konnectivity-ca\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790566    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-k8s-cni-cncf-io\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790575    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-multus\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790571    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790526    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-cnibin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790620    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-cni-bin\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-os-release\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790715    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-conf\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790755    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-tuned\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790808    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-os-release\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790812    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-registration-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.790951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790844    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-kubelet-config\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.790920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-host-slash\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791150    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-host-slash\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791161    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysconfig\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791022    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cni-binary-copy\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791189    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-conf-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791214    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-sys\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3065d8c-65a8-4b78-b859-3fd67a18a667-serviceca\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-lib-modules\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791256    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-multus-conf-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791278    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-var-lib-kubelet\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791378    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-tmp\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791412    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791438    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-env-overrides\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791467    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85sg9\" (UniqueName: \"kubernetes.io/projected/b621fc12-995c-44e1-ba45-99f3c0a45643-kube-api-access-85sg9\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3065d8c-65a8-4b78-b859-3fd67a18a667-host\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-system-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.791748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-hostroot\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-modprobe-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791640    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-run\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791648    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a017371e-d257-4773-bf5d-4af39c11965f-multus-daemon-config\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791661    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-etc-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791683    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f3065d8c-65a8-4b78-b859-3fd67a18a667-serviceca\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3065d8c-65a8-4b78-b859-3fd67a18a667-host\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791727    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-system-cni-dir\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791730    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-script-lib\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791743    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-hostroot\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-kubelet\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-var-lib-kubelet\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791814    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-multus-certs\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791854    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-system-cni-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a017371e-d257-4773-bf5d-4af39c11965f-host-run-multus-certs\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cnibin\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.792566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791934    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-system-cni-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791947    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.791995    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-cnibin\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vprb\" (UniqueName: \"kubernetes.io/projected/6ad72e05-81f9-449a-8f6f-a552cdbe9748-kube-api-access-8vprb\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-host\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792133    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-systemd-units\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-kubernetes\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792169    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792205    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf48q\" (UniqueName: \"kubernetes.io/projected/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-kube-api-access-vf48q\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792241    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-device-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.793388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.792270    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b621fc12-995c-44e1-ba45-99f3c0a45643-ovn-node-metrics-cert\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.801968 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.801583    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:13:52.801968 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.801606    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:13:52.801968 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.801621    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:52.801968 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.801586    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 22:13:52.801968 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.801682    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:53.301663512 +0000 UTC m=+2.996245364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:52.805792 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.805764    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2pl7\" (UniqueName: \"kubernetes.io/projected/a017371e-d257-4773-bf5d-4af39c11965f-kube-api-access-v2pl7\") pod \"multus-8zhn4\" (UID: \"a017371e-d257-4773-bf5d-4af39c11965f\") " pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:52.805903 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.805765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vprb\" (UniqueName: \"kubernetes.io/projected/6ad72e05-81f9-449a-8f6f-a552cdbe9748-kube-api-access-8vprb\") pod \"node-resolver-db84x\" (UID: \"6ad72e05-81f9-449a-8f6f-a552cdbe9748\") " pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:52.806063 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.806040    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkp86\" (UniqueName: \"kubernetes.io/projected/573654d3-237c-4a61-b89c-0fd4a7f9d6cd-kube-api-access-mkp86\") pod \"iptables-alerter-6z96p\" (UID: \"573654d3-237c-4a61-b89c-0fd4a7f9d6cd\") " pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:52.806501 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.806480    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szz7w\" (UniqueName: \"kubernetes.io/projected/89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b-kube-api-access-szz7w\") pod \"multus-additional-cni-plugins-4zxq5\" (UID: \"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b\") " pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:52.806570 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.806483    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59nc2\" (UniqueName: \"kubernetes.io/projected/f3065d8c-65a8-4b78-b859-3fd67a18a667-kube-api-access-59nc2\") pod \"node-ca-7bngx\" (UID: \"f3065d8c-65a8-4b78-b859-3fd67a18a667\") " pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.807425 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.807403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hkc\" (UniqueName: \"kubernetes.io/projected/889fca95-1e44-443e-87f4-dbf482146ac6-kube-api-access-p8hkc\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:52.892588 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892557    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-sys\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892588 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-lib-modules\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892623    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-var-lib-kubelet\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-tmp\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892697    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-sys\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892702    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-var-lib-kubelet\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892773    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-lib-modules\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.892808 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-env-overrides\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85sg9\" (UniqueName: \"kubernetes.io/projected/b621fc12-995c-44e1-ba45-99f3c0a45643-kube-api-access-85sg9\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892842    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892871    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-modprobe-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892922    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-run\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892943    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-etc-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892966    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-script-lib\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.892993    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-host\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-systemd-units\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893061    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-run\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893067    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-kubernetes\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893107    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf48q\" (UniqueName: \"kubernetes.io/projected/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-kube-api-access-vf48q\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893169 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-etc-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-device-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893218    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b621fc12-995c-44e1-ba45-99f3c0a45643-ovn-node-metrics-cert\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-node-log\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893288    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-env-overrides\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893299    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893356    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-kubernetes\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-dbus\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-modprobe-d\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-kubelet\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893478    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-slash\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893483    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-netns\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-systemd-units\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.893761 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-var-lib-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-host\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-slash\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893590    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-log-socket\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893602    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-kubelet\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-netd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893621    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-var-lib-openvswitch\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-etc-selinux\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893611    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-dbus\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-systemd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894129    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-device-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.893636    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-netns\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894327    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-bin\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894158    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-log-socket\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-bin\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.894522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-cni-netd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894591    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-systemd\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqf4\" (UniqueName: \"kubernetes.io/projected/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kube-api-access-dtqf4\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-ovn\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894831    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-node-log\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894926    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-config\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.894992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-run-ovn\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895024    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-script-lib\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895038    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-systemd\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895047    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-etc-selinux\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-socket-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.895173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-systemd\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-sys-fs\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-sys-fs\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895346    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-agent-certs\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-konnectivity-ca\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895432    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895461    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895594    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b621fc12-995c-44e1-ba45-99f3c0a45643-ovnkube-config\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.895694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895657    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-tmp\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.896020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895699    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-socket-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.896020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895752    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b621fc12-995c-44e1-ba45-99f3c0a45643-host-run-ovn-kubernetes\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.896020 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.895801    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:52.896020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.895848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-conf\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.896020 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:52.895917    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:53.395892162 +0000 UTC m=+3.090474017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:52.896264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896172    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-tuned\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.896264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896207    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-registration-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.896350 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896280    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-kubelet-config\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.896350 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896307    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysconfig\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.896438 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896373    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f13b359c-19ed-410b-9eaf-58148f522ff7-kubelet-config\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:52.896438 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896384    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysctl-conf\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.896526 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b621fc12-995c-44e1-ba45-99f3c0a45643-ovn-node-metrics-cert\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.896526 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6cf1860-e750-41b7-aff2-db9cb7ed008e-registration-dir\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.896526 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.896457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-sysconfig\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.898290 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.897190    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-konnectivity-ca\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.900053 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.900030    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9be0f3d4-96b6-4874-a4f7-8d9cffbc254a-agent-certs\") pod \"konnectivity-agent-fszkn\" (UID: \"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a\") " pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:52.900516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.900491    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-etc-tuned\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.902558 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.902505    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf48q\" (UniqueName: \"kubernetes.io/projected/f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6-kube-api-access-vf48q\") pod \"tuned-rjzn6\" (UID: \"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6\") " pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:52.903144 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.903125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqf4\" (UniqueName: \"kubernetes.io/projected/f6cf1860-e750-41b7-aff2-db9cb7ed008e-kube-api-access-dtqf4\") pod \"aws-ebs-csi-driver-node-92pbn\" (UID: \"f6cf1860-e750-41b7-aff2-db9cb7ed008e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:52.903399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.903378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85sg9\" (UniqueName: \"kubernetes.io/projected/b621fc12-995c-44e1-ba45-99f3c0a45643-kube-api-access-85sg9\") pod \"ovnkube-node-gsf85\" (UID: \"b621fc12-995c-44e1-ba45-99f3c0a45643\") " pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:52.979858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.979789    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7bngx"
Jul 16 22:13:52.990539 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:52.990514    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8zhn4"
Jul 16 22:13:53.010277 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.010244    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4zxq5"
Jul 16 22:13:53.016759 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.016739    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6z96p"
Jul 16 22:13:53.026305 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.026286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-db84x"
Jul 16 22:13:53.032858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.032834    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn"
Jul 16 22:13:53.042602 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.042576    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:13:53.049384 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.049358    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rjzn6"
Jul 16 22:13:53.056891 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.056872    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:13:53.299646 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.299581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:53.299776 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.299696    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:53.299776 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.299770    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:54.299748053 +0000 UTC m=+3.994329886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:53.400233 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.400193    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:53.400233 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.400236    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:53.400447 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400359    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:53.400447 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400373    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:13:53.400447 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400389    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:13:53.400447 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400402    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:53.400447 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400434    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:54.400410208 +0000 UTC m=+4.094992065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:53.400677 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.400454    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:54.400445992 +0000 UTC m=+4.095027825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:53.733745 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.733662    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:08:52 +0000 UTC" deadline="2028-01-21 08:38:38.635617041 +0000 UTC"
Jul 16 22:13:53.733745 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.733694    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13282h24m44.901927907s"
Jul 16 22:13:53.823344 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.823313    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:53.823344 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:53.823335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:53.823574 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.823442    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:13:53.823574 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:53.823556    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:13:53.961334 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.961242    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be0f3d4_96b6_4874_a4f7_8d9cffbc254a.slice/crio-bf4652381c6c7b0959df95714fe67c6bccd695da9d74e06893b64a289765cd96 WatchSource:0}: Error finding container bf4652381c6c7b0959df95714fe67c6bccd695da9d74e06893b64a289765cd96: Status 404 returned error can't find the container with id bf4652381c6c7b0959df95714fe67c6bccd695da9d74e06893b64a289765cd96
Jul 16 22:13:53.972288 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.972265    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf236335e_7dbd_4de9_94b2_a2bf3e2cd1d6.slice/crio-622aa1730a98f3c37a10607e07e491f5da478230c240245871e34800a4f28fe9 WatchSource:0}: Error finding container 622aa1730a98f3c37a10607e07e491f5da478230c240245871e34800a4f28fe9: Status 404 returned error can't find the container with id 622aa1730a98f3c37a10607e07e491f5da478230c240245871e34800a4f28fe9
Jul 16 22:13:53.974301 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.974183    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb621fc12_995c_44e1_ba45_99f3c0a45643.slice/crio-086236b0b4c39573d4a0a56268c7db5a01409c84d82a428542ecf529268ed1cc WatchSource:0}: Error finding container 086236b0b4c39573d4a0a56268c7db5a01409c84d82a428542ecf529268ed1cc: Status 404 returned error can't find the container with id 086236b0b4c39573d4a0a56268c7db5a01409c84d82a428542ecf529268ed1cc
Jul 16 22:13:53.976366 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.976335    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89c39ba4_5c7f_4b98_a0bf_6d0c2fb5712b.slice/crio-16fd8484fff22a9a140fa72a08767c4e8f06c611bb0f81ffdc33c16bdf74ac2b WatchSource:0}: Error finding container 16fd8484fff22a9a140fa72a08767c4e8f06c611bb0f81ffdc33c16bdf74ac2b: Status 404 returned error can't find the container with id 16fd8484fff22a9a140fa72a08767c4e8f06c611bb0f81ffdc33c16bdf74ac2b
Jul 16 22:13:53.978056 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.977348    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cf1860_e750_41b7_aff2_db9cb7ed008e.slice/crio-a6fb6fecf027963b0d4567a0976ff434c1a7b7141d0b12b1d12d1dbb7c2a3885 WatchSource:0}: Error finding container a6fb6fecf027963b0d4567a0976ff434c1a7b7141d0b12b1d12d1dbb7c2a3885: Status 404 returned error can't find the container with id a6fb6fecf027963b0d4567a0976ff434c1a7b7141d0b12b1d12d1dbb7c2a3885
Jul 16 22:13:53.978264 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.978178    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad72e05_81f9_449a_8f6f_a552cdbe9748.slice/crio-94c5efed6012bfd5d8ca50fc4bc7bdc1b3b6b6edbba39443bd0b5aa48d09baaf WatchSource:0}: Error finding container 94c5efed6012bfd5d8ca50fc4bc7bdc1b3b6b6edbba39443bd0b5aa48d09baaf: Status 404 returned error can't find the container with id 94c5efed6012bfd5d8ca50fc4bc7bdc1b3b6b6edbba39443bd0b5aa48d09baaf
Jul 16 22:13:53.980609 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.980063    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda017371e_d257_4773_bf5d_4af39c11965f.slice/crio-cb513dafa46d79e6acb4d1e48c9fb8356cb296f0df0b930591a3b7059bfe9bac WatchSource:0}: Error finding container cb513dafa46d79e6acb4d1e48c9fb8356cb296f0df0b930591a3b7059bfe9bac: Status 404 returned error can't find the container with id cb513dafa46d79e6acb4d1e48c9fb8356cb296f0df0b930591a3b7059bfe9bac
Jul 16 22:13:53.980724 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.980669    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573654d3_237c_4a61_b89c_0fd4a7f9d6cd.slice/crio-949455f451cd48828e2b649d6151f55c0f8b4cfd9d0be55de33492be2e17596a WatchSource:0}: Error finding container 949455f451cd48828e2b649d6151f55c0f8b4cfd9d0be55de33492be2e17596a: Status 404 returned error can't find the container with id 949455f451cd48828e2b649d6151f55c0f8b4cfd9d0be55de33492be2e17596a
Jul 16 22:13:53.982538 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:13:53.982368    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3065d8c_65a8_4b78_b859_3fd67a18a667.slice/crio-5f4d414972090f0e67ab149096fcde37ae165782092d8a80fe4b44b8ed6fbe21 WatchSource:0}: Error finding container 5f4d414972090f0e67ab149096fcde37ae165782092d8a80fe4b44b8ed6fbe21: Status 404 returned error can't find the container with id 5f4d414972090f0e67ab149096fcde37ae165782092d8a80fe4b44b8ed6fbe21
Jul 16 22:13:54.306173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.305874    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:54.306173 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.306013    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:54.306324 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.306177    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:56.306159826 +0000 UTC m=+6.000741675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:54.406595 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.406541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:54.406595 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.406584    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406689    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406730    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406747    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406759    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406762    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:56.406738598 +0000 UTC m=+6.101320434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:54.406811 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.406807    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:13:56.406791165 +0000 UTC m=+6.101373011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:54.734361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.734290    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 22:08:52 +0000 UTC" deadline="2028-01-03 23:49:55.904587413 +0000 UTC"
Jul 16 22:13:54.734361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.734326    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12865h36m1.170265996s"
Jul 16 22:13:54.824315 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.824284    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:54.833151 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:54.832313    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:13:54.848440 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.848394    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6z96p" event={"ID":"573654d3-237c-4a61-b89c-0fd4a7f9d6cd","Type":"ContainerStarted","Data":"949455f451cd48828e2b649d6151f55c0f8b4cfd9d0be55de33492be2e17596a"}
Jul 16 22:13:54.851844 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.851809    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-db84x" event={"ID":"6ad72e05-81f9-449a-8f6f-a552cdbe9748","Type":"ContainerStarted","Data":"94c5efed6012bfd5d8ca50fc4bc7bdc1b3b6b6edbba39443bd0b5aa48d09baaf"}
Jul 16 22:13:54.860105 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.860055    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rjzn6" event={"ID":"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6","Type":"ContainerStarted","Data":"622aa1730a98f3c37a10607e07e491f5da478230c240245871e34800a4f28fe9"}
Jul 16 22:13:54.875304 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.875210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" event={"ID":"94f9424feb20d8f7b53d0fbc4a5441ee","Type":"ContainerStarted","Data":"c3ae2cd5f76be1228d5a95c982ba4685cc0aad14dc3f6250abe40d029b1b2f4b"}
Jul 16 22:13:54.883057 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.883024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8zhn4" event={"ID":"a017371e-d257-4773-bf5d-4af39c11965f","Type":"ContainerStarted","Data":"cb513dafa46d79e6acb4d1e48c9fb8356cb296f0df0b930591a3b7059bfe9bac"}
Jul 16 22:13:54.897547 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.897415    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"086236b0b4c39573d4a0a56268c7db5a01409c84d82a428542ecf529268ed1cc"}
Jul 16 22:13:54.906586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.906552    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn" event={"ID":"f6cf1860-e750-41b7-aff2-db9cb7ed008e","Type":"ContainerStarted","Data":"a6fb6fecf027963b0d4567a0976ff434c1a7b7141d0b12b1d12d1dbb7c2a3885"}
Jul 16 22:13:54.914692 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.914602    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerStarted","Data":"16fd8484fff22a9a140fa72a08767c4e8f06c611bb0f81ffdc33c16bdf74ac2b"}
Jul 16 22:13:54.923566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.923490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fszkn" event={"ID":"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a","Type":"ContainerStarted","Data":"bf4652381c6c7b0959df95714fe67c6bccd695da9d74e06893b64a289765cd96"}
Jul 16 22:13:54.938883 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.938321    2556 generic.go:356] "Generic (PLEG): container finished" podID="dc92cae0202c6293dc0e89c9e7409eef" containerID="5dde84a5662e8776ac3dd698e11d4e8beabd54825e53dfb8713175e4d497bae6" exitCode=0
Jul 16 22:13:54.938883 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.938413    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"dc92cae0202c6293dc0e89c9e7409eef","Type":"ContainerDied","Data":"5dde84a5662e8776ac3dd698e11d4e8beabd54825e53dfb8713175e4d497bae6"}
Jul 16 22:13:54.954986 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.954929    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-196.ec2.internal" podStartSLOduration=2.954908918 podStartE2EDuration="2.954908918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:13:54.890424883 +0000 UTC m=+4.585006739" watchObservedRunningTime="2026-07-16 22:13:54.954908918 +0000 UTC m=+4.649490773"
Jul 16 22:13:54.958122 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:54.958074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bngx" event={"ID":"f3065d8c-65a8-4b78-b859-3fd67a18a667","Type":"ContainerStarted","Data":"5f4d414972090f0e67ab149096fcde37ae165782092d8a80fe4b44b8ed6fbe21"}
Jul 16 22:13:55.824032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:55.823359    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:55.824032 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:55.823505    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:13:55.824032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:55.823886    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:55.824032 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:55.823982    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:13:55.975965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:55.975928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" event={"ID":"dc92cae0202c6293dc0e89c9e7409eef","Type":"ContainerStarted","Data":"713b26a2efad1ae0617e79538b28eefb13104401e4ac730d6be7d5e5aeef7fef"}
Jul 16 22:13:56.321691 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:56.321605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:56.321897 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.321878    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:56.321986 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.321973    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:00.321954527 +0000 UTC m=+10.016536382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:56.423031    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:56.423080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423232    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423253    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423267    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423358    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:00.423308428 +0000 UTC m=+10.117890274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423757    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:56.423840 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.423804    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:00.423789824 +0000 UTC m=+10.118371671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:13:56.825363 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:56.825333    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:56.825809 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:56.825465    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:13:57.824005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:57.823975    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:57.824219 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:57.824139    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:13:57.824219 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:57.823978    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:57.824358 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:57.824302    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:13:58.823647 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:58.823608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:13:58.824126 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:58.823735    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:13:59.823423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:59.823317    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:13:59.823423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:13:59.823363    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:13:59.823671 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:59.823459    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:13:59.824047 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:13:59.823886    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:00.354152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:00.354039    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:00.354337 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.354201    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:00.354337 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.354268    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:08.35424861 +0000 UTC m=+18.048830456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:00.454653 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:00.454566    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:00.454653 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:00.454623    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454726    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454797    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:08.454778543 +0000 UTC m=+18.149360395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454823    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454847    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454859    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:00.455026 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.454908    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:08.45489242 +0000 UTC m=+18.149474254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:00.825671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:00.825120    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:00.825671 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:00.825260    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:01.823598 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:01.823027    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:01.823598 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:01.823059    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:01.823598 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:01.823217    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:01.823598 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:01.823328    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:02.826190 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:02.824782    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:02.826190 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:02.824919    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:03.823496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:03.823457    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:03.823496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:03.823496    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:03.823723 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:03.823602    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:03.823723 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:03.823708    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:04.824004 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:04.823910    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:04.824474 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:04.824044    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:05.823593 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:05.823552    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:05.823769 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:05.823574    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:05.823769 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:05.823699    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:05.823857 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:05.823760    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:06.823875 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:06.823839    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:06.824363 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:06.823954    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:07.823637 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:07.823606    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:07.825857 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:07.824037    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:07.825857 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:07.824057    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:07.825857 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:07.824199    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:08.413831 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:08.413792    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:08.414033 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.413969    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:08.414127 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.414036    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:24.414016589 +0000 UTC m=+34.108598421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:08.514263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:08.514226    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:08.514423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:08.514271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:08.514423 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514390    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:08.514540 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514431    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:14:08.514540 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514452    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:14:08.514540 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514464    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:08.514540 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514476    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:24.514452412 +0000 UTC m=+34.209034267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:08.514540 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.514516    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:24.514497584 +0000 UTC m=+34.209079429 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:08.824018 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:08.823949    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:08.824161 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:08.824078    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:09.823179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:09.823142    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:09.823566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:09.823151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:09.823566 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:09.823282    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:09.823566 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:09.823313    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:10.824379 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:10.824346    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:10.824818 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:10.824444    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:11.824036 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:11.823970    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:11.824220 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:11.824112    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:11.824220 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:11.824152    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:11.824339 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:11.824228    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:12.823490 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:12.823193    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:12.824021 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:12.823570    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:13.008558 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.008514    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7bngx" event={"ID":"f3065d8c-65a8-4b78-b859-3fd67a18a667","Type":"ContainerStarted","Data":"8323e1c9c465b1dbc673aa20ce2343fbc0b0be2780d6d3161e5b7fc32bbfe91a"}
Jul 16 22:14:13.011106 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.011059    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-db84x" event={"ID":"6ad72e05-81f9-449a-8f6f-a552cdbe9748","Type":"ContainerStarted","Data":"220cf17432fd21a9eccd15fb01566d5cf89babfee54e4bf7be3063e6e7a3394e"}
Jul 16 22:14:13.015841 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.015813    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rjzn6" event={"ID":"f236335e-7dbd-4de9-94b2-a2bf3e2cd1d6","Type":"ContainerStarted","Data":"10c4027c39b2e7bc20d5ad0db04006b99dbad29c71f67ffdf0e803277e5de27d"}
Jul 16 22:14:13.017101 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.017065    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8zhn4" event={"ID":"a017371e-d257-4773-bf5d-4af39c11965f","Type":"ContainerStarted","Data":"dc1d4344978d7f1ed99ceaee03c603f955700a637e9f5f344619c7f7ecf3409f"}
Jul 16 22:14:13.019173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.019155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"bbe3a730de0d9fd3d45207195c2fee6e600fef3f7e4b2a74663dcce0aed3e93c"}
Jul 16 22:14:13.019269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.019206    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"8828fa77b334d031775d087ffe60c9ff9b8bf473fd16f286d4289886e5f02d8f"}
Jul 16 22:14:13.019269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.019253    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"ac56d35eea6cee8d4a77044df886ac8dec6409b23fdee509237b6d84a4107811"}
Jul 16 22:14:13.019369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.019287    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"039ad31af5e900705fdeb596acfe74b3d6ce5bc601caed50477ec7e6c76ed48c"}
Jul 16 22:14:13.020432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.020416    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn" event={"ID":"f6cf1860-e750-41b7-aff2-db9cb7ed008e","Type":"ContainerStarted","Data":"73bdd9d1748901d675d14ae1d74061c95ab844d02ede1af0cf0b20ad9ecd884f"}
Jul 16 22:14:13.021712 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.021690    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="649705b11d67f489beb3a581a1ecd8584552b7a6211131f55e1a8eef2fea4f2f" exitCode=0
Jul 16 22:14:13.021771 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.021750    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"649705b11d67f489beb3a581a1ecd8584552b7a6211131f55e1a8eef2fea4f2f"}
Jul 16 22:14:13.023031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.023011    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fszkn" event={"ID":"9be0f3d4-96b6-4874-a4f7-8d9cffbc254a","Type":"ContainerStarted","Data":"7e8895e2d28c5fbc9cc737c0533848c65a322c9ddfcd553544e6503af2555a52"}
Jul 16 22:14:13.025643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.025603    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-196.ec2.internal" podStartSLOduration=21.025589004 podStartE2EDuration="21.025589004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:13:55.990857514 +0000 UTC m=+5.685439368" watchObservedRunningTime="2026-07-16 22:14:13.025589004 +0000 UTC m=+22.720170854"
Jul 16 22:14:13.026064 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.026038    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7bngx" podStartSLOduration=4.53575285 podStartE2EDuration="23.026029932s" totalImagesPullingTime="18.490277082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:13.025688201 +0000 UTC m=+22.720270059" watchObservedRunningTime="2026-07-16 22:14:13.026029932 +0000 UTC m=+22.720611786"
Jul 16 22:14:13.040128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.040066    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-db84x" podStartSLOduration=4.545252919 podStartE2EDuration="23.040052905s" totalImagesPullingTime="18.494799986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:13.03987288 +0000 UTC m=+22.734454725" watchObservedRunningTime="2026-07-16 22:14:13.040052905 +0000 UTC m=+22.734634759"
Jul 16 22:14:13.071709 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.071668    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fszkn" podStartSLOduration=12.58298226 podStartE2EDuration="23.071652678s" totalImagesPullingTime="10.488670418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:13.0714012 +0000 UTC m=+22.765983054" watchObservedRunningTime="2026-07-16 22:14:13.071652678 +0000 UTC m=+22.766234531"
Jul 16 22:14:13.087158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.087120    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rjzn6" podStartSLOduration=4.585404341 podStartE2EDuration="23.087085617s" totalImagesPullingTime="18.501681276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:13.086798925 +0000 UTC m=+22.781380778" watchObservedRunningTime="2026-07-16 22:14:13.087085617 +0000 UTC m=+22.781667472"
Jul 16 22:14:13.823223 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.823145    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:13.823422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.823145    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:13.823422 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:13.823254    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:13.823422 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:13.823335    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:13.986135 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:13.986110    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 22:14:14.028722 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.028687    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"9f21334bad62bac3af80538b953431b19aa92e37a9f178f8ba032c7d530201fe"}
Jul 16 22:14:14.028913 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.028766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"9ffc7424e32825db3ea4df5b1457c685c520624c8f3237b134fb688c47685e50"}
Jul 16 22:14:14.030678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.030654    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn" event={"ID":"f6cf1860-e750-41b7-aff2-db9cb7ed008e","Type":"ContainerStarted","Data":"830cc2c31ef201e3bb750e8ac68f7f13372daa2b1f4f1fe562d9ffc8811d3496"}
Jul 16 22:14:14.032294 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.032261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6z96p" event={"ID":"573654d3-237c-4a61-b89c-0fd4a7f9d6cd","Type":"ContainerStarted","Data":"728b0446143ccf059ea9cfd859227d371fc23405f038179aca7f8aefa6499a98"}
Jul 16 22:14:14.046773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.046716    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8zhn4" podStartSLOduration=5.48563281 podStartE2EDuration="24.046693911s" totalImagesPullingTime="18.561061101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:13.102008816 +0000 UTC m=+22.796590688" watchObservedRunningTime="2026-07-16 22:14:14.046693911 +0000 UTC m=+23.741275765"
Jul 16 22:14:14.759413 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.759242    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T22:14:13.986127721Z","UUID":"23d5ce62-63de-41d5-955b-0d31ebe5a9b2","Handler":null,"Name":"","Endpoint":""}
Jul 16 22:14:14.761301 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.761277    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 22:14:14.761446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.761326    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 22:14:14.823743 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:14.823707    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:14.824227 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:14.823847    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:15.823450 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:15.823252    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:15.823667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:15.823321    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:15.823667 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:15.823548    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:15.823667 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:15.823606    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:16.039211 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.039177    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"4d56d9b5590d52659ebedd30ab960b8aa8d49e57058e079dff63195140265102"}
Jul 16 22:14:16.041013 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.040991    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn" event={"ID":"f6cf1860-e750-41b7-aff2-db9cb7ed008e","Type":"ContainerStarted","Data":"52e8f90a09604a05463d3de140b2bb201bdbf8c68c29aaba2ebeb5f5d9b5578a"}
Jul 16 22:14:16.057566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.057519    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6z96p" podStartSLOduration=7.567052504 podStartE2EDuration="26.057506268s" totalImagesPullingTime="18.490453764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:14.046153149 +0000 UTC m=+23.740735004" watchObservedRunningTime="2026-07-16 22:14:16.057506268 +0000 UTC m=+25.752088123"
Jul 16 22:14:16.057864 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.057844    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-92pbn" podStartSLOduration=5.196250336 podStartE2EDuration="26.057840075s" totalImagesPullingTime="20.861589739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:16.057214173 +0000 UTC m=+25.751796040" watchObservedRunningTime="2026-07-16 22:14:16.057840075 +0000 UTC m=+25.752421929"
Jul 16 22:14:16.823766 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.823732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:16.824369 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:16.823833    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:16.920435 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.920383    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:14:16.920910 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:16.920880    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:14:17.824184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:17.823906    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:17.824184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:17.823925    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:17.824963 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:17.824206    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:17.824963 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:17.824267    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:18.048425 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.048387    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" event={"ID":"b621fc12-995c-44e1-ba45-99f3c0a45643","Type":"ContainerStarted","Data":"0a52fc57a36aa12f92950d97dd55438837a033f14793320c8c7eb7de16827ff3"}
Jul 16 22:14:18.048591 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.048574    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:18.048679 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.048630    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:18.048679 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.048672    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:18.064222 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.064199    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:18.064422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.064335    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:18.080791 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.080704    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85" podStartSLOduration=9.506403795 podStartE2EDuration="28.080688419s" totalImagesPullingTime="18.574284624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:18.073500669 +0000 UTC m=+27.768082544" watchObservedRunningTime="2026-07-16 22:14:18.080688419 +0000 UTC m=+27.775270279"
Jul 16 22:14:18.823295 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:18.823219    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:18.823444 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:18.823354    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:19.395432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.395396    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:14:19.395854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.395575    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 16 22:14:19.396338 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.396314    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fszkn"
Jul 16 22:14:19.749786 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.749765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4d5g"]
Jul 16 22:14:19.749894 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.749862    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:19.749995 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:19.749967    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:19.754770 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.754659    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zfg42"]
Jul 16 22:14:19.755709 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.755688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:19.755782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.755750    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vxtv5"]
Jul 16 22:14:19.755826 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:19.755803    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:19.755868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:19.755853    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:19.755995 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:19.755967    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:20.054681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:20.054586    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="7fbb5b43d61838906749f5839fe70d27a31f6f9c03b6abcb1160ecc78a21c097" exitCode=0
Jul 16 22:14:20.054847 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:20.054673    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"7fbb5b43d61838906749f5839fe70d27a31f6f9c03b6abcb1160ecc78a21c097"}
Jul 16 22:14:21.823352 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:21.823125    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:21.823807 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:21.823125    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:21.823807 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:21.823372    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:21.823807 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:21.823141    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:21.823807 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:21.823439    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:21.823807 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:21.823536    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:22.061953 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:22.061914    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="4e0f73b90bd5cca6d64c50bb25a8c6a4520e3df2b32621af7a8486b5603fb09c" exitCode=0
Jul 16 22:14:22.062155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:22.061977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"4e0f73b90bd5cca6d64c50bb25a8c6a4520e3df2b32621af7a8486b5603fb09c"}
Jul 16 22:14:23.066917 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:23.066839    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="a55e1c8ce04e3570eb46f39ca400825183aaf1e7fce97e64b35ce065750177e5" exitCode=0
Jul 16 22:14:23.066917 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:23.066880    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"a55e1c8ce04e3570eb46f39ca400825183aaf1e7fce97e64b35ce065750177e5"}
Jul 16 22:14:23.823292 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:23.823262    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:23.823454 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:23.823262    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:23.823454 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:23.823376    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p4d5g" podUID="f13b359c-19ed-410b-9eaf-58148f522ff7"
Jul 16 22:14:23.823552 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:23.823481    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:14:23.823552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:23.823263    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:23.823661 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:23.823559    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zfg42" podUID="27c62e9b-86f4-4583-9190-47e22395c837"
Jul 16 22:14:24.446365 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:24.446332    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:24.446751 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.446506    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:24.446751 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.446582    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:56.446558737 +0000 UTC m=+66.141140581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 22:14:24.547562 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:24.547524    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:24.547741 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:24.547571    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:24.547741 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547699    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:24.547741 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547717    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 22:14:24.547741 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547733    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 22:14:24.547741 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547746    2556 projected.go:196] Error preparing data for projected volume kube-api-access-tlwmm for pod openshift-network-diagnostics/network-check-target-zfg42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:24.547940 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547771    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret podName:f13b359c-19ed-410b-9eaf-58148f522ff7 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:56.547752873 +0000 UTC m=+66.242334708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret") pod "global-pull-secret-syncer-p4d5g" (UID: "f13b359c-19ed-410b-9eaf-58148f522ff7") : object "kube-system"/"original-pull-secret" not registered
Jul 16 22:14:24.547940 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:24.547795    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm podName:27c62e9b-86f4-4583-9190-47e22395c837 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:56.547779054 +0000 UTC m=+66.242360905 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlwmm" (UniqueName: "kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm") pod "network-check-target-zfg42" (UID: "27c62e9b-86f4-4583-9190-47e22395c837") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 22:14:25.132587 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.132539    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-196.ec2.internal" event="NodeReady"
Jul 16 22:14:25.132748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.132710    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 22:14:25.178942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.178890    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wr797"]
Jul 16 22:14:25.182535 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.182503    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.183004 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.182979    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bgqbq"]
Jul 16 22:14:25.184932 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.184910    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 22:14:25.185027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.184911    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gpmss\""
Jul 16 22:14:25.185027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.184915    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 22:14:25.186582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.186564    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.211010 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.210986    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 22:14:25.212033 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.211995    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wr797"]
Jul 16 22:14:25.212904 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.212847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bgqbq"]
Jul 16 22:14:25.216607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.216554    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zbvf\""
Jul 16 22:14:25.216985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.216554    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 22:14:25.216985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.216555    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 22:14:25.354420 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354386    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc26a0a7-6613-4580-a2f6-0cd5c3577633-config-volume\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.354671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354440    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.354671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354454    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc26a0a7-6613-4580-a2f6-0cd5c3577633-tmp-dir\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.354671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354477    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e45c7d47-d770-41aa-bfba-28497d321d0a-cert\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.354671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz677\" (UniqueName: \"kubernetes.io/projected/cc26a0a7-6613-4580-a2f6-0cd5c3577633-kube-api-access-mz677\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.354671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.354649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48cb\" (UniqueName: \"kubernetes.io/projected/e45c7d47-d770-41aa-bfba-28497d321d0a-kube-api-access-w48cb\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.455670 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455633    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc26a0a7-6613-4580-a2f6-0cd5c3577633-config-volume\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455698    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc26a0a7-6613-4580-a2f6-0cd5c3577633-tmp-dir\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e45c7d47-d770-41aa-bfba-28497d321d0a-cert\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz677\" (UniqueName: \"kubernetes.io/projected/cc26a0a7-6613-4580-a2f6-0cd5c3577633-kube-api-access-mz677\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.455811    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48cb\" (UniqueName: \"kubernetes.io/projected/e45c7d47-d770-41aa-bfba-28497d321d0a-kube-api-access-w48cb\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:25.455895    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:25.456263 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:25.455967    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:25.955946607 +0000 UTC m=+35.650528457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:25.456594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.456298    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc26a0a7-6613-4580-a2f6-0cd5c3577633-tmp-dir\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.456594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.456349    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc26a0a7-6613-4580-a2f6-0cd5c3577633-config-volume\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.460399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.460378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e45c7d47-d770-41aa-bfba-28497d321d0a-cert\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.463806 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.463741    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48cb\" (UniqueName: \"kubernetes.io/projected/e45c7d47-d770-41aa-bfba-28497d321d0a-kube-api-access-w48cb\") pod \"ingress-canary-bgqbq\" (UID: \"e45c7d47-d770-41aa-bfba-28497d321d0a\") " pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.463806 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.463786    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz677\" (UniqueName: \"kubernetes.io/projected/cc26a0a7-6613-4580-a2f6-0cd5c3577633-kube-api-access-mz677\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.499986 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.499953    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bgqbq"
Jul 16 22:14:25.647963 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.647722    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bgqbq"]
Jul 16 22:14:25.651787 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:14:25.651747    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45c7d47_d770_41aa_bfba_28497d321d0a.slice/crio-c3f8f3d973c6142e7ae5f170c0b8f603e563fcfd1c8de1d8f1d973b3cea5a568 WatchSource:0}: Error finding container c3f8f3d973c6142e7ae5f170c0b8f603e563fcfd1c8de1d8f1d973b3cea5a568: Status 404 returned error can't find the container with id c3f8f3d973c6142e7ae5f170c0b8f603e563fcfd1c8de1d8f1d973b3cea5a568
Jul 16 22:14:25.806296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.806201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:25.823237 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.823172    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:25.823237 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.823184    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:25.823237 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.823175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:25.825257 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825235    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 22:14:25.825382 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825235    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7sbmm\""
Jul 16 22:14:25.825382 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825285    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kkg55\""
Jul 16 22:14:25.825382 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825300    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 22:14:25.825537 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:14:25.825646 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.825613    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:14:25.899159 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.899122    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:25.958047 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.958010    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:25.961082 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:25.961055    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:25.961243 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:25.961223    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:25.961366 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:25.961287    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:26.961265685 +0000 UTC m=+36.655847517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:26.075715 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:26.075630    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bgqbq" event={"ID":"e45c7d47-d770-41aa-bfba-28497d321d0a","Type":"ContainerStarted","Data":"c3f8f3d973c6142e7ae5f170c0b8f603e563fcfd1c8de1d8f1d973b3cea5a568"}
Jul 16 22:14:26.969659 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:26.969621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:26.970126 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:26.969733    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:26.970126 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:26.969788    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:28.969775121 +0000 UTC m=+38.664356952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:28.083796 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:28.083735    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="a1fd7b777c0472925c1c71f137d40aab48fd2321817c45d9e7dd4b3db5f7b6fd" exitCode=0
Jul 16 22:14:28.083796 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:28.083794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"a1fd7b777c0472925c1c71f137d40aab48fd2321817c45d9e7dd4b3db5f7b6fd"}
Jul 16 22:14:28.986046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:28.985833    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:28.986233 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:28.986022    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:28.986233 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:28.986197    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:32.986174034 +0000 UTC m=+42.680755912 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:29.090778 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:29.090741    2556 generic.go:356] "Generic (PLEG): container finished" podID="89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b" containerID="2f5b30e7622f8c26260a53e833acea56fb36f7a012c3de834b76a70649afd5d9" exitCode=0
Jul 16 22:14:29.091235 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:29.090799    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerDied","Data":"2f5b30e7622f8c26260a53e833acea56fb36f7a012c3de834b76a70649afd5d9"}
Jul 16 22:14:30.096317 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.096281    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" event={"ID":"89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b","Type":"ContainerStarted","Data":"0ba8a2188012910e2a214866b20744703c182380645627d3a032ec4627bc26fe"}
Jul 16 22:14:30.097672 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.097652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bgqbq" event={"ID":"e45c7d47-d770-41aa-bfba-28497d321d0a","Type":"ContainerStarted","Data":"ba2641e951dc648912d763a1772944dee82bbe7adb9247e312e2d5114a2f5344"}
Jul 16 22:14:30.118182 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.118126    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4zxq5" podStartSLOduration=8.480882675 podStartE2EDuration="40.118107698s" totalImagesPullingTime="31.637225023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:30.11718587 +0000 UTC m=+39.811767739" watchObservedRunningTime="2026-07-16 22:14:30.118107698 +0000 UTC m=+39.812689542"
Jul 16 22:14:30.132551 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.132496    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bgqbq" podStartSLOduration=1.284370672 podStartE2EDuration="5.132480333s" totalImagesPullingTime="3.848109661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:14:30.132194318 +0000 UTC m=+39.826776197" watchObservedRunningTime="2026-07-16 22:14:30.132480333 +0000 UTC m=+39.827062187"
Jul 16 22:14:30.248694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.248660    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:30.340949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.340916    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:30.492610 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.492574    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:30.592254 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.592228    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:30.696454 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:30.696423    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:33.014491 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:33.014459    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:33.014928 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:33.014565    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:33.014928 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:33.014626    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:41.014612865 +0000 UTC m=+50.709194696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:33.885072 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:33.885034    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:33.949019 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:33.948987    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:34.385431 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:34.385398    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:34.442987 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:34.442954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:41.066043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:41.066000    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:41.066443 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:41.066159    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:41.066443 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:41.066232    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:14:57.066216636 +0000 UTC m=+66.760798468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:46.702568 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:46.702527    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:46.763173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:46.763148    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:50.065145 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:50.065117    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gsf85"
Jul 16 22:14:50.216701 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:50.216665    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:50.320170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:50.320079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:54.813058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:54.813019    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:54.871516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:54.871481    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:54.927181 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:54.927157    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:14:56.470732 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.470676    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:14:56.472923 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.472905    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 22:14:56.481140 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:56.481124    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 22:14:56.481197 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:56.481186    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:00.481171317 +0000 UTC m=+130.175753150 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : secret "metrics-daemon-secret" not found
Jul 16 22:14:56.571280 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.571237    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:56.571280 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.571284    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:56.573222 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.573204    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 22:14:56.573310 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.573293    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 22:14:56.583630 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.583606    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 22:14:56.585725 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.585698    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f13b359c-19ed-410b-9eaf-58148f522ff7-original-pull-secret\") pod \"global-pull-secret-syncer-p4d5g\" (UID: \"f13b359c-19ed-410b-9eaf-58148f522ff7\") " pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:56.594675 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.594651    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwmm\" (UniqueName: \"kubernetes.io/projected/27c62e9b-86f4-4583-9190-47e22395c837-kube-api-access-tlwmm\") pod \"network-check-target-zfg42\" (UID: \"27c62e9b-86f4-4583-9190-47e22395c837\") " pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:56.744363 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.744287    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kkg55\""
Jul 16 22:14:56.747593 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.747579    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p4d5g"
Jul 16 22:14:56.752929 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.752911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:14:56.889765 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.889692    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zfg42"]
Jul 16 22:14:56.894027 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:14:56.893997    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c62e9b_86f4_4583_9190_47e22395c837.slice/crio-1ac778651d10d94e6f01eedf2b6c4e3243e4339e76d4517125797d08c97e879c WatchSource:0}: Error finding container 1ac778651d10d94e6f01eedf2b6c4e3243e4339e76d4517125797d08c97e879c: Status 404 returned error can't find the container with id 1ac778651d10d94e6f01eedf2b6c4e3243e4339e76d4517125797d08c97e879c
Jul 16 22:14:56.903234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:56.903170    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p4d5g"]
Jul 16 22:14:56.906597 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:14:56.906575    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13b359c_19ed_410b_9eaf_58148f522ff7.slice/crio-2aeed25ae2fe30aa23c431a079c063f23d3a210034721b9bd66c8017a015ca68 WatchSource:0}: Error finding container 2aeed25ae2fe30aa23c431a079c063f23d3a210034721b9bd66c8017a015ca68: Status 404 returned error can't find the container with id 2aeed25ae2fe30aa23c431a079c063f23d3a210034721b9bd66c8017a015ca68
Jul 16 22:14:57.074503 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:57.074412    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:14:57.074721 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:57.074574    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:14:57.074721 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:14:57.074650    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:15:29.074633713 +0000 UTC m=+98.769215546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:14:57.157280 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:57.157244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4d5g" event={"ID":"f13b359c-19ed-410b-9eaf-58148f522ff7","Type":"ContainerStarted","Data":"2aeed25ae2fe30aa23c431a079c063f23d3a210034721b9bd66c8017a015ca68"}
Jul 16 22:14:57.158268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:14:57.158247    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zfg42" event={"ID":"27c62e9b-86f4-4583-9190-47e22395c837","Type":"ContainerStarted","Data":"1ac778651d10d94e6f01eedf2b6c4e3243e4339e76d4517125797d08c97e879c"}
Jul 16 22:15:01.831979 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:01.831947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:15:02.173055 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:02.172960    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p4d5g" event={"ID":"f13b359c-19ed-410b-9eaf-58148f522ff7","Type":"ContainerStarted","Data":"25505d6fbc5688c5aa765c4603a079c4856ba41df4959ab3231909d43f6f2e8d"}
Jul 16 22:15:02.174293 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:02.174268    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zfg42" event={"ID":"27c62e9b-86f4-4583-9190-47e22395c837","Type":"ContainerStarted","Data":"e72bc9669e87fbf7d850154c607d0e46b2f8a82016473c73e42bd8245b9b45f5"}
Jul 16 22:15:02.174426 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:02.174413    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:15:02.189069 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:02.189018    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p4d5g" podStartSLOduration=66.82832142 podStartE2EDuration="1m11.18900236s" totalImagesPullingTime="4.36068094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:13:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:02.188950321 +0000 UTC m=+71.883532199" watchObservedRunningTime="2026-07-16 22:15:02.18900236 +0000 UTC m=+71.883584215"
Jul 16 22:15:02.204423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:02.204385    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zfg42" podStartSLOduration=67.836207387 podStartE2EDuration="1m12.204374576s" totalImagesPullingTime="4.368167189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:15:02.203697981 +0000 UTC m=+71.898279836" watchObservedRunningTime="2026-07-16 22:15:02.204374576 +0000 UTC m=+71.898956444"
Jul 16 22:15:27.177768 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:27.177738    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:15:27.229359 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:27.229328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:15:29.083889 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:29.083849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:15:29.084257 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:15:29.083990    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:15:29.084257 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:15:29.084044    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:16:33.084030498 +0000 UTC m=+162.778612334 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:15:33.179529 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:15:33.179495    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zfg42"
Jul 16 22:16:00.498119 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:16:00.498028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:16:00.498620 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:00.498196    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 22:16:00.498620 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:00.498259    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:02.4982443 +0000 UTC m=+252.192826132 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : secret "metrics-daemon-secret" not found
Jul 16 22:16:28.194978 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:28.194931    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wr797" podUID="cc26a0a7-6613-4580-a2f6-0cd5c3577633"
Jul 16 22:16:28.378780 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:16:28.378750    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wr797"
Jul 16 22:16:28.836645 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:28.836609    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:16:33.113514 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:16:33.113464    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:16:33.113910 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:33.113640    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 22:16:33.113910 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:16:33.113717    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls podName:cc26a0a7-6613-4580-a2f6-0cd5c3577633 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:35.113694155 +0000 UTC m=+284.808276004 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls") pod "dns-default-wr797" (UID: "cc26a0a7-6613-4580-a2f6-0cd5c3577633") : secret "dns-default-metrics-tls" not found
Jul 16 22:16:40.824772 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:16:40.824741    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:17:39.109773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:17:39.109746    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:17:39.625853 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:17:39.625821    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:02.501193 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:02.501153    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:18:02.501670 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:02.501296    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 16 22:18:02.501670 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:02.501366    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs podName:889fca95-1e44-443e-87f4-dbf482146ac6 nodeName:}" failed. No retries permitted until 2026-07-16 22:20:04.501349488 +0000 UTC m=+374.195931320 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs") pod "network-metrics-daemon-vxtv5" (UID: "889fca95-1e44-443e-87f4-dbf482146ac6") : secret "metrics-daemon-secret" not found
Jul 16 22:18:07.366294 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.366238    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"]
Jul 16 22:18:07.368158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.368137    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.370428 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.370407    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 22:18:07.370648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.370627    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 22:18:07.370648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.370638    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 22:18:07.370648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.370642    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-bb7zm\""
Jul 16 22:18:07.371242 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.371224    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:18:07.399354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.399332    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6474b87dd6-w47mb"]
Jul 16 22:18:07.400857 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.400843    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.405707 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.405680    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 22:18:07.405813 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.405758    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d7rs9\""
Jul 16 22:18:07.408157 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:07.408133    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"router-metrics-certs-default\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret"
Jul 16 22:18:07.408698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.408641    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 22:18:07.408698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.408665    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 22:18:07.408698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.408665    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 22:18:07.408865 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.408712    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 22:18:07.410071 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.410053    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"]
Jul 16 22:18:07.434635 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.434606    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35ba01bc-9376-48d0-b96f-80ee68024cff-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.434858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.434672    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4sd8\" (UniqueName: \"kubernetes.io/projected/35ba01bc-9376-48d0-b96f-80ee68024cff-kube-api-access-x4sd8\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.434858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.434735    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35ba01bc-9376-48d0-b96f-80ee68024cff-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.435586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.435566    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6474b87dd6-w47mb"]
Jul 16 22:18:07.466771 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.466745    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"]
Jul 16 22:18:07.468263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.468250    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"]
Jul 16 22:18:07.468381 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.468365    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"
Jul 16 22:18:07.469657 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.469642    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"]
Jul 16 22:18:07.469750 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.469736    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.470955 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.470940    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.471582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.471566    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 22:18:07.471788 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.471773    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 22:18:07.471859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.471839    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-x2h8r\""
Jul 16 22:18:07.472330 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.472312    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hzz4k\""
Jul 16 22:18:07.473989 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.473975    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 16 22:18:07.479918 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.479898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 16 22:18:07.480015 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.479902    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 16 22:18:07.480015 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.480006    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-p85f4\""
Jul 16 22:18:07.480260 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.480242    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:18:07.480354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.480270    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 22:18:07.480354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.480312    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:18:07.480354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.480333    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:18:07.491569 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.491545    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"]
Jul 16 22:18:07.492498 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.492478    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"]
Jul 16 22:18:07.493354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.493338    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"]
Jul 16 22:18:07.535383 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535351    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.535547 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35ba01bc-9376-48d0-b96f-80ee68024cff-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.535547 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535440    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4838512-aba9-4b60-aac7-154dd2d3dfdc-config\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.535547 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-default-certificate\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.535547 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535484    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-stats-auth\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.535704 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.535704 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4838512-aba9-4b60-aac7-154dd2d3dfdc-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.535704 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535612    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rpcf\" (UniqueName: \"kubernetes.io/projected/a4838512-aba9-4b60-aac7-154dd2d3dfdc-kube-api-access-7rpcf\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.535704 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535645    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35ba01bc-9376-48d0-b96f-80ee68024cff-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.535871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535735    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc4wq\" (UniqueName: \"kubernetes.io/projected/0b15d6e1-4de4-403e-a418-125c81ca4701-kube-api-access-xc4wq\") pod \"volume-data-source-validator-78cf4c6f6-kkbnv\" (UID: \"0b15d6e1-4de4-403e-a418-125c81ca4701\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"
Jul 16 22:18:07.535871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535778    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xgc\" (UniqueName: \"kubernetes.io/projected/723ca0d0-33df-4714-af5e-9513f7ec0c81-kube-api-access-24xgc\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.535871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4sd8\" (UniqueName: \"kubernetes.io/projected/35ba01bc-9376-48d0-b96f-80ee68024cff-kube-api-access-x4sd8\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.535871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.535871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.535853    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjk4v\" (UniqueName: \"kubernetes.io/projected/7a76bb28-0498-4c32-b0a7-63af15b35b55-kube-api-access-zjk4v\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.536308 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.536286    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/35ba01bc-9376-48d0-b96f-80ee68024cff-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.537705 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.537690    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/35ba01bc-9376-48d0-b96f-80ee68024cff-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.546297 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.546271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4sd8\" (UniqueName: \"kubernetes.io/projected/35ba01bc-9376-48d0-b96f-80ee68024cff-kube-api-access-x4sd8\") pod \"kube-storage-version-migrator-operator-7658d9b749-t5xcs\" (UID: \"35ba01bc-9376-48d0-b96f-80ee68024cff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.636161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc4wq\" (UniqueName: \"kubernetes.io/projected/0b15d6e1-4de4-403e-a418-125c81ca4701-kube-api-access-xc4wq\") pod \"volume-data-source-validator-78cf4c6f6-kkbnv\" (UID: \"0b15d6e1-4de4-403e-a418-125c81ca4701\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"
Jul 16 22:18:07.636161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636136    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xgc\" (UniqueName: \"kubernetes.io/projected/723ca0d0-33df-4714-af5e-9513f7ec0c81-kube-api-access-24xgc\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.636161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjk4v\" (UniqueName: \"kubernetes.io/projected/7a76bb28-0498-4c32-b0a7-63af15b35b55-kube-api-access-zjk4v\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4838512-aba9-4b60-aac7-154dd2d3dfdc-config\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-default-certificate\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:07.636333    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:18:07.636397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-stats-auth\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.636689 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:07.636403    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls podName:7a76bb28-0498-4c32-b0a7-63af15b35b55 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:08.136384797 +0000 UTC m=+257.830966642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-ffnr5" (UID: "7a76bb28-0498-4c32-b0a7-63af15b35b55") : secret "samples-operator-tls" not found
Jul 16 22:18:07.636689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.636689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4838512-aba9-4b60-aac7-154dd2d3dfdc-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.636689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636520    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpcf\" (UniqueName: \"kubernetes.io/projected/a4838512-aba9-4b60-aac7-154dd2d3dfdc-kube-api-access-7rpcf\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.636689 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:07.636629    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:08.136610879 +0000 UTC m=+257.831192727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : configmap references non-existent config key: service-ca.crt
Jul 16 22:18:07.636881 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.636826    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4838512-aba9-4b60-aac7-154dd2d3dfdc-config\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.638572 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.638544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4838512-aba9-4b60-aac7-154dd2d3dfdc-serving-cert\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.638657 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.638605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-default-certificate\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.638787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.638771    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-stats-auth\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.645751 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.645725    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpcf\" (UniqueName: \"kubernetes.io/projected/a4838512-aba9-4b60-aac7-154dd2d3dfdc-kube-api-access-7rpcf\") pod \"service-ca-operator-5bcbc94c78-mln2v\" (UID: \"a4838512-aba9-4b60-aac7-154dd2d3dfdc\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.646049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.646032    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjk4v\" (UniqueName: \"kubernetes.io/projected/7a76bb28-0498-4c32-b0a7-63af15b35b55-kube-api-access-zjk4v\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:07.646274 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.646254    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc4wq\" (UniqueName: \"kubernetes.io/projected/0b15d6e1-4de4-403e-a418-125c81ca4701-kube-api-access-xc4wq\") pod \"volume-data-source-validator-78cf4c6f6-kkbnv\" (UID: \"0b15d6e1-4de4-403e-a418-125c81ca4701\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"
Jul 16 22:18:07.646924 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.646907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xgc\" (UniqueName: \"kubernetes.io/projected/723ca0d0-33df-4714-af5e-9513f7ec0c81-kube-api-access-24xgc\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:07.676818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.676795    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"
Jul 16 22:18:07.690874 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.690852    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:07.777853 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.777825    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"
Jul 16 22:18:07.787919 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.787894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"
Jul 16 22:18:07.797429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.797407    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs"]
Jul 16 22:18:07.801518 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:07.801488    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ba01bc_9376_48d0_b96f_80ee68024cff.slice/crio-7fedf450df52b4bee85e188d4d5bcd5c985153b5387ab3d34388633a1234dd3d WatchSource:0}: Error finding container 7fedf450df52b4bee85e188d4d5bcd5c985153b5387ab3d34388633a1234dd3d: Status 404 returned error can't find the container with id 7fedf450df52b4bee85e188d4d5bcd5c985153b5387ab3d34388633a1234dd3d
Jul 16 22:18:07.905320 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.905236    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv"]
Jul 16 22:18:07.908591 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:07.908566    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b15d6e1_4de4_403e_a418_125c81ca4701.slice/crio-b8dc7dfb5e8b0e431b62c82fd7c641c4e74fee1a956f14dccaea10c382be4f14 WatchSource:0}: Error finding container b8dc7dfb5e8b0e431b62c82fd7c641c4e74fee1a956f14dccaea10c382be4f14: Status 404 returned error can't find the container with id b8dc7dfb5e8b0e431b62c82fd7c641c4e74fee1a956f14dccaea10c382be4f14
Jul 16 22:18:07.921975 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:07.921949    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v"]
Jul 16 22:18:07.925925 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:07.925899    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4838512_aba9_4b60_aac7_154dd2d3dfdc.slice/crio-dbc8dbe867510049129d940933ce14f28c428afcbc262e3c441c68f4cc012801 WatchSource:0}: Error finding container dbc8dbe867510049129d940933ce14f28c428afcbc262e3c441c68f4cc012801: Status 404 returned error can't find the container with id dbc8dbe867510049129d940933ce14f28c428afcbc262e3c441c68f4cc012801
Jul 16 22:18:08.141045 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.141011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:08.141225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.141076    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:08.141225 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.141197    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:09.141177446 +0000 UTC m=+258.835759282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : configmap references non-existent config key: service-ca.crt
Jul 16 22:18:08.141225 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.141212    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:18:08.141400 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.141244    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls podName:7a76bb28-0498-4c32-b0a7-63af15b35b55 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:09.141237557 +0000 UTC m=+258.835819389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-ffnr5" (UID: "7a76bb28-0498-4c32-b0a7-63af15b35b55") : secret "samples-operator-tls" not found
Jul 16 22:18:08.248920 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.248892    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 22:18:08.257371 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.257346    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:18:08.257476 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.257418    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:08.75739705 +0000 UTC m=+258.451978883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : secret "router-metrics-certs-default" not found
Jul 16 22:18:08.606246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.606152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs" event={"ID":"35ba01bc-9376-48d0-b96f-80ee68024cff","Type":"ContainerStarted","Data":"7fedf450df52b4bee85e188d4d5bcd5c985153b5387ab3d34388633a1234dd3d"}
Jul 16 22:18:08.608049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.608015    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv" event={"ID":"0b15d6e1-4de4-403e-a418-125c81ca4701","Type":"ContainerStarted","Data":"b8dc7dfb5e8b0e431b62c82fd7c641c4e74fee1a956f14dccaea10c382be4f14"}
Jul 16 22:18:08.611710 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.611668    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v" event={"ID":"a4838512-aba9-4b60-aac7-154dd2d3dfdc","Type":"ContainerStarted","Data":"dbc8dbe867510049129d940933ce14f28c428afcbc262e3c441c68f4cc012801"}
Jul 16 22:18:08.849251 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.849218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:08.849520 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.849443    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:18:08.849520 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:08.849509    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:09.849490215 +0000 UTC m=+259.544072051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : secret "router-metrics-certs-default" not found
Jul 16 22:18:08.940126 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:08.940019    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:09.151583 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.151544    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:09.151807 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.151630    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:09.151807 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:09.151711    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:11.151688263 +0000 UTC m=+260.846270095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : configmap references non-existent config key: service-ca.crt
Jul 16 22:18:09.151807 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:09.151754    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:18:09.151962 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:09.151819    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls podName:7a76bb28-0498-4c32-b0a7-63af15b35b55 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:11.151801439 +0000 UTC m=+260.846383272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-ffnr5" (UID: "7a76bb28-0498-4c32-b0a7-63af15b35b55") : secret "samples-operator-tls" not found
Jul 16 22:18:09.616165 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.616126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv" event={"ID":"0b15d6e1-4de4-403e-a418-125c81ca4701","Type":"ContainerStarted","Data":"1fdcc585d5e8e6d3f9f7b4fcbd584dec872516ce9468978fb4abc398106ece1e"}
Jul 16 22:18:09.631020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.630964    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-kkbnv" podStartSLOduration=1.611886803 podStartE2EDuration="2.630945332s" totalImagesPullingTime="1.019058529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:09.630829767 +0000 UTC m=+259.325411623" watchObservedRunningTime="2026-07-16 22:18:09.630945332 +0000 UTC m=+259.325527187"
Jul 16 22:18:09.710279 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.710237    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:09.777146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.777074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:09.857950 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:09.857909    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:09.858131 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:09.858082    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:18:09.858206 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:09.858174    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:11.858152251 +0000 UTC m=+261.552734086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : secret "router-metrics-certs-default" not found
Jul 16 22:18:10.620669 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:10.620562    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v" event={"ID":"a4838512-aba9-4b60-aac7-154dd2d3dfdc","Type":"ContainerStarted","Data":"cfd91ba18ae1c6353a5f9c8433709e341536049ff155ce512bf3d8ff2f981f4c"}
Jul 16 22:18:10.622043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:10.622016    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs" event={"ID":"35ba01bc-9376-48d0-b96f-80ee68024cff","Type":"ContainerStarted","Data":"7a0108614cceb8bd881e4ab28422992010b08ee32d4677ff5b87890bb57a0a5b"}
Jul 16 22:18:10.635760 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:10.635711    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v" podStartSLOduration=1.288653854 podStartE2EDuration="3.635692803s" totalImagesPullingTime="2.347038949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:10.634971286 +0000 UTC m=+260.329553142" watchObservedRunningTime="2026-07-16 22:18:10.635692803 +0000 UTC m=+260.330274658"
Jul 16 22:18:10.649361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:10.649309    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs" podStartSLOduration=1.17860528 podStartE2EDuration="3.649287199s" totalImagesPullingTime="2.470681919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:10.648683235 +0000 UTC m=+260.343265090" watchObservedRunningTime="2026-07-16 22:18:10.649287199 +0000 UTC m=+260.343869056"
Jul 16 22:18:11.169276 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.169234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:11.169489 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.169322    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:11.169489 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.169400    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:18:11.169489 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.169468    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:15.1694496 +0000 UTC m=+264.864031445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : configmap references non-existent config key: service-ca.crt
Jul 16 22:18:11.169489 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.169487    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls podName:7a76bb28-0498-4c32-b0a7-63af15b35b55 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:15.169477975 +0000 UTC m=+264.864059811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-ffnr5" (UID: "7a76bb28-0498-4c32-b0a7-63af15b35b55") : secret "samples-operator-tls" not found
Jul 16 22:18:11.455310 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.455253    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g5f86"]
Jul 16 22:18:11.457149 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.457134    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.460436 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.460412    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 22:18:11.460554 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.460421    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5nldq\""
Jul 16 22:18:11.460672 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.460658    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 22:18:11.460887 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.460866    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 22:18:11.460991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.460871    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 22:18:11.473669 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.473646    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g5f86"]
Jul 16 22:18:11.489763 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.489736    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:11.563698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.563670    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:11.572585 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.572552    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ctlw\" (UniqueName: \"kubernetes.io/projected/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-api-access-5ctlw\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.572754 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.572614    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-crio-socket\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.572754 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.572664    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-data-volume\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.572754 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.572732    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.572890 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.572758    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.673996 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.673963    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ctlw\" (UniqueName: \"kubernetes.io/projected/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-api-access-5ctlw\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-crio-socket\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-data-volume\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674083    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.674163    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674174    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-crio-socket\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.674211    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:12.174196563 +0000 UTC m=+261.868778398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : configmap "kube-rbac-proxy" not found
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674228    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.674313    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:11.674451 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.674345    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:12.17433714 +0000 UTC m=+261.868918973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:11.674755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.674455    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-data-volume\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.681786 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.681763    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ctlw\" (UniqueName: \"kubernetes.io/projected/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-api-access-5ctlw\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:11.877061 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.876958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:11.877228 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.877123    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:18:11.877228 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:11.877204    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:15.877184573 +0000 UTC m=+265.571766405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : secret "router-metrics-certs-default" not found
Jul 16 22:18:11.931273 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.931235    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:11.984416 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:11.984373    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:12.179459 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:12.179339    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:12.179459 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:12.179391    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:12.179661 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:12.179482    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:18:12.179661 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:12.179572    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:13.179548746 +0000 UTC m=+262.874130596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : configmap "kube-rbac-proxy" not found
Jul 16 22:18:12.179661 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:12.179581    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:12.179661 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:12.179628    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:13.179614795 +0000 UTC m=+262.874196626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:12.192137 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:12.192104    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:12.273564 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:12.273528    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.182914 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.182885    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"]
Jul 16 22:18:13.184844 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.184828    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"
Jul 16 22:18:13.187045 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.187023    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 22:18:13.187404 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.187386    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wmkdx\""
Jul 16 22:18:13.187492 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.187410    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 22:18:13.188177 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.188161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:13.188255 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.188188    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:13.188305 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:13.188272    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:18:13.188350 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:13.188320    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:15.188307135 +0000 UTC m=+264.882888968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : configmap "kube-rbac-proxy" not found
Jul 16 22:18:13.188395 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:13.188350    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:13.188436 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:13.188403    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:15.188390347 +0000 UTC m=+264.882972181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:13.194444 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.194423    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"]
Jul 16 22:18:13.222005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.221970    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.284845 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.284814    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.288999 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.288975    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkv7m\" (UniqueName: \"kubernetes.io/projected/07121a77-cc80-4445-8c0a-76d471019689-kube-api-access-lkv7m\") pod \"migrator-7cc9c7ffd7-gsq2z\" (UID: \"07121a77-cc80-4445-8c0a-76d471019689\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"
Jul 16 22:18:13.389459 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.389422    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkv7m\" (UniqueName: \"kubernetes.io/projected/07121a77-cc80-4445-8c0a-76d471019689-kube-api-access-lkv7m\") pod \"migrator-7cc9c7ffd7-gsq2z\" (UID: \"07121a77-cc80-4445-8c0a-76d471019689\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"
Jul 16 22:18:13.397749 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.397719    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkv7m\" (UniqueName: \"kubernetes.io/projected/07121a77-cc80-4445-8c0a-76d471019689-kube-api-access-lkv7m\") pod \"migrator-7cc9c7ffd7-gsq2z\" (UID: \"07121a77-cc80-4445-8c0a-76d471019689\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"
Jul 16 22:18:13.476719 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.476693    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-db84x_6ad72e05-81f9-449a-8f6f-a552cdbe9748/dns-node-resolver/0.log"
Jul 16 22:18:13.494067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.494045    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"
Jul 16 22:18:13.610517 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.610485    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z"]
Jul 16 22:18:13.613805 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:13.613781    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07121a77_cc80_4445_8c0a_76d471019689.slice/crio-fba7cd5aa0ee478bfeec7a9b43de5f8ceb40a2edfbda6998d864f085abcb71f9 WatchSource:0}: Error finding container fba7cd5aa0ee478bfeec7a9b43de5f8ceb40a2edfbda6998d864f085abcb71f9: Status 404 returned error can't find the container with id fba7cd5aa0ee478bfeec7a9b43de5f8ceb40a2edfbda6998d864f085abcb71f9
Jul 16 22:18:13.632055 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.632025    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z" event={"ID":"07121a77-cc80-4445-8c0a-76d471019689","Type":"ContainerStarted","Data":"fba7cd5aa0ee478bfeec7a9b43de5f8ceb40a2edfbda6998d864f085abcb71f9"}
Jul 16 22:18:13.804892 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.804824    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-65zxx"]
Jul 16 22:18:13.807085 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.807071    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.810625 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.810601    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 16 22:18:13.810763 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.810661    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 16 22:18:13.810763 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.810601    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 16 22:18:13.810763 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.810693    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 16 22:18:13.811019 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.811005    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dxbll\""
Jul 16 22:18:13.818363 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.818339    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-65zxx"]
Jul 16 22:18:13.849869 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.849839    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.892770 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.892736    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-cabundle\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.892916 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.892800    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-key\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.892916 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.892854    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8x7j\" (UniqueName: \"kubernetes.io/projected/c4c5d35d-05e4-4983-b65b-e398761110f8-kube-api-access-w8x7j\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.906980 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.906955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:13.993647 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.993615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-cabundle\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.993823 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.993664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-key\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.993823 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.993694    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8x7j\" (UniqueName: \"kubernetes.io/projected/c4c5d35d-05e4-4983-b65b-e398761110f8-kube-api-access-w8x7j\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.994278 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.994259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-cabundle\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:13.996006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:13.995985    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c4c5d35d-05e4-4983-b65b-e398761110f8-signing-key\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:14.002070 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.002050    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8x7j\" (UniqueName: \"kubernetes.io/projected/c4c5d35d-05e4-4983-b65b-e398761110f8-kube-api-access-w8x7j\") pod \"service-ca-58598695f7-65zxx\" (UID: \"c4c5d35d-05e4-4983-b65b-e398761110f8\") " pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:14.115609 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.115520    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-65zxx"
Jul 16 22:18:14.242442 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.242411    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-65zxx"]
Jul 16 22:18:14.245729 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:14.245701    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c5d35d_05e4_4983_b65b_e398761110f8.slice/crio-1d75feb3d613c5b550842c4e40a8cf9fc24e1efe982d8b08fc9aa84f9d0d08d3 WatchSource:0}: Error finding container 1d75feb3d613c5b550842c4e40a8cf9fc24e1efe982d8b08fc9aa84f9d0d08d3: Status 404 returned error can't find the container with id 1d75feb3d613c5b550842c4e40a8cf9fc24e1efe982d8b08fc9aa84f9d0d08d3
Jul 16 22:18:14.477748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.477723    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7bngx_f3065d8c-65a8-4b78-b859-3fd67a18a667/node-ca/0.log"
Jul 16 22:18:14.641812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.641769    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-65zxx" event={"ID":"c4c5d35d-05e4-4983-b65b-e398761110f8","Type":"ContainerStarted","Data":"369d5de2446f188df729bcb92a1562e7faac8e3a34337332fcb8ddefe127d301"}
Jul 16 22:18:14.641978 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.641859    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-65zxx" event={"ID":"c4c5d35d-05e4-4983-b65b-e398761110f8","Type":"ContainerStarted","Data":"1d75feb3d613c5b550842c4e40a8cf9fc24e1efe982d8b08fc9aa84f9d0d08d3"}
Jul 16 22:18:14.660310 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.660243    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-65zxx" podStartSLOduration=1.660223297 podStartE2EDuration="1.660223297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:14.659969321 +0000 UTC m=+264.354551176" watchObservedRunningTime="2026-07-16 22:18:14.660223297 +0000 UTC m=+264.354805153"
Jul 16 22:18:14.767348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:14.767249    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:15.203954 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.203920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:15.204149 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.203964    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:15.204149 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.204013    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:15.204149 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.204044    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:15.204149 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204071    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204143    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204157    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204163    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:19.204146737 +0000 UTC m=+268.898728568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204252    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:23.204236967 +0000 UTC m=+272.898818810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : configmap references non-existent config key: service-ca.crt
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204265    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:19.204257864 +0000 UTC m=+268.898839696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : configmap "kube-rbac-proxy" not found
Jul 16 22:18:15.204328 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.204277    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls podName:7a76bb28-0498-4c32-b0a7-63af15b35b55 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:23.204269614 +0000 UTC m=+272.898851447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-ffnr5" (UID: "7a76bb28-0498-4c32-b0a7-63af15b35b55") : secret "samples-operator-tls" not found
Jul 16 22:18:15.277544 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.277520    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bgqbq_e45c7d47-d770-41aa-bfba-28497d321d0a/serve-healthcheck-canary/0.log"
Jul 16 22:18:15.646332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.646253    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z" event={"ID":"07121a77-cc80-4445-8c0a-76d471019689","Type":"ContainerStarted","Data":"f6015bec9e472ab1a6f92de121ec646c996b8caca075ee67b6a9dbc82461a4fc"}
Jul 16 22:18:15.646332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.646305    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z" event={"ID":"07121a77-cc80-4445-8c0a-76d471019689","Type":"ContainerStarted","Data":"db0b84ccc66cc0a33dc5c8b65d0d39143feaa9c0cedb2cbd965c4cd71d2beaa7"}
Jul 16 22:18:15.661485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.661434    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gsq2z" podStartSLOduration=1.46311117 podStartE2EDuration="2.661414768s" totalImagesPullingTime="1.198303598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:15.660419805 +0000 UTC m=+265.355001677" watchObservedRunningTime="2026-07-16 22:18:15.661414768 +0000 UTC m=+265.355996623"
Jul 16 22:18:15.911078 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:15.910997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:15.911246 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.911114    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 22:18:15.911246 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:15.911165    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs podName:723ca0d0-33df-4714-af5e-9513f7ec0c81 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:23.911152491 +0000 UTC m=+273.605734324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs") pod "router-default-6474b87dd6-w47mb" (UID: "723ca0d0-33df-4714-af5e-9513f7ec0c81") : secret "router-metrics-certs-default" not found
Jul 16 22:18:16.187573 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.187545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.380243 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.380208    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.482390 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.482304    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.583158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.583125    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.684057 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.684018    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.786015 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.785919    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:16.886399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:16.886364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.101015 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.100975    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.293421 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.293392    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.490249 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.490212    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.694124 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.694076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.765308 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.765223    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:18.903331 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:18.903297    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:19.236886 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:19.236852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:19.237299 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:19.236894    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:19.237299 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:19.237008    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:19.237299 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:19.237043    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 22:18:19.237299 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:19.237069    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:27.237054593 +0000 UTC m=+276.931636430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : secret "insights-runtime-extractor-tls" not found
Jul 16 22:18:19.237299 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:19.237111    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm podName:bee0c7fa-9c9e-4c2e-952a-099ac0774fbd nodeName:}" failed. No retries permitted until 2026-07-16 22:18:27.237081688 +0000 UTC m=+276.931663520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5f86" (UID: "bee0c7fa-9c9e-4c2e-952a-099ac0774fbd") : configmap "kube-rbac-proxy" not found
Jul 16 22:18:20.112604 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:20.112570    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:20.174435 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:20.174409    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:20.488432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:20.488385    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:20.684557 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:20.684502    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:21.628084 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:21.628054    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:21.728354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:21.728328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:22.086724 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:22.086692    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:22.635384 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:22.635351    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:22.737234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:22.737200    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:23.269601 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.269567    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:23.269802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.269613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:23.270203 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.270185    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/723ca0d0-33df-4714-af5e-9513f7ec0c81-service-ca-bundle\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:23.271976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.271944    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a76bb28-0498-4c32-b0a7-63af15b35b55-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-ffnr5\" (UID: \"7a76bb28-0498-4c32-b0a7-63af15b35b55\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:23.287605 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.287581    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:23.383684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.383653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"
Jul 16 22:18:23.504248 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.504217    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5"]
Jul 16 22:18:23.673564 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.673535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5" event={"ID":"7a76bb28-0498-4c32-b0a7-63af15b35b55","Type":"ContainerStarted","Data":"c11d67465b9caf7dfa5943ca5bd1f34bf3c95e078f3847a3c627965a0ee80fe7"}
Jul 16 22:18:23.686662 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.686637    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:23.975434 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.975397    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:23.977737 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:23.977717    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/723ca0d0-33df-4714-af5e-9513f7ec0c81-metrics-certs\") pod \"router-default-6474b87dd6-w47mb\" (UID: \"723ca0d0-33df-4714-af5e-9513f7ec0c81\") " pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:24.085655 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.085621    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:24.208565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.208529    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:24.352839 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.352812    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6474b87dd6-w47mb"]
Jul 16 22:18:24.355563 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:24.355536    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723ca0d0_33df_4714_af5e_9513f7ec0c81.slice/crio-c793c02bfa700eb24ee593bf3be0d8bbadf16e460f90dd09e28bd78846a47783 WatchSource:0}: Error finding container c793c02bfa700eb24ee593bf3be0d8bbadf16e460f90dd09e28bd78846a47783: Status 404 returned error can't find the container with id c793c02bfa700eb24ee593bf3be0d8bbadf16e460f90dd09e28bd78846a47783
Jul 16 22:18:24.495610 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.495577    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:24.679135 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.679046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6474b87dd6-w47mb" event={"ID":"723ca0d0-33df-4714-af5e-9513f7ec0c81","Type":"ContainerStarted","Data":"a4145e78d7eeac6878776cd0888d26f0307ba176710bc290c068faf8f8ba4a4c"}
Jul 16 22:18:24.679135 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.679106    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6474b87dd6-w47mb" event={"ID":"723ca0d0-33df-4714-af5e-9513f7ec0c81","Type":"ContainerStarted","Data":"c793c02bfa700eb24ee593bf3be0d8bbadf16e460f90dd09e28bd78846a47783"}
Jul 16 22:18:24.696446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:24.696382    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6474b87dd6-w47mb" podStartSLOduration=17.696362307 podStartE2EDuration="17.696362307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:24.695421276 +0000 UTC m=+274.390003168" watchObservedRunningTime="2026-07-16 22:18:24.696362307 +0000 UTC m=+274.390944162"
Jul 16 22:18:25.038453 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.038414    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.142395 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.142362    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.209503 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.209467    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:25.212603 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.212578    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:25.454682 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.454659    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.550873 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.550845    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.651202 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.651113    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:25.689151 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.689116    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5" event={"ID":"7a76bb28-0498-4c32-b0a7-63af15b35b55","Type":"ContainerStarted","Data":"e39d072741ced0ddc93c98f68f6369d1034ead8d584f6e6516a2bb002ae98930"}
Jul 16 22:18:25.689556 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.689258    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:25.689556 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.689294    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5" event={"ID":"7a76bb28-0498-4c32-b0a7-63af15b35b55","Type":"ContainerStarted","Data":"377fd7f31ad9e5eb14b393a572f2702450d26050038f0a22cd8981abcff55887"}
Jul 16 22:18:25.690485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.690466    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6474b87dd6-w47mb"
Jul 16 22:18:25.703935 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.703892    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-ffnr5" podStartSLOduration=16.852563222 podStartE2EDuration="18.703880792s" totalImagesPullingTime="1.85131757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:25.702685317 +0000 UTC m=+275.397267170" watchObservedRunningTime="2026-07-16 22:18:25.703880792 +0000 UTC m=+275.398462645"
Jul 16 22:18:25.898867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:25.898832    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.286481 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:26.286451    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.486110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:26.486051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:26.684607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:26.684523    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.092043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.092009    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.294133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.294083    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:27.301168 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.301143    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:27.301272 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.301175    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:27.301659 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.301641    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:27.303444 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.303428    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bee0c7fa-9c9e-4c2e-952a-099ac0774fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5f86\" (UID: \"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd\") " pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:27.365626 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.365540    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g5f86"
Jul 16 22:18:27.491947 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.491917    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g5f86"]
Jul 16 22:18:27.495181 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:27.495138    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee0c7fa_9c9e_4c2e_952a_099ac0774fbd.slice/crio-d6f80bae2ca9c42397a44988ca20b8fde8af9fd8d288ed46248cf3d99fe98e11 WatchSource:0}: Error finding container d6f80bae2ca9c42397a44988ca20b8fde8af9fd8d288ed46248cf3d99fe98e11: Status 404 returned error can't find the container with id d6f80bae2ca9c42397a44988ca20b8fde8af9fd8d288ed46248cf3d99fe98e11
Jul 16 22:18:27.697605 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:27.697569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5f86" event={"ID":"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd","Type":"ContainerStarted","Data":"d6f80bae2ca9c42397a44988ca20b8fde8af9fd8d288ed46248cf3d99fe98e11"}
Jul 16 22:18:30.484079 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:30.484046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:30.535845 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:30.535809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:31.379598 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:31.379544    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wr797" podUID="cc26a0a7-6613-4580-a2f6-0cd5c3577633"
Jul 16 22:18:31.712633 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:31.712600    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:32.717418 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:32.717384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5f86" event={"ID":"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd","Type":"ContainerStarted","Data":"f0e02d298b165ed03054aefacd820a95093a14be4633cf582cf47683c112f3f4"}
Jul 16 22:18:35.161404 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:35.161365    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:35.163674 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:35.163648    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc26a0a7-6613-4580-a2f6-0cd5c3577633-metrics-tls\") pod \"dns-default-wr797\" (UID: \"cc26a0a7-6613-4580-a2f6-0cd5c3577633\") " pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:35.316209 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:35.316175    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gpmss\""
Jul 16 22:18:35.324256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:35.324223    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:35.752382 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:35.752352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:36.015548 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:36.015479    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wr797"]
Jul 16 22:18:36.017928 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:36.017902    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc26a0a7_6613_4580_a2f6_0cd5c3577633.slice/crio-60aa3a4fbf6762b2247d9a6b6d8a7749f6d8217e2cf007e86dfd6df78ac3a7bc WatchSource:0}: Error finding container 60aa3a4fbf6762b2247d9a6b6d8a7749f6d8217e2cf007e86dfd6df78ac3a7bc: Status 404 returned error can't find the container with id 60aa3a4fbf6762b2247d9a6b6d8a7749f6d8217e2cf007e86dfd6df78ac3a7bc
Jul 16 22:18:36.721403 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:36.721365    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:36.732437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:36.732397    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wr797" event={"ID":"cc26a0a7-6613-4580-a2f6-0cd5c3577633","Type":"ContainerStarted","Data":"60aa3a4fbf6762b2247d9a6b6d8a7749f6d8217e2cf007e86dfd6df78ac3a7bc"}
Jul 16 22:18:36.734426 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:36.734391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5f86" event={"ID":"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd","Type":"ContainerStarted","Data":"812f33d30bbbffac67f5e79960a27b22b523634527635bfbda8df7b6d21ad785"}
Jul 16 22:18:36.824281 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:36.824228    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:37.163126 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:37.163000    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:37.438199 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:37.438123    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:37.498321 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:37.498287    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:37.598048 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:37.598022    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:37.743035 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:37.742997    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wr797" event={"ID":"cc26a0a7-6613-4580-a2f6-0cd5c3577633","Type":"ContainerStarted","Data":"8a9f02bcc2ef1e336788a64c037fe42e3ef3bc2eca5f504d416252f4e43a77fe"}
Jul 16 22:18:38.050681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.050591    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:38.156801 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.156767    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:38.250896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.250860    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:38.358993 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.358919    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:38.450565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.450535    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:38.748185 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.748150    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wr797" event={"ID":"cc26a0a7-6613-4580-a2f6-0cd5c3577633","Type":"ContainerStarted","Data":"3f61cdf354d721dcaf9b2ed1660cf1704e8094e31f1d12406077e7e454346112"}
Jul 16 22:18:38.748723 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.748258    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:38.764760 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:38.764709    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wr797" podStartSLOduration=252.194767505 podStartE2EDuration="4m13.764693505s" totalImagesPullingTime="1.569926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:38.76321864 +0000 UTC m=+288.457800504" watchObservedRunningTime="2026-07-16 22:18:38.764693505 +0000 UTC m=+288.459275414"
Jul 16 22:18:39.941650 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:39.941630    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:40.756423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:40.756383    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5f86" event={"ID":"bee0c7fa-9c9e-4c2e-952a-099ac0774fbd","Type":"ContainerStarted","Data":"d1bc0c69a0f91ba20600a88c233f8ead8e3ded458df539fd3667dd302958314e"}
Jul 16 22:18:40.772541 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:40.772495    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g5f86" podStartSLOduration=17.465675287 podStartE2EDuration="29.77247937s" totalImagesPullingTime="12.306804083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:18:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:40.771923374 +0000 UTC m=+290.466505251" watchObservedRunningTime="2026-07-16 22:18:40.77247937 +0000 UTC m=+290.467061223"
Jul 16 22:18:41.643944 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:41.643914    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:41.719391 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:41.719358    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:42.288812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:42.288780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:43.826218 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:43.826174    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vxtv5" podUID="889fca95-1e44-443e-87f4-dbf482146ac6"
Jul 16 22:18:44.337951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.337920    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:44.957046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.957016    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:18:44.989946 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.989919    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:18:44.990100 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.990012    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:44.992234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992214    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 22:18:44.992555 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992535    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 22:18:44.992697 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992549    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 22:18:44.992697 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992555    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 22:18:44.992857 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992810    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 22:18:44.993005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.992971    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 22:18:44.993119 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.993037    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 22:18:44.993233 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:44.993114    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ct4lz\""
Jul 16 22:18:45.032590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.032590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.032837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032613    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.032837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032676    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.032837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032721    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.032837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.032770    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvpg\" (UniqueName: \"kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134083 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134047    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvpg\" (UniqueName: \"kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134223    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134461 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134277    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.134911 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134887    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.135005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.134922    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.135058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.135037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.136351 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.136327    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.136634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.136613    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.141794 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.141774    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvpg\" (UniqueName: \"kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg\") pod \"console-598966bcdb-w25rp\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") " pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.299818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.299738    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:45.443670 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.443641    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:18:45.446821 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:45.446795    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97aad57e_2953_43bb_a28a_046ab02341c7.slice/crio-f7c1e42744e2ed718eb3e59bf1a3664815aba8eb269afde4ec62eb9bc60f094f WatchSource:0}: Error finding container f7c1e42744e2ed718eb3e59bf1a3664815aba8eb269afde4ec62eb9bc60f094f: Status 404 returned error can't find the container with id f7c1e42744e2ed718eb3e59bf1a3664815aba8eb269afde4ec62eb9bc60f094f
Jul 16 22:18:45.773949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.773918    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598966bcdb-w25rp" event={"ID":"97aad57e-2953-43bb-a28a-046ab02341c7","Type":"ContainerStarted","Data":"f7c1e42744e2ed718eb3e59bf1a3664815aba8eb269afde4ec62eb9bc60f094f"}
Jul 16 22:18:45.814407 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.814364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:45.955027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:45.954983    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.034316 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.034234    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"]
Jul 16 22:18:46.036672 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.036655    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.040295 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.040258    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 16 22:18:46.041557 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.041531    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 22:18:46.041766 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.041738    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fvvnt\""
Jul 16 22:18:46.041884 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.041783    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 16 22:18:46.042408 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.042387    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 22:18:46.042507 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.042426    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 22:18:46.042507 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.042484    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 16 22:18:46.062154 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.062126    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-87xpq"]
Jul 16 22:18:46.064544 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.064523    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"]
Jul 16 22:18:46.064675 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.064653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.067402 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.067378    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 22:18:46.067524 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.067427    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 22:18:46.069226 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.069203    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5n7ln\""
Jul 16 22:18:46.069361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.069342    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 22:18:46.088676 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.088651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.142437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-textfile\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142445    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-tls\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142484    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-sys\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142569    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wlst\" (UniqueName: \"kubernetes.io/projected/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-kube-api-access-6wlst\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142597    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-root\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142628    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142658    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10068efb-ca2a-45c4-a540-cf97919885c1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142680    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142733    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-wtmp\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142757    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142778    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffbds\" (UniqueName: \"kubernetes.io/projected/10068efb-ca2a-45c4-a540-cf97919885c1-kube-api-access-ffbds\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.142848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142824    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-metrics-client-ca\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.143321 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.142863    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.237143 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.237108    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.243155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243138    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-metrics-client-ca\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243170    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-textfile\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-tls\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243627 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.243730 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243658    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.243730 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243704    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-sys\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243737    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wlst\" (UniqueName: \"kubernetes.io/projected/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-kube-api-access-6wlst\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-root\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243802    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.243838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243803    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-textfile\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.244022 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-sys\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.244022 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.243958    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10068efb-ca2a-45c4-a540-cf97919885c1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.244022 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.244192 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:46.244078    2556 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 16 22:18:46.244240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-wtmp\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.244333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244211    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-metrics-client-ca\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.244333 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:46.244331    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls podName:10068efb-ca2a-45c4-a540-cf97919885c1 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:46.74413704 +0000 UTC m=+296.438718876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-sk5cz" (UID: "10068efb-ca2a-45c4-a540-cf97919885c1") : secret "kube-state-metrics-tls" not found
Jul 16 22:18:46.244463 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-wtmp\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.244463 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.244559 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.244498    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffbds\" (UniqueName: \"kubernetes.io/projected/10068efb-ca2a-45c4-a540-cf97919885c1-kube-api-access-ffbds\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.245896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.245720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10068efb-ca2a-45c4-a540-cf97919885c1-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.246004 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.245950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-root\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.246066 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.246010    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.246830 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.246808    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-accelerators-collector-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.247081 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.247055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.250953 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.249926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-tls\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.250953 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.250420    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.251691 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.251670    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.266646 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.266616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffbds\" (UniqueName: \"kubernetes.io/projected/10068efb-ca2a-45c4-a540-cf97919885c1-kube-api-access-ffbds\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.267184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.267154    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wlst\" (UniqueName: \"kubernetes.io/projected/4e880ff9-2a57-4efe-a266-9a62ef3b8b1c-kube-api-access-6wlst\") pod \"node-exporter-87xpq\" (UID: \"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c\") " pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.375447 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.375355    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-87xpq"
Jul 16 22:18:46.386413 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:46.386383    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e880ff9_2a57_4efe_a266_9a62ef3b8b1c.slice/crio-dc58150826da510be74a4641a956fbfec564c2133d726321e8b9556a536a4bfb WatchSource:0}: Error finding container dc58150826da510be74a4641a956fbfec564c2133d726321e8b9556a536a4bfb: Status 404 returned error can't find the container with id dc58150826da510be74a4641a956fbfec564c2133d726321e8b9556a536a4bfb
Jul 16 22:18:46.392989 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.392966    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.482472 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.482435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.678954 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.678865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:46.750450 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.750415    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.753248 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.753220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10068efb-ca2a-45c4-a540-cf97919885c1-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sk5cz\" (UID: \"10068efb-ca2a-45c4-a540-cf97919885c1\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:46.778695 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.778663    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87xpq" event={"ID":"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c","Type":"ContainerStarted","Data":"dc58150826da510be74a4641a956fbfec564c2133d726321e8b9556a536a4bfb"}
Jul 16 22:18:46.946209 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:46.946176    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"
Jul 16 22:18:47.074343 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.074224    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:47.120177 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.120032    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sk5cz"]
Jul 16 22:18:47.123459 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:47.123429    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10068efb_ca2a_45c4_a540_cf97919885c1.slice/crio-320d08e2c4ece2133fa8bd1172d781ca4979894d30fb9c7a4810e1d490f139e8 WatchSource:0}: Error finding container 320d08e2c4ece2133fa8bd1172d781ca4979894d30fb9c7a4810e1d490f139e8: Status 404 returned error can't find the container with id 320d08e2c4ece2133fa8bd1172d781ca4979894d30fb9c7a4810e1d490f139e8
Jul 16 22:18:47.224086 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.223964    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:47.225610 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.225490    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:47.227137 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.227119    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.229942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.229799    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 16 22:18:47.230238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.230221    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qzmfs\""
Jul 16 22:18:47.230487 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.230375    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 16 22:18:47.230675 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.230651    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 16 22:18:47.232124 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.231626    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 16 22:18:47.232124 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.231681    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 16 22:18:47.232274 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.232186    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 16 22:18:47.232274 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.232176    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 16 22:18:47.233042 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.232453    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 16 22:18:47.233042 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.232633    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 16 22:18:47.250032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.250003    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256708    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256746    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256780    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256842    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256868    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256891    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdlw7\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.256976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.257452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.256987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.257452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.257016    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.257452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.257045    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.257452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.257108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.257452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.257137    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.357985    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358145    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358238 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:47.358203    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle podName:6cdbc64d-7d7b-4d97-8fc7-667f031a5663 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:47.85817856 +0000 UTC m=+297.552760396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663") : configmap references non-existent config key: ca-bundle.crt
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358283    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358359    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358387    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358412    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358433    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdlw7\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.358684 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.358471    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.359289 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.359188    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.359739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.359448    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.362198 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.361813    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.362198 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.362067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.362198 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.362156    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.363299 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.363256    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.363841 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.363811    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.364073 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.364052    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.364180 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.364066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.364627 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.364590    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.366220 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.365059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.368062 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.368023    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdlw7\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.403914 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.403878    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:47.520656 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.520623    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:47.786133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.785919    2556 generic.go:356] "Generic (PLEG): container finished" podID="4e880ff9-2a57-4efe-a266-9a62ef3b8b1c" containerID="e497cbd3f506b7160be6974f736311796b9a833b27aaec982d83fec481257072" exitCode=0
Jul 16 22:18:47.786133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.786045    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87xpq" event={"ID":"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c","Type":"ContainerDied","Data":"e497cbd3f506b7160be6974f736311796b9a833b27aaec982d83fec481257072"}
Jul 16 22:18:47.787495 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.787471    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz" event={"ID":"10068efb-ca2a-45c4-a540-cf97919885c1","Type":"ContainerStarted","Data":"320d08e2c4ece2133fa8bd1172d781ca4979894d30fb9c7a4810e1d490f139e8"}
Jul 16 22:18:47.862431 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.862391    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:47.863479 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:47.863434    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:48.141256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:48.141171    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:18:48.753299 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:48.753269    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wr797"
Jul 16 22:18:49.030735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.030654    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.132115 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.132063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.235309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.234451    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.315134 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.313271    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:18:49.316815 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.316773    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:49.795765 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.795732    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598966bcdb-w25rp" event={"ID":"97aad57e-2953-43bb-a28a-046ab02341c7","Type":"ContainerStarted","Data":"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"}
Jul 16 22:18:49.797857 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.797825    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz" event={"ID":"10068efb-ca2a-45c4-a540-cf97919885c1","Type":"ContainerStarted","Data":"2b75e716cd43a7c1c2828a3c0495f4ffdd2bff280fb24618714f242bf977cfbb"}
Jul 16 22:18:49.797984 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.797877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz" event={"ID":"10068efb-ca2a-45c4-a540-cf97919885c1","Type":"ContainerStarted","Data":"613926671e5baa933859d278c9d8b00b9478f9d7043672d5ab528e71a2edc635"}
Jul 16 22:18:49.797984 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.797905    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz" event={"ID":"10068efb-ca2a-45c4-a540-cf97919885c1","Type":"ContainerStarted","Data":"38ca5d11ed43a84e245a52b24472e28ca74facce0896fdad3e01e4efb67c0a2f"}
Jul 16 22:18:49.799840 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.799818    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87xpq" event={"ID":"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c","Type":"ContainerStarted","Data":"aa36d1856dd24b990f05f5ae21fdcde63d356e65521b1185ffb82c1755c38c88"}
Jul 16 22:18:49.799953 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.799861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-87xpq" event={"ID":"4e880ff9-2a57-4efe-a266-9a62ef3b8b1c","Type":"ContainerStarted","Data":"52afe0f867526aab993aafb7498ae458f46693a9bda0fa9b12d151d38e20b9a7"}
Jul 16 22:18:49.800894 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.800876    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"f8c351cd7e4e15528e4157002f7d11e907314ba04dcd1ba5b27727447ba90afd"}
Jul 16 22:18:49.813432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.813397    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-598966bcdb-w25rp" podStartSLOduration=2.109996514 podStartE2EDuration="5.813384274s" totalImagesPullingTime="3.70338776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:49.812280691 +0000 UTC m=+299.506862546" watchObservedRunningTime="2026-07-16 22:18:49.813384274 +0000 UTC m=+299.507966128"
Jul 16 22:18:49.830181 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.830135    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-87xpq" podStartSLOduration=2.708039362 podStartE2EDuration="3.830119962s" totalImagesPullingTime="1.049021583s" totalInitContainerRuntime="73.059017ms" isStatefulPod=true podCreationTimestamp="2026-07-16 22:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:49.829565752 +0000 UTC m=+299.524147640" watchObservedRunningTime="2026-07-16 22:18:49.830119962 +0000 UTC m=+299.524701815"
Jul 16 22:18:49.845590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:49.845544    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-sk5cz" podStartSLOduration=1.818047898 podStartE2EDuration="3.845529974s" totalImagesPullingTime="2.027482076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:49.844754793 +0000 UTC m=+299.539336666" watchObservedRunningTime="2026-07-16 22:18:49.845529974 +0000 UTC m=+299.540111874"
Jul 16 22:18:50.699050 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.699021    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 22:18:50.806128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.806016    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567" exitCode=0
Jul 16 22:18:50.806297 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.806121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567"}
Jul 16 22:18:50.820590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.820561    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:50.831005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.830874    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"]
Jul 16 22:18:50.838824 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.834534    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:50.838824 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.836566    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 16 22:18:50.838824 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.836937    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-x4fh2\""
Jul 16 22:18:50.844108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.844065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"]
Jul 16 22:18:50.889852 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.889818    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:50.890152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.890125    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5szv\" (UniqueName: \"kubernetes.io/projected/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-kube-api-access-x5szv\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:50.924591 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.924560    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:50.991282 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.991253    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:50.991406 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:50.991297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5szv\" (UniqueName: \"kubernetes.io/projected/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-kube-api-access-x5szv\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:50.991444 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:50.991422    2556 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 16 22:18:50.991507 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:18:50.991495    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert podName:43fa0815-a91f-40a2-9e4d-f7ac88bc8324 nodeName:}" failed. No retries permitted until 2026-07-16 22:18:51.491473377 +0000 UTC m=+301.186055221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-r9vdc" (UID: "43fa0815-a91f-40a2-9e4d-f7ac88bc8324") : secret "monitoring-plugin-cert" not found
Jul 16 22:18:51.001941 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.001915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5szv\" (UniqueName: \"kubernetes.io/projected/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-kube-api-access-x5szv\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:51.197807 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.197780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:51.495436 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.495343    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:51.497781 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.497754    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/43fa0815-a91f-40a2-9e4d-f7ac88bc8324-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-r9vdc\" (UID: \"43fa0815-a91f-40a2-9e4d-f7ac88bc8324\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:51.744376 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.744339    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:51.878754 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.878725    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"]
Jul 16 22:18:51.881768 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:51.881726    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fa0815_a91f_40a2_9e4d_f7ac88bc8324.slice/crio-1d5bbcfab4ce82865316f81bd604ea4d52f42356b181e5fd409c05196b5f93c1 WatchSource:0}: Error finding container 1d5bbcfab4ce82865316f81bd604ea4d52f42356b181e5fd409c05196b5f93c1: Status 404 returned error can't find the container with id 1d5bbcfab4ce82865316f81bd604ea4d52f42356b181e5fd409c05196b5f93c1
Jul 16 22:18:51.884601 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:51.884574    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:18:52.306627 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.306582    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:18:52.311920 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.311893    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.313891 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.313713    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 22:18:52.314429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.314129    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 22:18:52.314429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.314237    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7vr69\""
Jul 16 22:18:52.314429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.314330    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 22:18:52.315347 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.314765    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 22:18:52.315347 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.314986    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 22:18:52.315347 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.315197    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cov6iqg6h8526\""
Jul 16 22:18:52.320650 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.320370    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 22:18:52.320650 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.320475    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 22:18:52.320818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.320802    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 22:18:52.320878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.320801    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 22:18:52.321078 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.321060    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 22:18:52.321176 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.321084    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 22:18:52.321234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.321217    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 22:18:52.321642 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.321600    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 22:18:52.322263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.321967    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 22:18:52.323418 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.323236    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 22:18:52.324032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.324005    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:52.335284 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.335213    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:18:52.394636 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.394601    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403625    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403822    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403893    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.403965    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404204    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404226    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404286    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zff6w\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404330    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404372    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.404851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.405787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.405787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404487    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.405787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404552    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.405787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.404574    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505080 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505119    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505148    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505175    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505243    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505268    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505353    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505379    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505500    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505545    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505579    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505633    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505656    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.505804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.505701    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zff6w\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.506845 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.506603    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.507215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.507187    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.508362 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.508179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.508660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.508630    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.509222 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.509186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.509802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.509720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.510192 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.509988    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.510643 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.510588    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.512601 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.512566    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.514819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.512921    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.514819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.513047    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.514819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.514509    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.515031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.514830    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.515755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.515693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.516870 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.516791    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.518155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.517811    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.518155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.517914    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.519150 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.519063    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.519925 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.519768    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zff6w\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.522503 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.522448    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.629686 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.629662    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:18:52.781714 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.781649    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:18:52.785929 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:18:52.785904    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a21dd3_bc68_4ea4_bca7_2cf182bdd19d.slice/crio-322f43f76aba06c122887d97492ce4a8a59e6bf7320743b0d594c5187c449f79 WatchSource:0}: Error finding container 322f43f76aba06c122887d97492ce4a8a59e6bf7320743b0d594c5187c449f79: Status 404 returned error can't find the container with id 322f43f76aba06c122887d97492ce4a8a59e6bf7320743b0d594c5187c449f79
Jul 16 22:18:52.817008 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.816978    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"322f43f76aba06c122887d97492ce4a8a59e6bf7320743b0d594c5187c449f79"}
Jul 16 22:18:52.821046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.820915    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"}
Jul 16 22:18:52.821046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.820973    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"}
Jul 16 22:18:52.821046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.820996    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"}
Jul 16 22:18:52.821046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.821018    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"}
Jul 16 22:18:52.821897 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:52.821878    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc" event={"ID":"43fa0815-a91f-40a2-9e4d-f7ac88bc8324","Type":"ContainerStarted","Data":"1d5bbcfab4ce82865316f81bd604ea4d52f42356b181e5fd409c05196b5f93c1"}
Jul 16 22:18:53.378449 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.378412    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:53.479565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.479540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:53.550660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.550621    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:53.830153 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.830129    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"}
Jul 16 22:18:53.831696 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.831673    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e" exitCode=0
Jul 16 22:18:53.831775 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:53.831708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e"}
Jul 16 22:18:54.539002 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.538961    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:54.597220 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.597187    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:54.838830 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.838742    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerStarted","Data":"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"}
Jul 16 22:18:54.840484 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.840461    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc" event={"ID":"43fa0815-a91f-40a2-9e4d-f7ac88bc8324","Type":"ContainerStarted","Data":"6c787a7460a100aad537464a135c3cf85592913a173f36b1f749ea9702791f9a"}
Jul 16 22:18:54.840638 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.840621    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:54.846838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.846814    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc"
Jul 16 22:18:54.864414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.864365    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.969224229 podStartE2EDuration="7.864347232s" totalImagesPullingTime="3.878429333s" totalInitContainerRuntime="16.69367ms" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:54.862582626 +0000 UTC m=+304.557164498" watchObservedRunningTime="2026-07-16 22:18:54.864347232 +0000 UTC m=+304.558929086"
Jul 16 22:18:54.878150 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:54.878074    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-r9vdc" podStartSLOduration=2.94437605 podStartE2EDuration="4.878054465s" totalImagesPullingTime="1.933678415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:54.876638859 +0000 UTC m=+304.571220735" watchObservedRunningTime="2026-07-16 22:18:54.878054465 +0000 UTC m=+304.572636321"
Jul 16 22:18:55.116390 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.116312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:55.173984 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.173954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:55.300841 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.300805    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:55.301014 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.300884    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:55.310259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.310234    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:55.849499 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:55.849473    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:18:56.144569 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:56.144475    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:56.198668 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:56.198636    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:56.516225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:56.516189    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:56.571352 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:56.571318    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:18:57.823612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:57.823577    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:18:57.857894 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:57.857857    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"}
Jul 16 22:18:57.858051 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:57.857940    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"}
Jul 16 22:18:59.869507 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:59.869475    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"}
Jul 16 22:18:59.869860 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:59.869535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"}
Jul 16 22:18:59.869860 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:59.869561    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"}
Jul 16 22:18:59.869860 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:59.869587    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerStarted","Data":"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"}
Jul 16 22:18:59.898146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:18:59.898076    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.751048345 podStartE2EDuration="7.898056994s" totalImagesPullingTime="5.126014893s" totalInitContainerRuntime="20.993756ms" isStatefulPod=false podCreationTimestamp="2026-07-16 22:18:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:18:59.895566724 +0000 UTC m=+309.590148595" watchObservedRunningTime="2026-07-16 22:18:59.898056994 +0000 UTC m=+309.592638848"
Jul 16 22:19:00.494321 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:00.494285    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:00.596647 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:00.596613    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:01.357744 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:01.357717    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:01.467346 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:01.467311    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:01.590607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:01.590573    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:01.654365 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:01.654294    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:02.630521 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:02.630481    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:19:02.766260 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:02.766229    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:02.833347 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:02.833314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:05.746436 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:05.746405    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:05.809191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:05.809157    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:08.161837 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.161805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:08.224538 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.224492    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:08.398305 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.398275    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:19:08.421645 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.421576    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:19:08.421782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.421705    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.429649 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.429624    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 22:19:08.559469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559427    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559470    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6f7z\" (UniqueName: \"kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559508    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559531    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559575    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.559782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.559630    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660464 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660464 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660461    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6f7z\" (UniqueName: \"kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660497    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660587    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.660706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.660613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.661309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.661280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.661414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.661281    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.661414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.661345    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.661414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.661408    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.663604 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.663577    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.663604 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.663593    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.669182 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.669151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6f7z\" (UniqueName: \"kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z\") pod \"console-864d9cd78-dv447\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") " pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.732582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.732543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:08.828701 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.828670    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:08.875381 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.875346    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:19:08.878600 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:19:08.878572    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4df4ee_d612_4008_9ad3_90d20046b4d3.slice/crio-5c4b2598b95d8e6abb041fc8bdce4c8d0bebc172e20f048258a7af0c2d99eaa8 WatchSource:0}: Error finding container 5c4b2598b95d8e6abb041fc8bdce4c8d0bebc172e20f048258a7af0c2d99eaa8: Status 404 returned error can't find the container with id 5c4b2598b95d8e6abb041fc8bdce4c8d0bebc172e20f048258a7af0c2d99eaa8
Jul 16 22:19:08.905603 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.905576    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d9cd78-dv447" event={"ID":"aa4df4ee-d612-4008-9ad3-90d20046b4d3","Type":"ContainerStarted","Data":"5c4b2598b95d8e6abb041fc8bdce4c8d0bebc172e20f048258a7af0c2d99eaa8"}
Jul 16 22:19:08.927697 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:08.927674    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:09.911064 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:09.911031    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d9cd78-dv447" event={"ID":"aa4df4ee-d612-4008-9ad3-90d20046b4d3","Type":"ContainerStarted","Data":"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"}
Jul 16 22:19:09.927763 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:09.927709    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864d9cd78-dv447" podStartSLOduration=1.927693186 podStartE2EDuration="1.927693186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:19:09.925876968 +0000 UTC m=+319.620458821" watchObservedRunningTime="2026-07-16 22:19:09.927693186 +0000 UTC m=+319.622275040"
Jul 16 22:19:15.486724 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:15.486689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:15.586727 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:15.586695    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:17.851365 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:17.851332    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:17.951950 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:17.951919    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:18.733065 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:18.733027    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:18.733279 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:18.733134    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:18.737964 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:18.737942    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:18.950762 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:18.950733    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:19:18.998070 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:18.997989    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:19:19.099461 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:19.099431    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:19.166232 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:19.166182    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:19.226868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:19.226835    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:26.982939 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:26.982904    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ba01bc-9376-48d0-b96f-80ee68024cff" containerID="7a0108614cceb8bd881e4ab28422992010b08ee32d4677ff5b87890bb57a0a5b" exitCode=0
Jul 16 22:19:26.983343 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:26.982948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs" event={"ID":"35ba01bc-9376-48d0-b96f-80ee68024cff","Type":"ContainerDied","Data":"7a0108614cceb8bd881e4ab28422992010b08ee32d4677ff5b87890bb57a0a5b"}
Jul 16 22:19:26.983343 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:26.983269    2556 scope.go:117] "RemoveContainer" containerID="7a0108614cceb8bd881e4ab28422992010b08ee32d4677ff5b87890bb57a0a5b"
Jul 16 22:19:27.988194 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:27.988160    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t5xcs" event={"ID":"35ba01bc-9376-48d0-b96f-80ee68024cff","Type":"ContainerStarted","Data":"3d3978444a984d81d2f682414a57da46af6565c6bfa4d71e5a76ccbde9b6ba7d"}
Jul 16 22:19:30.489302 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:30.489268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:30.586744 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:30.586711    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:19:41.045216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:41.045130    2556 generic.go:356] "Generic (PLEG): container finished" podID="a4838512-aba9-4b60-aac7-154dd2d3dfdc" containerID="cfd91ba18ae1c6353a5f9c8433709e341536049ff155ce512bf3d8ff2f981f4c" exitCode=0
Jul 16 22:19:41.045590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:41.045213    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v" event={"ID":"a4838512-aba9-4b60-aac7-154dd2d3dfdc","Type":"ContainerDied","Data":"cfd91ba18ae1c6353a5f9c8433709e341536049ff155ce512bf3d8ff2f981f4c"}
Jul 16 22:19:41.045590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:41.045511    2556 scope.go:117] "RemoveContainer" containerID="cfd91ba18ae1c6353a5f9c8433709e341536049ff155ce512bf3d8ff2f981f4c"
Jul 16 22:19:42.051167 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:42.051132    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-mln2v" event={"ID":"a4838512-aba9-4b60-aac7-154dd2d3dfdc","Type":"ContainerStarted","Data":"55c6b21a4318657dbd670e3b56f8267f7fcb8a1a71d25cd649a24cd449c94c19"}
Jul 16 22:19:44.026391 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.026334    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-598966bcdb-w25rp" podUID="97aad57e-2953-43bb-a28a-046ab02341c7" containerName="console" containerID="cri-o://7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467" gracePeriod=15
Jul 16 22:19:44.279407 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.279354    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:19:44.358974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.358948    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359195 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.358992    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359195 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359015    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359195 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359035    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359195 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359108    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcvpg\" (UniqueName: \"kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359195 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359142    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config\") pod \"97aad57e-2953-43bb-a28a-046ab02341c7\" (UID: \"97aad57e-2953-43bb-a28a-046ab02341c7\") "
Jul 16 22:19:44.359461 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359432    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca" (OuterVolumeSpecName: "service-ca") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:44.359535 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359458    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:44.359535 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.359520    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config" (OuterVolumeSpecName: "console-config") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:19:44.361414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.361381    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:44.361414 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.361402    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:19:44.361544 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.361468    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg" (OuterVolumeSpecName: "kube-api-access-gcvpg") pod "97aad57e-2953-43bb-a28a-046ab02341c7" (UID: "97aad57e-2953-43bb-a28a-046ab02341c7"). InnerVolumeSpecName "kube-api-access-gcvpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:19:44.460678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460639    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcvpg\" (UniqueName: \"kubernetes.io/projected/97aad57e-2953-43bb-a28a-046ab02341c7-kube-api-access-gcvpg\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:44.460678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460670    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-console-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:44.460678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460681    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:44.460678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460689    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97aad57e-2953-43bb-a28a-046ab02341c7-console-oauth-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:44.460932 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460697    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:44.460932 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:44.460705    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97aad57e-2953-43bb-a28a-046ab02341c7-oauth-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:19:45.063639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.063606    2556 generic.go:356] "Generic (PLEG): container finished" podID="97aad57e-2953-43bb-a28a-046ab02341c7" containerID="7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467" exitCode=0
Jul 16 22:19:45.064068 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.063652    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598966bcdb-w25rp"
Jul 16 22:19:45.064068 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.063667    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598966bcdb-w25rp" event={"ID":"97aad57e-2953-43bb-a28a-046ab02341c7","Type":"ContainerDied","Data":"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"}
Jul 16 22:19:45.064068 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.063705    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598966bcdb-w25rp" event={"ID":"97aad57e-2953-43bb-a28a-046ab02341c7","Type":"ContainerDied","Data":"f7c1e42744e2ed718eb3e59bf1a3664815aba8eb269afde4ec62eb9bc60f094f"}
Jul 16 22:19:45.064068 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.063736    2556 scope.go:117] "RemoveContainer" containerID="7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"
Jul 16 22:19:45.072036 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.072016    2556 scope.go:117] "RemoveContainer" containerID="7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"
Jul 16 22:19:45.072332 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:19:45.072311    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467\": container with ID starting with 7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467 not found: ID does not exist" containerID="7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"
Jul 16 22:19:45.072387 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.072340    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467"} err="failed to get container status \"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467\": rpc error: code = NotFound desc = could not find container \"7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467\": container with ID starting with 7fa78f9230f0fbf95a657d948cdd8ead857e7940d04ce84a998013c769798467 not found: ID does not exist"
Jul 16 22:19:45.077582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.077558    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:19:45.081218 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:45.081198    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-598966bcdb-w25rp"]
Jul 16 22:19:46.828612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:46.828579    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97aad57e-2953-43bb-a28a-046ab02341c7" path="/var/lib/kubelet/pods/97aad57e-2953-43bb-a28a-046ab02341c7/volumes"
Jul 16 22:19:52.630512 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:52.630477    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:19:52.646465 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:52.646435    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:19:53.113715 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:19:53.113685    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:01.780352 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:01.780315    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:01.857951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:01.857922    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:02.231139 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:02.231110    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:02.330654 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:02.330618    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:04.543891 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:04.543849    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:20:04.546134 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:04.546106    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/889fca95-1e44-443e-87f4-dbf482146ac6-metrics-certs\") pod \"network-metrics-daemon-vxtv5\" (UID: \"889fca95-1e44-443e-87f4-dbf482146ac6\") " pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:20:04.726057 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:04.726025    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7sbmm\""
Jul 16 22:20:04.734348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:04.734322    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vxtv5"
Jul 16 22:20:04.867542 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:04.867513    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vxtv5"]
Jul 16 22:20:04.871028 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:20:04.871003    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889fca95_1e44_443e_87f4_dbf482146ac6.slice/crio-017ddfeb6ebd44068d029329a14b8e668481cb9662ea65287526a77763c61f97 WatchSource:0}: Error finding container 017ddfeb6ebd44068d029329a14b8e668481cb9662ea65287526a77763c61f97: Status 404 returned error can't find the container with id 017ddfeb6ebd44068d029329a14b8e668481cb9662ea65287526a77763c61f97
Jul 16 22:20:05.144723 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:05.144635    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vxtv5" event={"ID":"889fca95-1e44-443e-87f4-dbf482146ac6","Type":"ContainerStarted","Data":"017ddfeb6ebd44068d029329a14b8e668481cb9662ea65287526a77763c61f97"}
Jul 16 22:20:06.150639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.150555    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vxtv5" event={"ID":"889fca95-1e44-443e-87f4-dbf482146ac6","Type":"ContainerStarted","Data":"0290a82cf0541fd380bdccf85c1ac05dd324576e3abb0a4149b60ced9d5112f0"}
Jul 16 22:20:06.150639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.150603    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vxtv5" event={"ID":"889fca95-1e44-443e-87f4-dbf482146ac6","Type":"ContainerStarted","Data":"9cffb6484924ebe657b2ed16c7e5fb67c726b40ef3f6aa35b08fd2cb3973f45c"}
Jul 16 22:20:06.169362 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.169306    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vxtv5" podStartSLOduration=375.271242918 podStartE2EDuration="6m16.169288015s" totalImagesPullingTime="898.045097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:13:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:06.167340295 +0000 UTC m=+375.861922170" watchObservedRunningTime="2026-07-16 22:20:06.169288015 +0000 UTC m=+375.863869866"
Jul 16 22:20:06.271442 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.271414    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:06.341049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.341020    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:06.508694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.508656    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:20:06.508994 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.508954    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="alertmanager" containerID="cri-o://8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4" gracePeriod=120
Jul 16 22:20:06.509065 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.509018    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-metric" containerID="cri-o://f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c" gracePeriod=120
Jul 16 22:20:06.509161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.509055    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-web" containerID="cri-o://0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a" gracePeriod=120
Jul 16 22:20:06.509161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.509062    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="prom-label-proxy" containerID="cri-o://ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a" gracePeriod=120
Jul 16 22:20:06.509161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.509101    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="config-reloader" containerID="cri-o://02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6" gracePeriod=120
Jul 16 22:20:06.509161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:06.509150    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy" containerID="cri-o://06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894" gracePeriod=120
Jul 16 22:20:07.157667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157630    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a" exitCode=0
Jul 16 22:20:07.157667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157661    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c" exitCode=0
Jul 16 22:20:07.157667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157667    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894" exitCode=0
Jul 16 22:20:07.157667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157673    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6" exitCode=0
Jul 16 22:20:07.157667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157678    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4" exitCode=0
Jul 16 22:20:07.158191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157702    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"}
Jul 16 22:20:07.158191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157770    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"}
Jul 16 22:20:07.158191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157797    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"}
Jul 16 22:20:07.158191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157821    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"}
Jul 16 22:20:07.158191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.157843    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"}
Jul 16 22:20:07.761461 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.761435    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:20:07.873246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873154    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873195    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdlw7\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873216    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873245    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873356    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873389    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873432    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873491    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873521    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873557    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873585    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873621    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873661    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") pod \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\" (UID: \"6cdbc64d-7d7b-4d97-8fc7-667f031a5663\") "
Jul 16 22:20:07.873774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873689    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:07.874031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.873926    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-main-db\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.874594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.874283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:07.874594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.874553    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:07.877240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877112    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.877240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877121    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7" (OuterVolumeSpecName: "kube-api-access-bdlw7") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "kube-api-access-bdlw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:07.877240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877258    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.877240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877287    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.877240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877482    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume" (OuterVolumeSpecName: "config-volume") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.877819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877553    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:07.877819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877699    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.877928 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.877909    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out" (OuterVolumeSpecName: "config-out") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:07.881366 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.880996    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.888348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.888317    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config" (OuterVolumeSpecName: "web-config") pod "6cdbc64d-7d7b-4d97-8fc7-667f031a5663" (UID: "6cdbc64d-7d7b-4d97-8fc7-667f031a5663"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:07.975268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975212    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-metrics-client-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975261    2556 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-volume\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975270    2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-config-out\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975278    2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-tls-assets\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975288    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975298    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975307    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975316    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdlw7\" (UniqueName: \"kubernetes.io/projected/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-kube-api-access-bdlw7\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975325    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-main-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975333    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975342    2556 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-cluster-tls-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:07.975561 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:07.975350    2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cdbc64d-7d7b-4d97-8fc7-667f031a5663-web-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:08.165063 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.164983    2556 generic.go:356] "Generic (PLEG): container finished" podID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerID="0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a" exitCode=0
Jul 16 22:20:08.165063 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.165026    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"}
Jul 16 22:20:08.165518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.165071    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 16 22:20:08.165518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.165081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cdbc64d-7d7b-4d97-8fc7-667f031a5663","Type":"ContainerDied","Data":"f8c351cd7e4e15528e4157002f7d11e907314ba04dcd1ba5b27727447ba90afd"}
Jul 16 22:20:08.165518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.165141    2556 scope.go:117] "RemoveContainer" containerID="ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"
Jul 16 22:20:08.173587 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.173567    2556 scope.go:117] "RemoveContainer" containerID="f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"
Jul 16 22:20:08.181977 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.181950    2556 scope.go:117] "RemoveContainer" containerID="06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"
Jul 16 22:20:08.189121 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.189060    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:20:08.191433 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.191409    2556 scope.go:117] "RemoveContainer" containerID="0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"
Jul 16 22:20:08.193136 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.193069    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 16 22:20:08.200023 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.200005    2556 scope.go:117] "RemoveContainer" containerID="02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"
Jul 16 22:20:08.207591 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.207573    2556 scope.go:117] "RemoveContainer" containerID="8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"
Jul 16 22:20:08.216469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.216451    2556 scope.go:117] "RemoveContainer" containerID="f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567"
Jul 16 22:20:08.226053 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.226030    2556 scope.go:117] "RemoveContainer" containerID="ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"
Jul 16 22:20:08.226744 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.226720    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a\": container with ID starting with ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a not found: ID does not exist" containerID="ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"
Jul 16 22:20:08.226816 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.226760    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a"} err="failed to get container status \"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a\": rpc error: code = NotFound desc = could not find container \"ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a\": container with ID starting with ecb9bcd51ace50c3fbc19f32c8afa2e8764ee7e7588ab0d0639c6c740dfb576a not found: ID does not exist"
Jul 16 22:20:08.226816 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.226789    2556 scope.go:117] "RemoveContainer" containerID="f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"
Jul 16 22:20:08.227081 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.227061    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c\": container with ID starting with f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c not found: ID does not exist" containerID="f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"
Jul 16 22:20:08.227296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227102    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c"} err="failed to get container status \"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c\": rpc error: code = NotFound desc = could not find container \"f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c\": container with ID starting with f2ddfc44ef6ac838494b6fd42b08a14ea3377e08d97aac12c7fd339e491e8e9c not found: ID does not exist"
Jul 16 22:20:08.227296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227125    2556 scope.go:117] "RemoveContainer" containerID="06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"
Jul 16 22:20:08.227397 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.227369    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894\": container with ID starting with 06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894 not found: ID does not exist" containerID="06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"
Jul 16 22:20:08.227452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227395    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894"} err="failed to get container status \"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894\": rpc error: code = NotFound desc = could not find container \"06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894\": container with ID starting with 06237b0c91901528c46139483e994e8ca865a7f64b28922008c67cfc3b91a894 not found: ID does not exist"
Jul 16 22:20:08.227452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227415    2556 scope.go:117] "RemoveContainer" containerID="0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"
Jul 16 22:20:08.227668 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.227647    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a\": container with ID starting with 0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a not found: ID does not exist" containerID="0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"
Jul 16 22:20:08.227729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227676    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a"} err="failed to get container status \"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a\": rpc error: code = NotFound desc = could not find container \"0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a\": container with ID starting with 0a4bef3425f1bae8648ba7862f6cc021770f704fdde1d517e9e3acaa37c75f8a not found: ID does not exist"
Jul 16 22:20:08.227729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227699    2556 scope.go:117] "RemoveContainer" containerID="02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"
Jul 16 22:20:08.227960 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.227943    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6\": container with ID starting with 02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6 not found: ID does not exist" containerID="02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"
Jul 16 22:20:08.227997 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227965    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6"} err="failed to get container status \"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6\": rpc error: code = NotFound desc = could not find container \"02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6\": container with ID starting with 02cdf08d4e20a9925a021950dc528d77777d1b34b2ab46eb6f2491c5e9904ad6 not found: ID does not exist"
Jul 16 22:20:08.227997 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.227978    2556 scope.go:117] "RemoveContainer" containerID="8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"
Jul 16 22:20:08.228212 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.228194    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4\": container with ID starting with 8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4 not found: ID does not exist" containerID="8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"
Jul 16 22:20:08.228259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.228217    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4"} err="failed to get container status \"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4\": rpc error: code = NotFound desc = could not find container \"8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4\": container with ID starting with 8afb15f47a9b96a906ea00be0043de7af6613892b8ac0ac4695e241c26eb0dc4 not found: ID does not exist"
Jul 16 22:20:08.228259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.228233    2556 scope.go:117] "RemoveContainer" containerID="f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567"
Jul 16 22:20:08.228456 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:08.228438    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567\": container with ID starting with f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567 not found: ID does not exist" containerID="f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567"
Jul 16 22:20:08.228499 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.228461    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567"} err="failed to get container status \"f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567\": rpc error: code = NotFound desc = could not find container \"f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567\": container with ID starting with f5ad8a1ff789b3d64c26affc96ed9761e6b0e75c7c7a78397c822c57dbdba567 not found: ID does not exist"
Jul 16 22:20:08.828668 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:08.828634    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" path="/var/lib/kubelet/pods/6cdbc64d-7d7b-4d97-8fc7-667f031a5663/volumes"
Jul 16 22:20:09.939275 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:09.939244    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:10.000352 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.000317    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:10.753950 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.753916    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:10.754311 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754273    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="prometheus" containerID="cri-o://b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4" gracePeriod=600
Jul 16 22:20:10.754311 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754300    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab" gracePeriod=600
Jul 16 22:20:10.754513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754299    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="thanos-sidecar" containerID="cri-o://4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64" gracePeriod=600
Jul 16 22:20:10.754513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754330    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-web" containerID="cri-o://4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e" gracePeriod=600
Jul 16 22:20:10.754513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754347    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="config-reloader" containerID="cri-o://d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c" gracePeriod=600
Jul 16 22:20:10.754513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:10.754275    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy" containerID="cri-o://651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10" gracePeriod=600
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181123    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab" exitCode=0
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181147    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10" exitCode=0
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181154    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64" exitCode=0
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181160    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c" exitCode=0
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181165    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4" exitCode=0
Jul 16 22:20:11.181206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"}
Jul 16 22:20:11.181693 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"}
Jul 16 22:20:11.181693 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181314    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"}
Jul 16 22:20:11.181693 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181339    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"}
Jul 16 22:20:11.181693 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:11.181363    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"}
Jul 16 22:20:12.016942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.016917    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.110056 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.109974    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110056 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110020    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110056 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110039    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110066    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110105    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110120    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110134    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110348 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110157    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110458    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110479    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.110600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110526    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110576    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110606    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110642    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110666    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110711    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110743    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.110804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110765    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.111110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.111110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110839    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zff6w\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.111110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110881    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\" (UID: \"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d\") "
Jul 16 22:20:12.111110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.110887    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.111330 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.111160    2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.111330 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.111180    2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.112573 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.111778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.112573 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.112163    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.112936 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.112902    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.113067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.113031    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.113257 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.113219    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.113390 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.113271    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:12.113849 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.113817    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:12.114328 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.114291    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out" (OuterVolumeSpecName: "config-out") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:20:12.114328 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.114301    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.114460 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.114394    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config" (OuterVolumeSpecName: "config") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.114654 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.114627    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.114989 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.114961    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.115108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.115020    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:12.115548 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.115522    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.115631 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.115601    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.115703 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.115682    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w" (OuterVolumeSpecName: "kube-api-access-zff6w") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "kube-api-access-zff6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:12.116250 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.116231    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.125078 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.125056    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config" (OuterVolumeSpecName: "web-config") pod "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" (UID: "d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:12.188066 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.188030    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerID="4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e" exitCode=0
Jul 16 22:20:12.188552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.188112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"}
Jul 16 22:20:12.188552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.188125    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.188552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.188182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d","Type":"ContainerDied","Data":"322f43f76aba06c122887d97492ce4a8a59e6bf7320743b0d594c5187c449f79"}
Jul 16 22:20:12.188552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.188217    2556 scope.go:117] "RemoveContainer" containerID="330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"
Jul 16 22:20:12.196844 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.196738    2556 scope.go:117] "RemoveContainer" containerID="651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"
Jul 16 22:20:12.203768 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.203752    2556 scope.go:117] "RemoveContainer" containerID="4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"
Jul 16 22:20:12.210144 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.210129    2556 scope.go:117] "RemoveContainer" containerID="4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"
Jul 16 22:20:12.211809 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211585    2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config-out\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.211809 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211786    2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.211809 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211803    2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211817    2556 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-metrics-client-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211828    2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-web-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211840    2556 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-kube-rbac-proxy\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211853    2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-tls-assets\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211864    2556 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211872    2556 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211881    2556 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-metrics-client-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211890    2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211899    2556 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211907    2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-db\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211919    2556 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-grpc-tls\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211932    2556 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211946    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zff6w\" (UniqueName: \"kubernetes.io/projected/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-kube-api-access-zff6w\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211955    2556 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.211964    2556 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:12.212946 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.212930    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:12.217629 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.217609    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:12.217991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.217975    2556 scope.go:117] "RemoveContainer" containerID="d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"
Jul 16 22:20:12.224573 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.224557    2556 scope.go:117] "RemoveContainer" containerID="b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"
Jul 16 22:20:12.231454 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.231438    2556 scope.go:117] "RemoveContainer" containerID="21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e"
Jul 16 22:20:12.237948 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.237930    2556 scope.go:117] "RemoveContainer" containerID="330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"
Jul 16 22:20:12.238175 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.238155    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab\": container with ID starting with 330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab not found: ID does not exist" containerID="330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"
Jul 16 22:20:12.238272 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238182    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab"} err="failed to get container status \"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab\": rpc error: code = NotFound desc = could not find container \"330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab\": container with ID starting with 330ab5a2e7371f0ac8c201cd1da16939bf4e002f04f115cead86739d4fb989ab not found: ID does not exist"
Jul 16 22:20:12.238272 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238203    2556 scope.go:117] "RemoveContainer" containerID="651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"
Jul 16 22:20:12.238457 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.238440    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10\": container with ID starting with 651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10 not found: ID does not exist" containerID="651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"
Jul 16 22:20:12.238498 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238462    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10"} err="failed to get container status \"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10\": rpc error: code = NotFound desc = could not find container \"651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10\": container with ID starting with 651f971bc3669a3cc572bb47f8667431721ca2e9e15b698945c4dbab93361a10 not found: ID does not exist"
Jul 16 22:20:12.238498 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238479    2556 scope.go:117] "RemoveContainer" containerID="4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"
Jul 16 22:20:12.238720 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.238706    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e\": container with ID starting with 4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e not found: ID does not exist" containerID="4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"
Jul 16 22:20:12.238767 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238751    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e"} err="failed to get container status \"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e\": rpc error: code = NotFound desc = could not find container \"4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e\": container with ID starting with 4226fecdcbbde98118a74485cc18333df2dacebc727b663f36ce5011eb77401e not found: ID does not exist"
Jul 16 22:20:12.238767 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238764    2556 scope.go:117] "RemoveContainer" containerID="4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"
Jul 16 22:20:12.238975 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.238959    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64\": container with ID starting with 4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64 not found: ID does not exist" containerID="4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"
Jul 16 22:20:12.239013 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.238978    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64"} err="failed to get container status \"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64\": rpc error: code = NotFound desc = could not find container \"4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64\": container with ID starting with 4ad56c3988716c8a5b6f6a857fa50e7cb54ea29bbe25c67f37f1b9965594eb64 not found: ID does not exist"
Jul 16 22:20:12.239013 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239002    2556 scope.go:117] "RemoveContainer" containerID="d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"
Jul 16 22:20:12.239236 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.239214    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c\": container with ID starting with d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c not found: ID does not exist" containerID="d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"
Jul 16 22:20:12.239304 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239238    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c"} err="failed to get container status \"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c\": rpc error: code = NotFound desc = could not find container \"d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c\": container with ID starting with d1badec59429f23f81361541a669b91892425e3e6863078813cfa293bb778b6c not found: ID does not exist"
Jul 16 22:20:12.239304 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239250    2556 scope.go:117] "RemoveContainer" containerID="b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"
Jul 16 22:20:12.239462 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.239447    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4\": container with ID starting with b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4 not found: ID does not exist" containerID="b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"
Jul 16 22:20:12.239501 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239464    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4"} err="failed to get container status \"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4\": rpc error: code = NotFound desc = could not find container \"b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4\": container with ID starting with b0438da3167afe52c5f9aa6f6cd36c5c7e975cb5d9c26d40808bd1bb2fc791d4 not found: ID does not exist"
Jul 16 22:20:12.239501 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239475    2556 scope.go:117] "RemoveContainer" containerID="21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e"
Jul 16 22:20:12.239687 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:12.239673    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e\": container with ID starting with 21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e not found: ID does not exist" containerID="21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e"
Jul 16 22:20:12.239723 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.239690    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e"} err="failed to get container status \"21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e\": rpc error: code = NotFound desc = could not find container \"21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e\": container with ID starting with 21d292e0a306b479cf8e0923f6d0182a687635d954fd89043c014a51c605ce2e not found: ID does not exist"
Jul 16 22:20:12.247546 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247524    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:12.247818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247806    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247820    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247832    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97aad57e-2953-43bb-a28a-046ab02341c7" containerName="console"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247839    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="97aad57e-2953-43bb-a28a-046ab02341c7" containerName="console"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247845    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="prometheus"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247851    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="prometheus"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247858    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="alertmanager"
Jul 16 22:20:12.247867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247862    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="alertmanager"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247871    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-thanos"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247880    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-thanos"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247888    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247894    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247903    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="prom-label-proxy"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247908    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="prom-label-proxy"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247915    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247920    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247927    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-metric"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247932    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-metric"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247936    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247942    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247956    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247962    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247968    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="thanos-sidecar"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247973    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="thanos-sidecar"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247977    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247982    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247990    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="init-config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.247995    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="init-config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248001    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="init-config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248006    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="init-config-reloader"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248052    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-metric"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248061    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="prom-label-proxy"
Jul 16 22:20:12.248059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248067    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248073    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="alertmanager"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248079    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy-web"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248085    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy-thanos"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248117    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="prometheus"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248125    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="97aad57e-2953-43bb-a28a-046ab02341c7" containerName="console"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248131    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="config-reloader"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248137    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="config-reloader"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248142    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cdbc64d-7d7b-4d97-8fc7-667f031a5663" containerName="kube-rbac-proxy"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248146    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="kube-rbac-proxy"
Jul 16 22:20:12.248827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.248151    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" containerName="thanos-sidecar"
Jul 16 22:20:12.253504 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.253486    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.256301 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.256284    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 22:20:12.256548 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.256528    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 22:20:12.256764 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.256749    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 22:20:12.256896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.256753    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 22:20:12.256896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.256824    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 22:20:12.257035 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257019    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 22:20:12.257035 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257029    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 22:20:12.257191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257083    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 22:20:12.257191 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257154    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cov6iqg6h8526\""
Jul 16 22:20:12.257422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257349    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 22:20:12.257422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257369    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 22:20:12.257422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257396    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-7vr69\""
Jul 16 22:20:12.257422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257401    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 22:20:12.257422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257409    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 22:20:12.257720 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.257400    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 22:20:12.260146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.259991    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 22:20:12.263435 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.263419    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 22:20:12.267410 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.267389    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:12.412774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412693    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412731    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412774 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412797    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412874    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412898    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k9gf\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-kube-api-access-2k9gf\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412927    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412950    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.412974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.412984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413124    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413150    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413167    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413185    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413221    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413243    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413282    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.413333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.413299    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514618    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514674    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k9gf\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-kube-api-access-2k9gf\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514709    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514768    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.514829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514815    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514840    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514868    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514905    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514934    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514961    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.514994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515061    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515085    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515150    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515182    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515447    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515538    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515531    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.515999 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.515977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.517269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.516917    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.517269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.517227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.517269 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.517242    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.518289 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.518265    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.518734 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.518711    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.519197 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.519145    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.519716 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.519653    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f7603c3-9316-4bfa-bb67-1312bade264b-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.519716 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.519659    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.519878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.519834    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.520008 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.519958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.520080 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.520011    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.520223 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.520201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.520223 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.520217    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f7603c3-9316-4bfa-bb67-1312bade264b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.520447 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.520428    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.521060 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.521037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f7603c3-9316-4bfa-bb67-1312bade264b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.522121 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.522077    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k9gf\" (UniqueName: \"kubernetes.io/projected/4f7603c3-9316-4bfa-bb67-1312bade264b-kube-api-access-2k9gf\") pod \"prometheus-k8s-0\" (UID: \"4f7603c3-9316-4bfa-bb67-1312bade264b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.563025 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.562993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:12.706176 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.706140    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 22:20:12.714668 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:20:12.714645    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7603c3_9316_4bfa_bb67_1312bade264b.slice/crio-8953c7cf2724f83fbea595d8d226b85a8c9f8b293ac285c795cb55f3960283e8 WatchSource:0}: Error finding container 8953c7cf2724f83fbea595d8d226b85a8c9f8b293ac285c795cb55f3960283e8: Status 404 returned error can't find the container with id 8953c7cf2724f83fbea595d8d226b85a8c9f8b293ac285c795cb55f3960283e8
Jul 16 22:20:12.828132 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:12.828084    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d" path="/var/lib/kubelet/pods/d2a21dd3-bc68-4ea4-bca7-2cf182bdd19d/volumes"
Jul 16 22:20:13.194078 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:13.194047    2556 generic.go:356] "Generic (PLEG): container finished" podID="4f7603c3-9316-4bfa-bb67-1312bade264b" containerID="0b80e2ff6b4784f578ee4d689a23e864ecc36b4e7c95aa6699215a9becdfda77" exitCode=0
Jul 16 22:20:13.194515 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:13.194129    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerDied","Data":"0b80e2ff6b4784f578ee4d689a23e864ecc36b4e7c95aa6699215a9becdfda77"}
Jul 16 22:20:13.194515 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:13.194212    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"8953c7cf2724f83fbea595d8d226b85a8c9f8b293ac285c795cb55f3960283e8"}
Jul 16 22:20:14.135959 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.135925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:14.192955 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.192896    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:14.202533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202505    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"3e5e97714bc07365fe2ede989aa856a58bcf0d61556e861eea60e83d079bd5b6"}
Jul 16 22:20:14.202888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202564    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"190194389a41b1e517934bb748d4ddaffcadcc4ccc24a1d35322abb12dc416c2"}
Jul 16 22:20:14.202888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202595    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"99405a44d6fdc30808cd02997fea2d608f124e87fbf3857f4a47cefc74f49162"}
Jul 16 22:20:14.202888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202620    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"eea3be9301cfd8f20dba93f1ab6463bac4ffd15893c66f1492a245588889598e"}
Jul 16 22:20:14.202888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"70178463c84eb6bac2cd7361576982c24154cb996951566336e5baaaa626175b"}
Jul 16 22:20:14.202888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.202674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f7603c3-9316-4bfa-bb67-1312bade264b","Type":"ContainerStarted","Data":"3415e46fc00e827ca15d1be661722068d21b9a02256f95341a64e4e7472ab6ac"}
Jul 16 22:20:14.230489 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:14.230425    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.211687583 podStartE2EDuration="2.230402326s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.714743ms" isStatefulPod=false podCreationTimestamp="2026-07-16 22:20:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:20:14.228663704 +0000 UTC m=+383.923245591" watchObservedRunningTime="2026-07-16 22:20:14.230402326 +0000 UTC m=+383.924984197"
Jul 16 22:20:15.194069 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:15.194035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:15.258839 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:15.258807    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:15.324721 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:15.324691    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:15.926919 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:15.926881    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:17.563533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:17.563498    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:20:25.957298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:25.957259    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:20:26.094949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:26.094919    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:26.191969 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:26.191930    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:26.249011 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:26.248909    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:30.519647 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:30.519607    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:30.620884 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:30.620850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:50.978556 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:50.978471    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-864d9cd78-dv447" podUID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" containerName="console" containerID="cri-o://e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae" gracePeriod=15
Jul 16 22:20:51.249625 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.249598    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:20:51.342138 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342110    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342138 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342139    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342173    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342244    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342281    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6f7z\" (UniqueName: \"kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342319    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342569 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342521    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config\") pod \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\" (UID: \"aa4df4ee-d612-4008-9ad3-90d20046b4d3\") "
Jul 16 22:20:51.342716 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342689    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config" (OuterVolumeSpecName: "console-config") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:51.342716 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342699    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342708    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342728    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342802    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-trusted-ca-bundle\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342817    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342826    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.342838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.342836    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa4df4ee-d612-4008-9ad3-90d20046b4d3-oauth-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.344612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.344589    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:51.344740 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.344626    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:20:51.344791 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.344765    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z" (OuterVolumeSpecName: "kube-api-access-s6f7z") pod "aa4df4ee-d612-4008-9ad3-90d20046b4d3" (UID: "aa4df4ee-d612-4008-9ad3-90d20046b4d3"). InnerVolumeSpecName "kube-api-access-s6f7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:20:51.346447 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.346423    2556 generic.go:356] "Generic (PLEG): container finished" podID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" containerID="e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae" exitCode=0
Jul 16 22:20:51.346566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.346463    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d9cd78-dv447" event={"ID":"aa4df4ee-d612-4008-9ad3-90d20046b4d3","Type":"ContainerDied","Data":"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"}
Jul 16 22:20:51.346566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.346476    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864d9cd78-dv447"
Jul 16 22:20:51.346566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.346497    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864d9cd78-dv447" event={"ID":"aa4df4ee-d612-4008-9ad3-90d20046b4d3","Type":"ContainerDied","Data":"5c4b2598b95d8e6abb041fc8bdce4c8d0bebc172e20f048258a7af0c2d99eaa8"}
Jul 16 22:20:51.346566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.346516    2556 scope.go:117] "RemoveContainer" containerID="e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"
Jul 16 22:20:51.359565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.359548    2556 scope.go:117] "RemoveContainer" containerID="e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"
Jul 16 22:20:51.359873 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:20:51.359846    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae\": container with ID starting with e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae not found: ID does not exist" containerID="e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"
Jul 16 22:20:51.359971 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.359884    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae"} err="failed to get container status \"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae\": rpc error: code = NotFound desc = could not find container \"e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae\": container with ID starting with e7d149b49672dfa8815ad1fa82b9c5e82b9beae2536326cb9c81ba25ea6ccaae not found: ID does not exist"
Jul 16 22:20:51.369148 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.369125    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:20:51.372688 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.372669    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-864d9cd78-dv447"]
Jul 16 22:20:51.444270 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.444235    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6f7z\" (UniqueName: \"kubernetes.io/projected/aa4df4ee-d612-4008-9ad3-90d20046b4d3-kube-api-access-s6f7z\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.444270 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.444262    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-oauth-config\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:51.444270 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:51.444276    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa4df4ee-d612-4008-9ad3-90d20046b4d3-console-serving-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:20:52.828130 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:52.828072    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" path="/var/lib/kubelet/pods/aa4df4ee-d612-4008-9ad3-90d20046b4d3/volumes"
Jul 16 22:20:55.553397 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:55.553365    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:20:55.633046 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:20:55.633017    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:07.796020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:07.795986    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:12.564220 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:12.564135    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:21:12.579439 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:12.579414    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:21:13.445198 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:13.445173    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 22:21:15.627258 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:15.627220    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:15.709240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:15.709207    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:17.311157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:17.311130    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:18.371620 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:18.371565    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:20.903875 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:20.903845    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:21.323937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:21.323903    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:22.577025 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:22.576994    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:22.686938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:22.686906    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:25.113706 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:25.113666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:25.512720 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:25.512685    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:27.746552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:27.746523    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:27.803949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:27.803917    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:32.327795 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:32.327766    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:46.038616 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:46.038580    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:46.135052 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:46.135021    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:49.203846 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:49.203812    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:49.263210 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:49.263174    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:51.753981 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:51.753950    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:51.851592 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:51.851558    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:53.759989 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:53.759958    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:53.860426 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:53.860393    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:21:53.960891 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:21:53.960864    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:04.117563 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.117526    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:04.194635 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.194601    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"]
Jul 16 22:22:04.194896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.194885    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" containerName="console"
Jul 16 22:22:04.194938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.194897    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" containerName="console"
Jul 16 22:22:04.194972 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.194947    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4df4ee-d612-4008-9ad3-90d20046b4d3" containerName="console"
Jul 16 22:22:04.199058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.199039    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.201437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.201411    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jqctg\""
Jul 16 22:22:04.201566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.201455    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 22:22:04.201566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.201481    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 22:22:04.201566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.201420    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 22:22:04.201566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.201521    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 22:22:04.202472 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.202459    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 22:22:04.208991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.208967    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"]
Jul 16 22:22:04.247198 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.247164    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:04.301283 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.301251    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7ck8\" (UniqueName: \"kubernetes.io/projected/99f3c696-2099-4aeb-9665-f9800efe68e5-kube-api-access-f7ck8\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.301456 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.301301    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-metrics-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.301456 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.301400    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/99f3c696-2099-4aeb-9665-f9800efe68e5-manager-config\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.301575 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.301536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.402298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.402201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.402298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.402264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7ck8\" (UniqueName: \"kubernetes.io/projected/99f3c696-2099-4aeb-9665-f9800efe68e5-kube-api-access-f7ck8\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.402298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.402293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-metrics-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.402575 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.402326    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/99f3c696-2099-4aeb-9665-f9800efe68e5-manager-config\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.403158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.403132    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/99f3c696-2099-4aeb-9665-f9800efe68e5-manager-config\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.405380 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.405354    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.405586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.405566    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/99f3c696-2099-4aeb-9665-f9800efe68e5-metrics-cert\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.415868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.415837    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7ck8\" (UniqueName: \"kubernetes.io/projected/99f3c696-2099-4aeb-9665-f9800efe68e5-kube-api-access-f7ck8\") pod \"lws-controller-manager-f787d6dbd-n4lnb\" (UID: \"99f3c696-2099-4aeb-9665-f9800efe68e5\") " pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.509015 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.508972    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:04.647252 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:04.647222    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"]
Jul 16 22:22:04.650373 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:22:04.650350    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f3c696_2099_4aeb_9665_f9800efe68e5.slice/crio-92c813e64c04cec31e2903183b2b3693b50daf0fa349012539c2bde00ab09d79 WatchSource:0}: Error finding container 92c813e64c04cec31e2903183b2b3693b50daf0fa349012539c2bde00ab09d79: Status 404 returned error can't find the container with id 92c813e64c04cec31e2903183b2b3693b50daf0fa349012539c2bde00ab09d79
Jul 16 22:22:05.580146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:05.580082    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:05.623285 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:05.623254    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb" event={"ID":"99f3c696-2099-4aeb-9665-f9800efe68e5","Type":"ContainerStarted","Data":"92c813e64c04cec31e2903183b2b3693b50daf0fa349012539c2bde00ab09d79"}
Jul 16 22:22:05.683221 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:05.683191    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.270617 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.270586    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.394865 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.394799    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.503450 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.503420    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.549546 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.549468    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"]
Jul 16 22:22:06.554330 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.554304    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.556455 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.556429    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 22:22:06.556579 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.556473    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4psbw\""
Jul 16 22:22:06.556695 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.556678    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 22:22:06.556764 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.556749    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 22:22:06.557308 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.557290    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 22:22:06.580509 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.580480    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"]
Jul 16 22:22:06.620150 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.620112    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-manifests\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.620332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.620181    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-webhook-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.620332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.620212    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.620332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.620250    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-tmp\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.620332 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.620293    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4jvj\" (UniqueName: \"kubernetes.io/projected/c2766481-5285-4194-81e8-87a5aad77882-kube-api-access-b4jvj\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.679032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.678999    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.721364 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721328    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-manifests\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.721549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721380    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-webhook-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.721549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721402    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.721549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721429    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-tmp\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.721549 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721445    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4jvj\" (UniqueName: \"kubernetes.io/projected/c2766481-5285-4194-81e8-87a5aad77882-kube-api-access-b4jvj\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.721811 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.721790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-manifests\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.723769 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.723737    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2766481-5285-4194-81e8-87a5aad77882-tmp\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.723906 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.723865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.724054 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.724036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2766481-5285-4194-81e8-87a5aad77882-webhook-cert\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.729298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.729276    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4jvj\" (UniqueName: \"kubernetes.io/projected/c2766481-5285-4194-81e8-87a5aad77882-kube-api-access-b4jvj\") pod \"opendatahub-operator-controller-manager-7846b6849-dpdwb\" (UID: \"c2766481-5285-4194-81e8-87a5aad77882\") " pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.779680 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.779648    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.865914 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.865834    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:06.896199 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.896155    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:06.986822 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:06.986780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:07.005254 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:07.005194    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"]
Jul 16 22:22:07.007986 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:22:07.007952    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2766481_5285_4194_81e8_87a5aad77882.slice/crio-b6b4b31795b6d71fcfccd250e2f633aa8f8097bfbab7d8ae31f5947577a758b1 WatchSource:0}: Error finding container b6b4b31795b6d71fcfccd250e2f633aa8f8097bfbab7d8ae31f5947577a758b1: Status 404 returned error can't find the container with id b6b4b31795b6d71fcfccd250e2f633aa8f8097bfbab7d8ae31f5947577a758b1
Jul 16 22:22:07.089951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:07.089910    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:07.633485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:07.633442    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb" event={"ID":"c2766481-5285-4194-81e8-87a5aad77882","Type":"ContainerStarted","Data":"b6b4b31795b6d71fcfccd250e2f633aa8f8097bfbab7d8ae31f5947577a758b1"}
Jul 16 22:22:10.649928 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:10.649886    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb" event={"ID":"c2766481-5285-4194-81e8-87a5aad77882","Type":"ContainerStarted","Data":"aee4c76c1e793507cc5b89a271a790aa45540fc55fb0b193dea8b732dc0f5bfd"}
Jul 16 22:22:11.655889 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:11.655856    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2766481-5285-4194-81e8-87a5aad77882" containerID="aee4c76c1e793507cc5b89a271a790aa45540fc55fb0b193dea8b732dc0f5bfd" exitCode=0
Jul 16 22:22:11.656215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:11.655942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb" event={"ID":"c2766481-5285-4194-81e8-87a5aad77882","Type":"ContainerDied","Data":"aee4c76c1e793507cc5b89a271a790aa45540fc55fb0b193dea8b732dc0f5bfd"}
Jul 16 22:22:12.662118 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.662066    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb" event={"ID":"c2766481-5285-4194-81e8-87a5aad77882","Type":"ContainerStarted","Data":"8bf19f0cf83ec6ddfe9d251e0a45030aaff8279459c198da3eccaff231380e81"}
Jul 16 22:22:12.662560 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.662209    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:12.663466 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.663442    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb" event={"ID":"99f3c696-2099-4aeb-9665-f9800efe68e5","Type":"ContainerStarted","Data":"c3b3916a1f7c9e32287ed681746d585313242202f6f32878e60a3e0105c51ad2"}
Jul 16 22:22:12.663600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.663565    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:12.683619 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.683545    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb" podStartSLOduration=1.993375323 podStartE2EDuration="6.683513551s" totalImagesPullingTime="3.300201116s" totalInitContainerRuntime="1.389937112s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:12.683512523 +0000 UTC m=+502.378094392" watchObservedRunningTime="2026-07-16 22:22:12.683513551 +0000 UTC m=+502.378095406"
Jul 16 22:22:12.699985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:12.699937    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb" podStartSLOduration=1.708102744 podStartE2EDuration="8.699921579s" totalImagesPullingTime="6.991818835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:12.698113584 +0000 UTC m=+502.392695447" watchObservedRunningTime="2026-07-16 22:22:12.699921579 +0000 UTC m=+502.394503433"
Jul 16 22:22:13.540409 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:13.540368    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:13.601936 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:13.601905    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:23.671965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:23.671936    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f787d6dbd-n4lnb"
Jul 16 22:22:23.672460 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:23.672025    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7846b6849-dpdwb"
Jul 16 22:22:25.376301 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:25.376263    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:25.548683 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:25.548649    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:25.969471 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:25.969439    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:26.146962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.146928    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"]
Jul 16 22:22:26.162667 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.162624    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:26.249854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.249765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"]
Jul 16 22:22:26.250003 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.249881    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.252579 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.252560    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9lnc9\""
Jul 16 22:22:26.252708 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.252599    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 22:22:26.252708 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.252644    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 22:22:26.393464 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.393431    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5scb2\" (UniqueName: \"kubernetes.io/projected/b919f83e-d47b-4852-9af1-13a32ed03a4d-kube-api-access-5scb2\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.393464 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.393468    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b919f83e-d47b-4852-9af1-13a32ed03a4d-tmp\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.393867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.393562    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b919f83e-d47b-4852-9af1-13a32ed03a4d-tls-certs\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.459681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.459646    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:26.494922 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.494892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b919f83e-d47b-4852-9af1-13a32ed03a4d-tls-certs\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.495117 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.494936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5scb2\" (UniqueName: \"kubernetes.io/projected/b919f83e-d47b-4852-9af1-13a32ed03a4d-kube-api-access-5scb2\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.495117 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.494954    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b919f83e-d47b-4852-9af1-13a32ed03a4d-tmp\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.497222 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.497193    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b919f83e-d47b-4852-9af1-13a32ed03a4d-tmp\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.497385 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.497363    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b919f83e-d47b-4852-9af1-13a32ed03a4d-tls-certs\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.503456 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.503397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5scb2\" (UniqueName: \"kubernetes.io/projected/b919f83e-d47b-4852-9af1-13a32ed03a4d-kube-api-access-5scb2\") pod \"kube-auth-proxy-5f4cc9ff56-bhmkb\" (UID: \"b919f83e-d47b-4852-9af1-13a32ed03a4d\") " pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.559320 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.559291    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"
Jul 16 22:22:26.565264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.565236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:26.667789 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.667749    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:26.720552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.716615    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb"]
Jul 16 22:22:26.724509 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:22:26.724485    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb919f83e_d47b_4852_9af1_13a32ed03a4d.slice/crio-09016cb9b0dd930ba9b91c23d0d67117d070e77380c8b3c458fc0bb6a810ca77 WatchSource:0}: Error finding container 09016cb9b0dd930ba9b91c23d0d67117d070e77380c8b3c458fc0bb6a810ca77: Status 404 returned error can't find the container with id 09016cb9b0dd930ba9b91c23d0d67117d070e77380c8b3c458fc0bb6a810ca77
Jul 16 22:22:26.763576 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:26.763493    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:27.487552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:27.487520    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:27.672318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:27.672269    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:27.729940 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:27.729910    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb" event={"ID":"b919f83e-d47b-4852-9af1-13a32ed03a4d","Type":"ContainerStarted","Data":"09016cb9b0dd930ba9b91c23d0d67117d070e77380c8b3c458fc0bb6a810ca77"}
Jul 16 22:22:27.856446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:27.856363    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:28.780225 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:28.780194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:29.741264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:29.741227    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb" event={"ID":"b919f83e-d47b-4852-9af1-13a32ed03a4d","Type":"ContainerStarted","Data":"87477c77344791a76ab4c2d3df9c9f87168c2791e5f71f6c42646653bfb58855"}
Jul 16 22:22:29.758960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:29.758907    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f4cc9ff56-bhmkb" podStartSLOduration=1.416664508 podStartE2EDuration="3.758891215s" totalImagesPullingTime="2.342226707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:22:29.757739006 +0000 UTC m=+519.452320877" watchObservedRunningTime="2026-07-16 22:22:29.758891215 +0000 UTC m=+519.453473068"
Jul 16 22:22:33.347232 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:33.347201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:33.628730 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:33.628650    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:33.739407 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:33.739370    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:33.977648 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:33.977620    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:34.180641 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:34.180613    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:34.388081 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:34.387991    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:37.922071 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:37.922011    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:38.084301 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:38.084271    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:38.167443 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:38.167415    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:38.288960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:38.288929    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:38.442568 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:38.442536    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:38.618668 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:38.618597    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:39.241710 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:39.241676    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:39.453276 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:39.453247    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:22:54.129583 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:22:54.129503    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:26.856994 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:26.856961    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:28.131546 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:28.131513    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:49.823534 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:49.823495    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:53.303822 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:53.303792    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:55.229878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:55.229837    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:56.813282 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:56.813247    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:57.111481 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:57.111404    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:23:57.215827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:23:57.215798    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:00.460051 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:00.460016    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:00.817882 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:00.817806    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:00.905419 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:00.905385    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:02.555970 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.555928    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"]
Jul 16 22:24:02.559031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.559013    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.560875 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.560850    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 16 22:24:02.560991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.560856    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 22:24:02.560991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.560856    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 16 22:24:02.561136 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.561116    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dwmx8\""
Jul 16 22:24:02.561187 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.561175    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 22:24:02.568972 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.568945    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"]
Jul 16 22:24:02.589320 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.589290    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.589320 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.589328    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.589522 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.589353    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdt9c\" (UniqueName: \"kubernetes.io/projected/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-kube-api-access-tdt9c\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.668563 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.668532    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:02.690448 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.690413    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.690448 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.690449    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.690620 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.690467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdt9c\" (UniqueName: \"kubernetes.io/projected/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-kube-api-access-tdt9c\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.690620 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:24:02.690583    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 16 22:24:02.690695 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:24:02.690669    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert podName:1ae77ea2-37a4-43f8-9360-c6f8b747dc29 nodeName:}" failed. No retries permitted until 2026-07-16 22:24:03.190646476 +0000 UTC m=+612.885228308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-d2p6f" (UID: "1ae77ea2-37a4-43f8-9360-c6f8b747dc29") : secret "plugin-serving-cert" not found
Jul 16 22:24:02.691180 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.691162    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.701911 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.701883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdt9c\" (UniqueName: \"kubernetes.io/projected/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-kube-api-access-tdt9c\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:02.821217 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:02.821130    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:03.194406 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.194361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:03.196790 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.196773    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae77ea2-37a4-43f8-9360-c6f8b747dc29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d2p6f\" (UID: \"1ae77ea2-37a4-43f8-9360-c6f8b747dc29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:03.482313 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.482227    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"
Jul 16 22:24:03.635180 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.635151    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f"]
Jul 16 22:24:03.637115 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:24:03.637067    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ae77ea2_37a4_43f8_9360_c6f8b747dc29.slice/crio-36fccafda3d0d0e122c0fff429fcc1689cbc26de918dacbf6d48b2cfc41e0d7d WatchSource:0}: Error finding container 36fccafda3d0d0e122c0fff429fcc1689cbc26de918dacbf6d48b2cfc41e0d7d: Status 404 returned error can't find the container with id 36fccafda3d0d0e122c0fff429fcc1689cbc26de918dacbf6d48b2cfc41e0d7d
Jul 16 22:24:03.638334 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.638313    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:24:03.676723 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:03.676684    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:04.125325 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:04.125291    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f" event={"ID":"1ae77ea2-37a4-43f8-9360-c6f8b747dc29","Type":"ContainerStarted","Data":"36fccafda3d0d0e122c0fff429fcc1689cbc26de918dacbf6d48b2cfc41e0d7d"}
Jul 16 22:24:05.103531 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:05.103496    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:05.179168 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:05.179124    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:12.782686 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:12.782352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:12.979038 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:12.979007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:16.965216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:16.965186    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bd646fcdd-wbgxb"]
Jul 16 22:24:17.000437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.000405    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bd646fcdd-wbgxb"]
Jul 16 22:24:17.000573 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.000524    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.002804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.002783    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 22:24:17.002937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.002811    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 22:24:17.003334 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.003314    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ct4lz\""
Jul 16 22:24:17.003724 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.003703    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 22:24:17.004031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.004006    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 22:24:17.004138 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.004049    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 22:24:17.004789 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.004389    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 22:24:17.004789 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.004404    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 22:24:17.010524 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.010506    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 22:24:17.034007 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.033982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:17.118358 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.118330    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:17.125209 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125184    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-service-ca\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125240    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-oauth-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125292    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-oauth-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125317    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125342    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-trusted-ca-bundle\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125503 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125357    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq4bj\" (UniqueName: \"kubernetes.io/projected/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-kube-api-access-sq4bj\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.125503 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.125387    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.226925 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.226846    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.226925 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.226881    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-trusted-ca-bundle\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.226925 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.226924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq4bj\" (UniqueName: \"kubernetes.io/projected/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-kube-api-access-sq4bj\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.226971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.227032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-service-ca\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.227260    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-oauth-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.227305    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-oauth-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227870 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.227845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.227870 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.227867    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-trusted-ca-bundle\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.228131 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.228114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-service-ca\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.228309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.228285    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-oauth-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.229968 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.229943    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-serving-cert\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.230120 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.230079    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-console-oauth-config\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.234795 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.234776    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq4bj\" (UniqueName: \"kubernetes.io/projected/0ae657ca-e87f-4ce0-abee-7fe6e1440ce6-kube-api-access-sq4bj\") pod \"console-bd646fcdd-wbgxb\" (UID: \"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6\") " pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:17.313518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:17.313487    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:18.446109 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:18.446058    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:18.551582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:18.551553    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:19.092027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:19.091988    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:21.452400 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:21.452363    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:21.553892 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:21.553849    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:25.483765 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:25.483746    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bd646fcdd-wbgxb"]
Jul 16 22:24:25.488330 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:24:25.488307    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae657ca_e87f_4ce0_abee_7fe6e1440ce6.slice/crio-e217a17ff1f446d908522bb75b95b5925789a59faf19130e1147b92e90d42c35 WatchSource:0}: Error finding container e217a17ff1f446d908522bb75b95b5925789a59faf19130e1147b92e90d42c35: Status 404 returned error can't find the container with id e217a17ff1f446d908522bb75b95b5925789a59faf19130e1147b92e90d42c35
Jul 16 22:24:26.232257 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.232218    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f" event={"ID":"1ae77ea2-37a4-43f8-9360-c6f8b747dc29","Type":"ContainerStarted","Data":"0825aa082d9433b21f5046d4bec8d88fd3a9a98f270b8e270fc308a4f00400a9"}
Jul 16 22:24:26.233966 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.233932    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd646fcdd-wbgxb" event={"ID":"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6","Type":"ContainerStarted","Data":"21b97ea8d09c11f62c5eb091ceff512684922b518fb9ec62948c55c3a04f661f"}
Jul 16 22:24:26.234076 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.234001    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd646fcdd-wbgxb" event={"ID":"0ae657ca-e87f-4ce0-abee-7fe6e1440ce6","Type":"ContainerStarted","Data":"e217a17ff1f446d908522bb75b95b5925789a59faf19130e1147b92e90d42c35"}
Jul 16 22:24:26.261992 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.261942    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d2p6f" podStartSLOduration=2.468586621 podStartE2EDuration="24.261929055s" totalImagesPullingTime="21.793342434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:26.260631406 +0000 UTC m=+635.955213247" watchObservedRunningTime="2026-07-16 22:24:26.261929055 +0000 UTC m=+635.956510908"
Jul 16 22:24:26.269846 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.269824    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:26.294317 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.294269    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bd646fcdd-wbgxb" podStartSLOduration=10.294252212 podStartE2EDuration="10.294252212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:26.293249585 +0000 UTC m=+635.987831439" watchObservedRunningTime="2026-07-16 22:24:26.294252212 +0000 UTC m=+635.988834066"
Jul 16 22:24:26.374285 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:26.374258    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:27.314208 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:27.314167    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:27.314641 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:27.314236    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:27.318973 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:27.318950    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:28.245591 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:28.245561    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bd646fcdd-wbgxb"
Jul 16 22:24:28.416336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:28.416309    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:28.515945 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:28.515867    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:28.616438 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:28.616407    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:34.078052 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:34.078004    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:34.141287 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:34.141253    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:39.244389 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:39.244352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:39.346157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:39.346125    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:42.244945 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:42.244906    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:42.391840 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:42.391806    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:44.307585 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:44.307546    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:44.641247 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:44.641162    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:46.276826 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.276795    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:46.398682 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.398648    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:46.533005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.532924    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:46.633577 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.633546    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:24:46.635800 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.635784    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.637844 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.637822    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 22:24:46.645421 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.645396    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:24:46.661479 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.661446    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:24:46.681170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.681139    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b45869b-1334-4e93-b0be-65955acd6202-config-file\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.681318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.681196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qktwv\" (UniqueName: \"kubernetes.io/projected/3b45869b-1334-4e93-b0be-65955acd6202-kube-api-access-qktwv\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.781983 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.781929    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qktwv\" (UniqueName: \"kubernetes.io/projected/3b45869b-1334-4e93-b0be-65955acd6202-kube-api-access-qktwv\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.782221 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.782042    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b45869b-1334-4e93-b0be-65955acd6202-config-file\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.782669 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.782646    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3b45869b-1334-4e93-b0be-65955acd6202-config-file\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.790047 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.789983    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qktwv\" (UniqueName: \"kubernetes.io/projected/3b45869b-1334-4e93-b0be-65955acd6202-kube-api-access-qktwv\") pod \"limitador-limitador-54d6d988d5-ggpdz\" (UID: \"3b45869b-1334-4e93-b0be-65955acd6202\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:46.946987 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:46.946953    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:47.079570 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:47.079495    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:24:47.082792 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:24:47.082763    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b45869b_1334_4e93_b0be_65955acd6202.slice/crio-60a8811f52604fa913ff20b5109810fee9bdef379c73ebac0153836c987ee768 WatchSource:0}: Error finding container 60a8811f52604fa913ff20b5109810fee9bdef379c73ebac0153836c987ee768: Status 404 returned error can't find the container with id 60a8811f52604fa913ff20b5109810fee9bdef379c73ebac0153836c987ee768
Jul 16 22:24:47.177264 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:47.177233    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:47.259729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:47.259699    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:47.321350 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:47.321315    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz" event={"ID":"3b45869b-1334-4e93-b0be-65955acd6202","Type":"ContainerStarted","Data":"60a8811f52604fa913ff20b5109810fee9bdef379c73ebac0153836c987ee768"}
Jul 16 22:24:50.339598 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:50.339500    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz" event={"ID":"3b45869b-1334-4e93-b0be-65955acd6202","Type":"ContainerStarted","Data":"0e6a62a2d89003f3da71dc7b4bce99e477d2f7cc5f6c0b7218d0cec6151dd825"}
Jul 16 22:24:50.339990 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:50.339616    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:24:50.357756 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:50.357697    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz" podStartSLOduration=1.4412717769999999 podStartE2EDuration="4.35767908s" totalImagesPullingTime="2.916407303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:24:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:24:50.356360911 +0000 UTC m=+660.050942785" watchObservedRunningTime="2026-07-16 22:24:50.35767908 +0000 UTC m=+660.052260933"
Jul 16 22:24:54.299954 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:54.299922    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:24:54.421912 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:24:54.421869    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:00.648275 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:00.648240    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:00.749279 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:00.749245    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:01.345263 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:01.345235    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"
Jul 16 22:25:01.367948 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:01.367914    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:01.438370 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:01.438325    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:04.447391 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:04.447361    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:04.548669 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:04.548614    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:07.512889 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:07.512856    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:07.709852 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:07.709824    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:07.981941 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:07.981911    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:20.403562 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.403528    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:20.729523 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.729481    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:20.859765 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.859733    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-5z5pt"]
Jul 16 22:25:20.862905 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.862887    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:20.864983 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.864960    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7xb4r\""
Jul 16 22:25:20.864983 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.864974    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 16 22:25:20.872465 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.872440    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-5z5pt"]
Jul 16 22:25:20.907389 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.907357    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-5z5pt"]
Jul 16 22:25:20.907628 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:25:20.907609    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fnxl8 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-5z5pt" podUID="7147b2b9-2621-4f28-b803-3e254b33f697"
Jul 16 22:25:20.987742 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.987641    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:20.987742 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:20.987712    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnxl8\" (UniqueName: \"kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.088439 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.088406    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnxl8\" (UniqueName: \"kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.088614 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.088512    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.091081 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.091059    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.096056 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.096028    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnxl8\" (UniqueName: \"kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8\") pod \"authorino-76877994-5z5pt\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") " pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.488218 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.488188    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.495236 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.495214    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:21.592299 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.592263    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert\") pod \"7147b2b9-2621-4f28-b803-3e254b33f697\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") "
Jul 16 22:25:21.592493 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.592387    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnxl8\" (UniqueName: \"kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8\") pod \"7147b2b9-2621-4f28-b803-3e254b33f697\" (UID: \"7147b2b9-2621-4f28-b803-3e254b33f697\") "
Jul 16 22:25:21.594400 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.594374    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7147b2b9-2621-4f28-b803-3e254b33f697" (UID: "7147b2b9-2621-4f28-b803-3e254b33f697"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:25:21.594515 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.594435    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8" (OuterVolumeSpecName: "kube-api-access-fnxl8") pod "7147b2b9-2621-4f28-b803-3e254b33f697" (UID: "7147b2b9-2621-4f28-b803-3e254b33f697"). InnerVolumeSpecName "kube-api-access-fnxl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:25:21.693325 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.693291    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7147b2b9-2621-4f28-b803-3e254b33f697-tls-cert\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:25:21.693325 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:21.693319    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnxl8\" (UniqueName: \"kubernetes.io/projected/7147b2b9-2621-4f28-b803-3e254b33f697-kube-api-access-fnxl8\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:25:22.492670 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:22.492641    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-5z5pt"
Jul 16 22:25:22.518878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:22.518849    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-5z5pt"]
Jul 16 22:25:22.522353 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:22.522331    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-5z5pt"]
Jul 16 22:25:22.828665 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:22.828582    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7147b2b9-2621-4f28-b803-3e254b33f697" path="/var/lib/kubelet/pods/7147b2b9-2621-4f28-b803-3e254b33f697/volumes"
Jul 16 22:25:26.932447 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:26.932413    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:27.034759 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:27.034728    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:27.136673 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:27.136643    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:27.208495 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:27.208463    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:30.296622 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:30.296582    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:25:30.394185 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:25:30.394154    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:03.842510 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:03.842427    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:03.940533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:03.940501    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:04.443213 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:04.443185    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:04.906537 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:04.906452    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:05.000103 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:05.000059    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:05.080364 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:05.080337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:05.158473 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:05.158381    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:26:15.356968 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:15.356933    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:15.448080 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:15.448046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:24.981140 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.981110    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"]
Jul 16 22:26:24.984562 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.984546    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:24.986742 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.986717    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 22:26:24.986863 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.986813    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 22:26:24.987134 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.987115    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2bhmx\""
Jul 16 22:26:24.987231 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.987151    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 22:26:24.999817 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:24.999796    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"]
Jul 16 22:26:25.006599 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.006563    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:25.031053 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.031241 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031058    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.031241 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031082    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.031315 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031249    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c793bc1a-af86-42e8-b0b9-49c1d940d443-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.031315 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031286    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqttk\" (UniqueName: \"kubernetes.io/projected/c793bc1a-af86-42e8-b0b9-49c1d940d443-kube-api-access-mqttk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.031396 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.031376    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132507 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132570    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c793bc1a-af86-42e8-b0b9-49c1d940d443-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132707    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqttk\" (UniqueName: \"kubernetes.io/projected/c793bc1a-af86-42e8-b0b9-49c1d940d443-kube-api-access-mqttk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.132993 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.132937    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.133137 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.133082    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.133190 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.133151    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.134829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.134804    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c793bc1a-af86-42e8-b0b9-49c1d940d443-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.135131 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.135114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c793bc1a-af86-42e8-b0b9-49c1d940d443-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.141236 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.141215    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqttk\" (UniqueName: \"kubernetes.io/projected/c793bc1a-af86-42e8-b0b9-49c1d940d443-kube-api-access-mqttk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8\" (UID: \"c793bc1a-af86-42e8-b0b9-49c1d940d443\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.295006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.294907    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:25.432020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.431989    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"]
Jul 16 22:26:25.435308 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:26:25.435281    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc793bc1a_af86_42e8_b0b9_49c1d940d443.slice/crio-30c8838f53bacd081907874d662c8b59f1b3fbd629a8fce988a98aff246ff9e9 WatchSource:0}: Error finding container 30c8838f53bacd081907874d662c8b59f1b3fbd629a8fce988a98aff246ff9e9: Status 404 returned error can't find the container with id 30c8838f53bacd081907874d662c8b59f1b3fbd629a8fce988a98aff246ff9e9
Jul 16 22:26:25.449121 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.449073    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:26:25.768650 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:25.768602    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8" event={"ID":"c793bc1a-af86-42e8-b0b9-49c1d940d443","Type":"ContainerStarted","Data":"30c8838f53bacd081907874d662c8b59f1b3fbd629a8fce988a98aff246ff9e9"}
Jul 16 22:26:26.776230 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:26.776132    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8" event={"ID":"c793bc1a-af86-42e8-b0b9-49c1d940d443","Type":"ContainerStarted","Data":"c11584cf0a7e2644ee0b2025b8827b7fb61d0059ea0ae60df34baed2f4c428b7"}
Jul 16 22:26:26.794391 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:26.794325    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8" podStartSLOduration=1.7871941470000001 podStartE2EDuration="2.794307391s" totalImagesPullingTime="1.007113244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:26:26.792328602 +0000 UTC m=+756.486910478" watchObservedRunningTime="2026-07-16 22:26:26.794307391 +0000 UTC m=+756.488889244"
Jul 16 22:26:35.295831 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:35.295787    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:35.296437 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:35.295866    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:35.309067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:35.309040    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:35.828391 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:35.828359    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8"
Jul 16 22:26:42.266032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:42.265999    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:26:43.044417 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:26:43.044383    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:27:01.267236 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:01.267204    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:02.445321 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:02.445286    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:27:15.805702 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:15.805617    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:18.579976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:18.579938    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:20.235854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:20.235818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:27:26.760488 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:26.760446    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:29.040160 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:29.040130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:27:37.121465 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.121430    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"]
Jul 16 22:27:37.124897 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.124879    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.126955 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.126938    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 16 22:27:37.135646 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.135629    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"]
Jul 16 22:27:37.243212 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.243184    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:37.263917 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.263885    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.264059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.263927    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhmj6\" (UniqueName: \"kubernetes.io/projected/89e021f5-461e-48f8-9a8e-244e0251504d-kube-api-access-hhmj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.264059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.264008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e021f5-461e-48f8-9a8e-244e0251504d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.264059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.264047    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.264194 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.264065    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.264194 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.264155    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.344389 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.344346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:37.365590 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e021f5-461e-48f8-9a8e-244e0251504d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.365685 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.365685 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365623    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.365685 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365655    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.365821 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.365821 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.365721    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhmj6\" (UniqueName: \"kubernetes.io/projected/89e021f5-461e-48f8-9a8e-244e0251504d-kube-api-access-hhmj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.366116 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.366072    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.366227 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.366205    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.366290 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.366268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.368003 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.367977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e021f5-461e-48f8-9a8e-244e0251504d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.368259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.368243    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e021f5-461e-48f8-9a8e-244e0251504d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.378775 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.378377    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhmj6\" (UniqueName: \"kubernetes.io/projected/89e021f5-461e-48f8-9a8e-244e0251504d-kube-api-access-hhmj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2\" (UID: \"89e021f5-461e-48f8-9a8e-244e0251504d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.433294 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.433260    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:37.574209 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.574179    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"]
Jul 16 22:27:37.577180 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:27:37.577151    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89e021f5_461e_48f8_9a8e_244e0251504d.slice/crio-0c49efa91696ecb684447bc6586829a24483458767ffa544dd663d3ace4a63cc WatchSource:0}: Error finding container 0c49efa91696ecb684447bc6586829a24483458767ffa544dd663d3ace4a63cc: Status 404 returned error can't find the container with id 0c49efa91696ecb684447bc6586829a24483458767ffa544dd663d3ace4a63cc
Jul 16 22:27:37.867420 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:37.867369    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:38.081219 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:38.081135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2" event={"ID":"89e021f5-461e-48f8-9a8e-244e0251504d","Type":"ContainerStarted","Data":"8657fa8b9b964cc7a7fa240d48e7c72ded4c9732465b2f7f58fd2ef30f189e84"}
Jul 16 22:27:38.081219 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:38.081182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2" event={"ID":"89e021f5-461e-48f8-9a8e-244e0251504d","Type":"ContainerStarted","Data":"0c49efa91696ecb684447bc6586829a24483458767ffa544dd663d3ace4a63cc"}
Jul 16 22:27:38.098912 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:38.098864    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2" podStartSLOduration=0.89370934 podStartE2EDuration="1.098849193s" totalImagesPullingTime="205.139853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:27:38.097715779 +0000 UTC m=+827.792297654" watchObservedRunningTime="2026-07-16 22:27:38.098849193 +0000 UTC m=+827.793431046"
Jul 16 22:27:39.550323 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:39.550277    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:27:47.434275 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:47.434234    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:47.434753 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:47.434294    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:47.446655 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:47.446632    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:48.137361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:48.137327    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2"
Jul 16 22:27:58.786495 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:58.786463    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:27:58.867435 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:27:58.867404    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:30.731402 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:28:30.731369    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:32.402636 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:28:32.402604    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:28:58.656686 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:28:58.656643    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:29:09.235403 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:29:09.235369    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:29:19.649440 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:29:19.649401    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:29:45.553117 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:29:45.553075    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:00.151286 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.151249    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"]
Jul 16 22:30:00.154524 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.154508    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"
Jul 16 22:30:00.156640 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.156621    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:30:00.163855 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.163839    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"
Jul 16 22:30:00.165740 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.165716    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"]
Jul 16 22:30:00.316181 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.316151    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"]
Jul 16 22:30:00.319142 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:30:00.319075    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eba2648_7ccf_424c_b8cb_ba41d6538415.slice/crio-7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6 WatchSource:0}: Error finding container 7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6: Status 404 returned error can't find the container with id 7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6
Jul 16 22:30:00.321063 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.321046    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 22:30:00.709233 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:00.709199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj" event={"ID":"3eba2648-7ccf-424c-b8cb-ba41d6538415","Type":"ContainerStarted","Data":"7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6"}
Jul 16 22:30:03.725108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:03.725050    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj" event={"ID":"3eba2648-7ccf-424c-b8cb-ba41d6538415","Type":"ContainerStarted","Data":"d5eda2e36be9a665ec7b43a1ebc4369a98f702b671f9a91c6be2f3dfdee9e947"}
Jul 16 22:30:03.745123 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:03.744825    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj" podStartSLOduration=1.021736142 podStartE2EDuration="3.744806088s" totalImagesPullingTime="2.723069946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:30:03.744362204 +0000 UTC m=+973.438944059" watchObservedRunningTime="2026-07-16 22:30:03.744806088 +0000 UTC m=+973.439387943"
Jul 16 22:30:04.730917 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:04.730879    2556 generic.go:356] "Generic (PLEG): container finished" podID="3eba2648-7ccf-424c-b8cb-ba41d6538415" containerID="d5eda2e36be9a665ec7b43a1ebc4369a98f702b671f9a91c6be2f3dfdee9e947" exitCode=0
Jul 16 22:30:04.731351 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:04.730965    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj" event={"ID":"3eba2648-7ccf-424c-b8cb-ba41d6538415","Type":"ContainerDied","Data":"d5eda2e36be9a665ec7b43a1ebc4369a98f702b671f9a91c6be2f3dfdee9e947"}
Jul 16 22:30:05.874255 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:05.874231    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"
Jul 16 22:30:06.740074 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:06.740044    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj"
Jul 16 22:30:06.740277 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:06.740044    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737350-gqdzj" event={"ID":"3eba2648-7ccf-424c-b8cb-ba41d6538415","Type":"ContainerDied","Data":"7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6"}
Jul 16 22:30:06.740277 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:06.740170    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bf17ed599496e04ac6abb44cfbf169655397d81337448020d9d1c75134aded6"
Jul 16 22:30:36.209578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:36.209548    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:37.461731 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:37.461694    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:30:59.845781 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:30:59.845744    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:31:12.338119 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:12.338071    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:31:18.177767 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:18.177697    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:18.302655 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:18.302629    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:18.420937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:18.420908    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:24.543017 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:24.542980    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:24.673848 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:24.673812    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:24.771020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:24.770988    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:24.927810 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:24.927731    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:30.769451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:30.769418    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:30.844606 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:30.844571    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:31:47.390254 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:47.390177    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:31:57.240465 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:31:57.240436    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:05.854200 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:05.850167    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:17.243944 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:17.243912    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:27.435432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:27.435401    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:28.277917 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:28.277885    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:28.378048 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:28.378016    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:30.568151 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:30.568123    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:32:42.938504 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:42.938473    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:49.945896 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:49.945861    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:32:57.643012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:32:57.642975    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:33:20.268279 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:33:20.268245    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:33:32.390223 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:33:32.390193    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:33:55.565721 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:33:55.565683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:34:08.150615 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:34:08.150581    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:34:24.653753 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:34:24.653718    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:34:34.730666 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:34:34.730630    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:34:44.349579 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:34:44.349486    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:34:53.648861 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:34:53.648828    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:35:03.643660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:35:03.643624    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:36:53.144677 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:36:53.144646    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:04.340868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:04.340831    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:13.155298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:13.155263    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:26.444787 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:26.444753    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:29.883444 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:29.883412    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:37:35.737991 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:35.737957    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:45.041963 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:45.041883    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:37:55.445977 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:37:55.445941    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:38:05.240429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:38:05.240394    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:38:17.242499 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:38:17.242463    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:38:25.946179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:38:25.946147    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:38:37.036646 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:38:37.036603    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:38:58.338272 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:38:58.338235    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:39:07.649819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:07.649780    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:39:13.964392 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:13.964314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:15.203379 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:15.203338    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:16.945290 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:16.945254    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:39:28.839632 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:28.839599    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:39:42.923739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:42.923709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:39:45.642260 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:45.642227    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:39:56.140878 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:39:56.140835    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:05.542152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:05.542115    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:15.338702 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:15.338668    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:24.655712 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:24.655675    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:34.434574 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:34.434541    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:47.537112 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:47.537009    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:40:56.635528 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:40:56.635493    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:41:05.951355 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:41:05.951319    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:41:17.041325 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:41:17.041288    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:41:28.539552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:41:28.539509    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:41:35.243906 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:41:35.243868    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:41:51.639487 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:41:51.639453    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:02.662268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:02.662235    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:12.466658 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:12.466580    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:20.643703 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:20.643672    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:22.723888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:22.723861    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:30.213602 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:30.213572    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:33.647755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:33.647726    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:38.574043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:38.574015    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:43.656727 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:43.656695    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:46.806006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:46.805978    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:46.940999 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:46.940968    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:47.327485 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:47.327436    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:49.250727 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:49.250695    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:42:54.011707 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:54.011674    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:54.140243 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:54.140218    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:54.366959 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:54.366929    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:54.471016 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:54.470982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:55.476114 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:55.476050    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:55.582625 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:55.582596    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:55.705654 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:55.705629    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:55.916050 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:55.915933    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:56.372298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:56.372265    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:56.494482 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:56.494448    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:58.121903 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:58.121872    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:42:59.548551 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:42:59.548515    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:43:00.542042 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.542006    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:00.542358 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.542346    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eba2648-7ccf-424c-b8cb-ba41d6538415" containerName="cleanup"
Jul 16 22:43:00.542405 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.542360    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eba2648-7ccf-424c-b8cb-ba41d6538415" containerName="cleanup"
Jul 16 22:43:00.542440 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.542430    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eba2648-7ccf-424c-b8cb-ba41d6538415" containerName="cleanup"
Jul 16 22:43:00.545568 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.545552    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:00.547552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.547532    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:43:00.547742 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.547730    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:43:00.553702 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.553680    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:00.561746 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.561719    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz\" (UID: \"d2cf402f-1c48-4792-bec2-e544903d7bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:00.662985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.662953    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz\" (UID: \"d2cf402f-1c48-4792-bec2-e544903d7bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:00.663580 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.663562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz\" (UID: \"d2cf402f-1c48-4792-bec2-e544903d7bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:00.857380 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:00.857309    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:01.010798 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:01.010765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:01.013038 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:43:01.013012    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2cf402f_1c48_4792_bec2_e544903d7bf4.slice/crio-63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d WatchSource:0}: Error finding container 63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d: Status 404 returned error can't find the container with id 63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d
Jul 16 22:43:01.136589 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:01.136505    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerStarted","Data":"63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d"}
Jul 16 22:43:02.142755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:02.142716    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerID="6e63fc3df686b1f622a77c870cf56608c90f843690e1a73ab9cdf48a9b1e880e" exitCode=7
Jul 16 22:43:02.143152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:02.142776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerDied","Data":"6e63fc3df686b1f622a77c870cf56608c90f843690e1a73ab9cdf48a9b1e880e"}
Jul 16 22:43:02.143152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:02.143016    2556 scope.go:117] "RemoveContainer" containerID="6e63fc3df686b1f622a77c870cf56608c90f843690e1a73ab9cdf48a9b1e880e"
Jul 16 22:43:03.149452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:03.149361    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerID="677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b" exitCode=7
Jul 16 22:43:03.149452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:03.149411    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerDied","Data":"677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b"}
Jul 16 22:43:03.149887 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:03.149457    2556 scope.go:117] "RemoveContainer" containerID="6e63fc3df686b1f622a77c870cf56608c90f843690e1a73ab9cdf48a9b1e880e"
Jul 16 22:43:03.149887 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:03.149651    2556 scope.go:117] "RemoveContainer" containerID="677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b"
Jul 16 22:43:03.149887 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:43:03.149861    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz_odh-ai-gateway-infra(d2cf402f-1c48-4792-bec2-e544903d7bf4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4"
Jul 16 22:43:04.156043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:04.156017    2556 scope.go:117] "RemoveContainer" containerID="677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b"
Jul 16 22:43:04.156458 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:43:04.156234    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz_odh-ai-gateway-infra(d2cf402f-1c48-4792-bec2-e544903d7bf4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4"
Jul 16 22:43:15.824203 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:15.824172    2556 scope.go:117] "RemoveContainer" containerID="677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b"
Jul 16 22:43:16.208161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:16.208135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerStarted","Data":"b00d9309384ce5aec126dd77d2b1a0ce9a3301636fcec272d49c32a05e57e470"}
Jul 16 22:43:16.226006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:16.225966    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" podStartSLOduration=16.225952028 podStartE2EDuration="16.225952028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:43:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:43:16.224636966 +0000 UTC m=+1765.919218831" watchObservedRunningTime="2026-07-16 22:43:16.225952028 +0000 UTC m=+1765.920533881"
Jul 16 22:43:17.214496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:17.214463    2556 generic.go:356] "Generic (PLEG): container finished" podID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerID="b00d9309384ce5aec126dd77d2b1a0ce9a3301636fcec272d49c32a05e57e470" exitCode=0
Jul 16 22:43:17.214949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:17.214540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerDied","Data":"b00d9309384ce5aec126dd77d2b1a0ce9a3301636fcec272d49c32a05e57e470"}
Jul 16 22:43:17.214949 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:17.214600    2556 scope.go:117] "RemoveContainer" containerID="677f857b7a8f3e6485c2d144ef1e4ab280abbaf18682494a684a9bc82021469b"
Jul 16 22:43:17.253710 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:17.253684    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:18.354845 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:18.354822    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:18.414861 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:18.414832    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca\") pod \"d2cf402f-1c48-4792-bec2-e544903d7bf4\" (UID: \"d2cf402f-1c48-4792-bec2-e544903d7bf4\") "
Jul 16 22:43:18.415234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:18.415214    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2cf402f-1c48-4792-bec2-e544903d7bf4" (UID: "d2cf402f-1c48-4792-bec2-e544903d7bf4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:43:18.516305 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:18.516235    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2cf402f-1c48-4792-bec2-e544903d7bf4-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:43:19.225889 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:19.225854    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz" event={"ID":"d2cf402f-1c48-4792-bec2-e544903d7bf4","Type":"ContainerDied","Data":"63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d"}
Jul 16 22:43:19.225889 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:19.225878    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"
Jul 16 22:43:19.226149 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:19.225909    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63bc3b2988f8ff6b2431189553d2e20c46fa13e5d0e6e15a04b5a3f9a0bb944d"
Jul 16 22:43:19.253853 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:19.253820    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:19.260312 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:19.260282    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-dlwnz"]
Jul 16 22:43:20.829389 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:20.829355    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" path="/var/lib/kubelet/pods/d2cf402f-1c48-4792-bec2-e544903d7bf4/volumes"
Jul 16 22:43:50.606386 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606348    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"]
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606670    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606680    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606701    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606707    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606751    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.606797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.606764    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:43:50.609788 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.609771    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:50.611916 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.611890    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:43:50.611993 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.611941    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:43:50.620693 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.620648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"]
Jul 16 22:43:50.793136 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.793074    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-knx49\" (UID: \"353369c8-0b84-4141-9c68-3cdeda9b48fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:50.894333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.894254    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-knx49\" (UID: \"353369c8-0b84-4141-9c68-3cdeda9b48fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:50.897050 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.897032    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:43:50.905819 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.905795    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f24d3bde-knx49\" (UID: \"353369c8-0b84-4141-9c68-3cdeda9b48fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:50.923379 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.923356    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:43:50.931880 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:50.931861    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:51.069771 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:51.069730    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"]
Jul 16 22:43:51.072580 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:43:51.072537    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353369c8_0b84_4141_9c68_3cdeda9b48fc.slice/crio-bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d WatchSource:0}: Error finding container bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d: Status 404 returned error can't find the container with id bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d
Jul 16 22:43:51.365996 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:51.365966    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49" event={"ID":"353369c8-0b84-4141-9c68-3cdeda9b48fc","Type":"ContainerStarted","Data":"68767f186bc8903df339f6957c63f7844be7f893b0b8a960e032023206dfa1fa"}
Jul 16 22:43:51.366166 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:51.366024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49" event={"ID":"353369c8-0b84-4141-9c68-3cdeda9b48fc","Type":"ContainerStarted","Data":"bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d"}
Jul 16 22:43:51.384640 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:51.384599    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49" podStartSLOduration=1.38458549 podStartE2EDuration="1.38458549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:43:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:43:51.382705644 +0000 UTC m=+1801.077287519" watchObservedRunningTime="2026-07-16 22:43:51.38458549 +0000 UTC m=+1801.079167338"
Jul 16 22:43:52.371318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:52.371282    2556 generic.go:356] "Generic (PLEG): container finished" podID="353369c8-0b84-4141-9c68-3cdeda9b48fc" containerID="68767f186bc8903df339f6957c63f7844be7f893b0b8a960e032023206dfa1fa" exitCode=0
Jul 16 22:43:52.371721 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:52.371374    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49" event={"ID":"353369c8-0b84-4141-9c68-3cdeda9b48fc","Type":"ContainerDied","Data":"68767f186bc8903df339f6957c63f7844be7f893b0b8a960e032023206dfa1fa"}
Jul 16 22:43:53.505534 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:53.505512    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:53.518851 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:53.518832    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca\") pod \"353369c8-0b84-4141-9c68-3cdeda9b48fc\" (UID: \"353369c8-0b84-4141-9c68-3cdeda9b48fc\") "
Jul 16 22:43:53.519159 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:53.519139    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "353369c8-0b84-4141-9c68-3cdeda9b48fc" (UID: "353369c8-0b84-4141-9c68-3cdeda9b48fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:43:53.619718 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:53.619688    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/353369c8-0b84-4141-9c68-3cdeda9b48fc-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:43:54.382628 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:54.382596    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49"
Jul 16 22:43:54.382909 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:54.382597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f24d3bde-knx49" event={"ID":"353369c8-0b84-4141-9c68-3cdeda9b48fc","Type":"ContainerDied","Data":"bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d"}
Jul 16 22:43:54.382909 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:43:54.382795    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd9bb80368f67fabfe6c7988eed1976e0dd6cf7de4e4b0eb6bfa2fbce4d6c20d"
Jul 16 22:44:01.522791 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:01.522719    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:04.482482 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:04.482450    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:44:05.730550 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:05.730514    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:05.926175 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:05.926147    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:06.059357 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:06.059284    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:12.459776 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:12.459749    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:12.573388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:12.573351    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:12.859931 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:12.859848    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:12.966975 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:12.966950    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:13.073663 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:13.073612    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:14.503490 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:14.503460    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:14.646540 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:14.646504    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:14.836633 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:14.836543    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:15.164452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:15.164377    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:15.281981 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:15.281947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:15.988373 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:15.988346    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:44:18.804317 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804284    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804613    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804624    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804646    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353369c8-0b84-4141-9c68-3cdeda9b48fc" containerName="revoke-keys"
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804652    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="353369c8-0b84-4141-9c68-3cdeda9b48fc" containerName="revoke-keys"
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804700    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2cf402f-1c48-4792-bec2-e544903d7bf4" containerName="revoke-keys"
Jul 16 22:44:18.804726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.804709    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="353369c8-0b84-4141-9c68-3cdeda9b48fc" containerName="revoke-keys"
Jul 16 22:44:18.807532 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.807516    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:18.809641 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.809620    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:44:18.809765 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.809644    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:44:18.817866 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.817844    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:18.924293 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:18.924265    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-58xfq\" (UID: \"2759e011-377c-4c99-b0e1-f9ea61de9d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:19.025041 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:19.025007    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-58xfq\" (UID: \"2759e011-377c-4c99-b0e1-f9ea61de9d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:19.025676 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:19.025658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-58xfq\" (UID: \"2759e011-377c-4c99-b0e1-f9ea61de9d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:19.118718 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:19.118649    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:19.250564 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:19.250539    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:19.253512 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:44:19.253481    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2759e011_377c_4c99_b0e1_f9ea61de9d98.slice/crio-6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596 WatchSource:0}: Error finding container 6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596: Status 404 returned error can't find the container with id 6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596
Jul 16 22:44:19.494957 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:19.494923    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" event={"ID":"2759e011-377c-4c99-b0e1-f9ea61de9d98","Type":"ContainerStarted","Data":"6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596"}
Jul 16 22:44:20.501133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:20.501081    2556 generic.go:356] "Generic (PLEG): container finished" podID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerID="c6e0ee700ab684ae29d1a0c676a181a0e5a13679fe54dc85f29fcb473565868a" exitCode=7
Jul 16 22:44:20.501512 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:20.501163    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" event={"ID":"2759e011-377c-4c99-b0e1-f9ea61de9d98","Type":"ContainerDied","Data":"c6e0ee700ab684ae29d1a0c676a181a0e5a13679fe54dc85f29fcb473565868a"}
Jul 16 22:44:20.501512 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:20.501311    2556 scope.go:117] "RemoveContainer" containerID="c6e0ee700ab684ae29d1a0c676a181a0e5a13679fe54dc85f29fcb473565868a"
Jul 16 22:44:21.508554 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:21.508520    2556 generic.go:356] "Generic (PLEG): container finished" podID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerID="39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51" exitCode=7
Jul 16 22:44:21.508988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:21.508589    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" event={"ID":"2759e011-377c-4c99-b0e1-f9ea61de9d98","Type":"ContainerDied","Data":"39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51"}
Jul 16 22:44:21.508988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:21.508649    2556 scope.go:117] "RemoveContainer" containerID="c6e0ee700ab684ae29d1a0c676a181a0e5a13679fe54dc85f29fcb473565868a"
Jul 16 22:44:21.508988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:21.508800    2556 scope.go:117] "RemoveContainer" containerID="39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51"
Jul 16 22:44:21.509192 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:44:21.509014    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-907fee31-58xfq_odh-ai-gateway-infra(2759e011-377c-4c99-b0e1-f9ea61de9d98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98"
Jul 16 22:44:22.515375 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:22.515347    2556 scope.go:117] "RemoveContainer" containerID="39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51"
Jul 16 22:44:22.515762 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:44:22.515524    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-907fee31-58xfq_odh-ai-gateway-infra(2759e011-377c-4c99-b0e1-f9ea61de9d98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98"
Jul 16 22:44:23.118729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:23.118696    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:44:37.823725 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:37.823693    2556 scope.go:117] "RemoveContainer" containerID="39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51"
Jul 16 22:44:38.590167 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:38.590139    2556 generic.go:356] "Generic (PLEG): container finished" podID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerID="98e1514b4fa8dd5c739d20d9fc7b4c244827223b3d63bf08e4c04a8a4f9f1c59" exitCode=0
Jul 16 22:44:38.590289 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:38.590203    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" event={"ID":"2759e011-377c-4c99-b0e1-f9ea61de9d98","Type":"ContainerDied","Data":"98e1514b4fa8dd5c739d20d9fc7b4c244827223b3d63bf08e4c04a8a4f9f1c59"}
Jul 16 22:44:38.590289 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:38.590260    2556 scope.go:117] "RemoveContainer" containerID="39a1a13d096b11fd8517ba959ce7bb7ba738ee36b5f5151c1dc676f34dfd6b51"
Jul 16 22:44:39.620737 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:39.620699    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:39.725250 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:39.725224    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:39.805394 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:39.805367    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca\") pod \"2759e011-377c-4c99-b0e1-f9ea61de9d98\" (UID: \"2759e011-377c-4c99-b0e1-f9ea61de9d98\") "
Jul 16 22:44:39.805671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:39.805650    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2759e011-377c-4c99-b0e1-f9ea61de9d98" (UID: "2759e011-377c-4c99-b0e1-f9ea61de9d98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:44:39.905987 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:39.905915    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2759e011-377c-4c99-b0e1-f9ea61de9d98-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:44:40.602660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.602629    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"
Jul 16 22:44:40.602862 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.602631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq" event={"ID":"2759e011-377c-4c99-b0e1-f9ea61de9d98","Type":"ContainerDied","Data":"6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596"}
Jul 16 22:44:40.602862 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.602741    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6996b55f70115d00e9df5fcbaa6a2affef8418102d07d648a0a2be8803660596"
Jul 16 22:44:40.636455 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.636419    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:40.642061 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.642036    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-58xfq"]
Jul 16 22:44:40.829325 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:44:40.829297    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" path="/var/lib/kubelet/pods/2759e011-377c-4c99-b0e1-f9ea61de9d98/volumes"
Jul 16 22:45:00.148341 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148309    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"]
Jul 16 22:45:00.148773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148677    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.148773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148690    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.148773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148702    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.148773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148712    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.148773 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148774    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.148935 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.148785    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.151852 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.151835    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"]
Jul 16 22:45:00.151960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.151941    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"
Jul 16 22:45:00.152159 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.152147    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.152199 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.152159    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.152234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.152216    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2759e011-377c-4c99-b0e1-f9ea61de9d98" containerName="revoke-keys"
Jul 16 22:45:00.153939 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.153920    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:45:00.154982 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.154968    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"
Jul 16 22:45:00.160341 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.160322    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"]
Jul 16 22:45:00.163494 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.163472    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"
Jul 16 22:45:00.168729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.167666    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"]
Jul 16 22:45:00.171179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.171159    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"
Jul 16 22:45:00.305516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.305484    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"]
Jul 16 22:45:00.308646 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:45:00.308611    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7d5b83_3009_4645_b5ab_0b586e279d74.slice/crio-d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f WatchSource:0}: Error finding container d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f: Status 404 returned error can't find the container with id d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f
Jul 16 22:45:00.327865 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.327842    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"]
Jul 16 22:45:00.328677 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:45:00.328657    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb332d32a_dc62_4775_80d5_df7015f9009b.slice/crio-49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a WatchSource:0}: Error finding container 49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a: Status 404 returned error can't find the container with id 49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a
Jul 16 22:45:00.693106 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.693049    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh" event={"ID":"b332d32a-dc62-4775-80d5-df7015f9009b","Type":"ContainerStarted","Data":"b1604840fff800150cf79dd2a309d1dcf1faf6f443876eddbcc6173824e38d88"}
Jul 16 22:45:00.693286 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.693134    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh" event={"ID":"b332d32a-dc62-4775-80d5-df7015f9009b","Type":"ContainerStarted","Data":"49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a"}
Jul 16 22:45:00.694654 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.694634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch" event={"ID":"bd7d5b83-3009-4645-b5ab-0b586e279d74","Type":"ContainerStarted","Data":"45f142383706b3fd97a8aaf3424ef3abe7f13667bf83fbdc622930c8b70fe450"}
Jul 16 22:45:00.694732 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.694674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch" event={"ID":"bd7d5b83-3009-4645-b5ab-0b586e279d74","Type":"ContainerStarted","Data":"d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f"}
Jul 16 22:45:00.715804 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.715753    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh" podStartSLOduration=0.715739355 podStartE2EDuration="715.739355ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:00.713914143 +0000 UTC m=+1870.408496016" watchObservedRunningTime="2026-07-16 22:45:00.715739355 +0000 UTC m=+1870.410321211"
Jul 16 22:45:00.731868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:00.731825    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch" podStartSLOduration=0.731814239 podStartE2EDuration="731.814239ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:00.729699378 +0000 UTC m=+1870.424281232" watchObservedRunningTime="2026-07-16 22:45:00.731814239 +0000 UTC m=+1870.426396093"
Jul 16 22:45:01.700790 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:01.700753    2556 generic.go:356] "Generic (PLEG): container finished" podID="b332d32a-dc62-4775-80d5-df7015f9009b" containerID="b1604840fff800150cf79dd2a309d1dcf1faf6f443876eddbcc6173824e38d88" exitCode=0
Jul 16 22:45:01.701240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:01.700837    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh" event={"ID":"b332d32a-dc62-4775-80d5-df7015f9009b","Type":"ContainerDied","Data":"b1604840fff800150cf79dd2a309d1dcf1faf6f443876eddbcc6173824e38d88"}
Jul 16 22:45:01.702284 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:01.702259    2556 generic.go:356] "Generic (PLEG): container finished" podID="bd7d5b83-3009-4645-b5ab-0b586e279d74" containerID="45f142383706b3fd97a8aaf3424ef3abe7f13667bf83fbdc622930c8b70fe450" exitCode=0
Jul 16 22:45:01.702393 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:01.702337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch" event={"ID":"bd7d5b83-3009-4645-b5ab-0b586e279d74","Type":"ContainerDied","Data":"45f142383706b3fd97a8aaf3424ef3abe7f13667bf83fbdc622930c8b70fe450"}
Jul 16 22:45:02.848580 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:02.848548    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"
Jul 16 22:45:02.851771 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:02.851749    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"
Jul 16 22:45:03.712268 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.712242    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh"
Jul 16 22:45:03.712453 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.712242    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737365-m7tfh" event={"ID":"b332d32a-dc62-4775-80d5-df7015f9009b","Type":"ContainerDied","Data":"49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a"}
Jul 16 22:45:03.712453 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.712366    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49cdc2568b6ef617e3d855c7de1d50c314c5b679696ba1b13a462cec234b057a"
Jul 16 22:45:03.713477 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.713457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch" event={"ID":"bd7d5b83-3009-4645-b5ab-0b586e279d74","Type":"ContainerDied","Data":"d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f"}
Jul 16 22:45:03.713572 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.713492    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d12bee65071c4103bf3c4727717a24d55913d03b00b1678349e0befa46e0461f"
Jul 16 22:45:03.713572 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:03.713490    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"
Jul 16 22:45:18.869309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869233    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"]
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869559    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7d5b83-3009-4645-b5ab-0b586e279d74" containerName="cleanup"
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869569    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7d5b83-3009-4645-b5ab-0b586e279d74" containerName="cleanup"
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869583    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b332d32a-dc62-4775-80d5-df7015f9009b" containerName="cleanup"
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869588    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b332d32a-dc62-4775-80d5-df7015f9009b" containerName="cleanup"
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869641    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7d5b83-3009-4645-b5ab-0b586e279d74" containerName="cleanup"
Jul 16 22:45:18.869739 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.869653    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b332d32a-dc62-4775-80d5-df7015f9009b" containerName="cleanup"
Jul 16 22:45:18.872877 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.872861    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:18.875178 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.875153    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:45:18.875322 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.875239    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:45:18.882207 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.882186    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"]
Jul 16 22:45:18.941611 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.941581    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-v75ck\" (UID: \"554ddccd-4025-4b10-aa29-ab092027ddad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:18.992480 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:18.992455    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:19.042267 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.042240    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-v75ck\" (UID: \"554ddccd-4025-4b10-aa29-ab092027ddad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:19.042846 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.042827    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-907fee31-v75ck\" (UID: \"554ddccd-4025-4b10-aa29-ab092027ddad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:19.093419 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.093360    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:19.184755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.184681    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:19.311418 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.311378    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"]
Jul 16 22:45:19.314010 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:45:19.313985    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554ddccd_4025_4b10_aa29_ab092027ddad.slice/crio-f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf WatchSource:0}: Error finding container f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf: Status 404 returned error can't find the container with id f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf
Jul 16 22:45:19.787105 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.787060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck" event={"ID":"554ddccd-4025-4b10-aa29-ab092027ddad","Type":"ContainerStarted","Data":"71d675df7249c294834c6046ace2af636316549dad0691dfcd99792ec69de9e0"}
Jul 16 22:45:19.787275 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.787137    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck" event={"ID":"554ddccd-4025-4b10-aa29-ab092027ddad","Type":"ContainerStarted","Data":"f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf"}
Jul 16 22:45:19.805030 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:19.804944    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck" podStartSLOduration=1.804930948 podStartE2EDuration="1.804930948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:45:19.804336942 +0000 UTC m=+1889.498918796" watchObservedRunningTime="2026-07-16 22:45:19.804930948 +0000 UTC m=+1889.499512802"
Jul 16 22:45:20.792267 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:20.792236    2556 generic.go:356] "Generic (PLEG): container finished" podID="554ddccd-4025-4b10-aa29-ab092027ddad" containerID="71d675df7249c294834c6046ace2af636316549dad0691dfcd99792ec69de9e0" exitCode=0
Jul 16 22:45:20.792662 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:20.792324    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck" event={"ID":"554ddccd-4025-4b10-aa29-ab092027ddad","Type":"ContainerDied","Data":"71d675df7249c294834c6046ace2af636316549dad0691dfcd99792ec69de9e0"}
Jul 16 22:45:21.748072 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:21.748038    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:45:21.920797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:21.920777    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:22.065468 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.065393    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca\") pod \"554ddccd-4025-4b10-aa29-ab092027ddad\" (UID: \"554ddccd-4025-4b10-aa29-ab092027ddad\") "
Jul 16 22:45:22.065725 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.065704    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "554ddccd-4025-4b10-aa29-ab092027ddad" (UID: "554ddccd-4025-4b10-aa29-ab092027ddad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:45:22.166058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.166028    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/554ddccd-4025-4b10-aa29-ab092027ddad-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:45:22.802993 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.802967    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck"
Jul 16 22:45:22.803179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.803013    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-907fee31-v75ck" event={"ID":"554ddccd-4025-4b10-aa29-ab092027ddad","Type":"ContainerDied","Data":"f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf"}
Jul 16 22:45:22.803179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:22.803055    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f96b7ab697eee392c77d8f0b0b7c6fb1998ba33beb7a322e0e54bc77c4395edf"
Jul 16 22:45:25.249206 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:25.249169    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"]
Jul 16 22:45:25.252972 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:25.252948    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-907fee31-29737365-h9xch"]
Jul 16 22:45:25.255005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:25.254977    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:25.757062 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:25.757033    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:25.851237 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:25.851208    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:26.829405 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:26.829374    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7d5b83-3009-4645-b5ab-0b586e279d74" path="/var/lib/kubelet/pods/bd7d5b83-3009-4645-b5ab-0b586e279d74/volumes"
Jul 16 22:45:31.291782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:31.291746    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:31.391659 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:31.391634    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:31.553275 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:31.553201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:31.647099 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:31.647071    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:32.758140 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:32.758112    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:32.839487 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:32.839445    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:32.917331 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:32.917291    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:33.045958 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:33.045868    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:33.573742 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:33.573703    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:33.671726 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:33.671681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:34.847534 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:34.847510    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:38.046673 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.046639    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"]
Jul 16 22:45:38.047108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.046975    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="554ddccd-4025-4b10-aa29-ab092027ddad" containerName="revoke-keys"
Jul 16 22:45:38.047108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.046986    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="554ddccd-4025-4b10-aa29-ab092027ddad" containerName="revoke-keys"
Jul 16 22:45:38.047108 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.047048    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="554ddccd-4025-4b10-aa29-ab092027ddad" containerName="revoke-keys"
Jul 16 22:45:38.050300 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.050277    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:38.052317 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.052291    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:45:38.052431 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.052373    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:45:38.058440 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.057846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"]
Jul 16 22:45:38.209077 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.209031    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-667ee841-blb4v\" (UID: \"136a456d-e8a7-49b6-8b7a-c6d3451a304a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:38.310304 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.310208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-667ee841-blb4v\" (UID: \"136a456d-e8a7-49b6-8b7a-c6d3451a304a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:38.310838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.310814    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-667ee841-blb4v\" (UID: \"136a456d-e8a7-49b6-8b7a-c6d3451a304a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:38.364184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.364155    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:38.701514 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.701483    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"]
Jul 16 22:45:38.704278 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:45:38.704254    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136a456d_e8a7_49b6_8b7a_c6d3451a304a.slice/crio-5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc WatchSource:0}: Error finding container 5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc: Status 404 returned error can't find the container with id 5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc
Jul 16 22:45:38.884883 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:38.884844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v" event={"ID":"136a456d-e8a7-49b6-8b7a-c6d3451a304a","Type":"ContainerStarted","Data":"5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc"}
Jul 16 22:45:39.891234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:39.891197    2556 generic.go:356] "Generic (PLEG): container finished" podID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerID="21eafd49e5d05742ee24ef98c8e6f5b866ca97966b39302321bec84730eb4679" exitCode=7
Jul 16 22:45:39.891612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:39.891283    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v" event={"ID":"136a456d-e8a7-49b6-8b7a-c6d3451a304a","Type":"ContainerDied","Data":"21eafd49e5d05742ee24ef98c8e6f5b866ca97966b39302321bec84730eb4679"}
Jul 16 22:45:39.891612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:39.891457    2556 scope.go:117] "RemoveContainer" containerID="21eafd49e5d05742ee24ef98c8e6f5b866ca97966b39302321bec84730eb4679"
Jul 16 22:45:40.347080 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:40.347046    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:45:40.897309 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:40.897219    2556 generic.go:356] "Generic (PLEG): container finished" podID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerID="3760052f2c1b3301b6f8900692678dc75d9eb754d4a7795c1671e5f556b94ee3" exitCode=0
Jul 16 22:45:40.897762 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:40.897312    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v" event={"ID":"136a456d-e8a7-49b6-8b7a-c6d3451a304a","Type":"ContainerDied","Data":"3760052f2c1b3301b6f8900692678dc75d9eb754d4a7795c1671e5f556b94ee3"}
Jul 16 22:45:40.897762 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:40.897386    2556 scope.go:117] "RemoveContainer" containerID="21eafd49e5d05742ee24ef98c8e6f5b866ca97966b39302321bec84730eb4679"
Jul 16 22:45:42.032435 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.032409    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:42.143536 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.143497    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca\") pod \"136a456d-e8a7-49b6-8b7a-c6d3451a304a\" (UID: \"136a456d-e8a7-49b6-8b7a-c6d3451a304a\") "
Jul 16 22:45:42.143916 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.143896    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "136a456d-e8a7-49b6-8b7a-c6d3451a304a" (UID: "136a456d-e8a7-49b6-8b7a-c6d3451a304a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:45:42.244448 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.244414    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/136a456d-e8a7-49b6-8b7a-c6d3451a304a-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:45:42.909805 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.909775    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v"
Jul 16 22:45:42.909979 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.909810    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-667ee841-blb4v" event={"ID":"136a456d-e8a7-49b6-8b7a-c6d3451a304a","Type":"ContainerDied","Data":"5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc"}
Jul 16 22:45:42.909979 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:42.909861    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e8dac06a86b9edcb0d8056a65fbdff40a74c9c56cabaca02d80aca51f57cacc"
Jul 16 22:45:48.738449 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:48.738400    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:51.445357 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:51.445323    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:45:53.180813 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:53.180773    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:54.459294 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:54.459255    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:45:54.570058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:45:54.570025    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:00.634429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:00.634390    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:00.741037 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:00.741007    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:00.838799 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:00.838767    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:00.973387 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:00.973339    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:01.071106 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:01.071035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:02.261700 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:02.261636    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:02.413296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:02.413238    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:02.540186 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:02.540062    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:02.729307 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:02.729269    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:03.279868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:03.279825    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:03.395277 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:03.395223    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:04.439240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:04.439209    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:46:06.562012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.561975    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562321    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562333    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562344    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562350    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562418    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.562429 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.562428    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="136a456d-e8a7-49b6-8b7a-c6d3451a304a" containerName="revoke-keys"
Jul 16 22:46:06.565390 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.565374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:06.568410 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.568391    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:46:06.569163 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.569149    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:46:06.586368 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.586342    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:06.661423 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.661387    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8\" (UID: \"efe5aef4-a40c-4163-a5c7-7549c42332cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:06.762533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.762497    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8\" (UID: \"efe5aef4-a40c-4163-a5c7-7549c42332cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:06.763175 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.763156    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8\" (UID: \"efe5aef4-a40c-4163-a5c7-7549c42332cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:06.875426 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:06.875333    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:07.010963 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:07.010940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:07.013340 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:46:07.013314    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefe5aef4_a40c_4163_a5c7_7549c42332cd.slice/crio-07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9 WatchSource:0}: Error finding container 07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9: Status 404 returned error can't find the container with id 07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9
Jul 16 22:46:07.025953 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:07.025928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerStarted","Data":"07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9"}
Jul 16 22:46:08.032352 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:08.032316    2556 generic.go:356] "Generic (PLEG): container finished" podID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerID="771c10f616e8dc5d20e20822aa9c5a09a2a1df11940e09a49b291ef19e4eeace" exitCode=7
Jul 16 22:46:08.032756 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:08.032389    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerDied","Data":"771c10f616e8dc5d20e20822aa9c5a09a2a1df11940e09a49b291ef19e4eeace"}
Jul 16 22:46:08.032756 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:08.032646    2556 scope.go:117] "RemoveContainer" containerID="771c10f616e8dc5d20e20822aa9c5a09a2a1df11940e09a49b291ef19e4eeace"
Jul 16 22:46:08.445923 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:08.445894    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:46:09.038814 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:09.038729    2556 generic.go:356] "Generic (PLEG): container finished" podID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerID="023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f" exitCode=7
Jul 16 22:46:09.039258 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:09.038813    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerDied","Data":"023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f"}
Jul 16 22:46:09.039258 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:09.038875    2556 scope.go:117] "RemoveContainer" containerID="771c10f616e8dc5d20e20822aa9c5a09a2a1df11940e09a49b291ef19e4eeace"
Jul 16 22:46:09.039258 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:09.039067    2556 scope.go:117] "RemoveContainer" containerID="023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f"
Jul 16 22:46:09.039368 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:46:09.039307    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8_odh-ai-gateway-infra(efe5aef4-a40c-4163-a5c7-7549c42332cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd"
Jul 16 22:46:10.046012 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:10.045987    2556 scope.go:117] "RemoveContainer" containerID="023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f"
Jul 16 22:46:10.046392 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:46:10.046186    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8_odh-ai-gateway-infra(efe5aef4-a40c-4163-a5c7-7549c42332cd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd"
Jul 16 22:46:24.823797 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:24.823769    2556 scope.go:117] "RemoveContainer" containerID="023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f"
Jul 16 22:46:25.119474 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:25.119389    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerStarted","Data":"1e4aa4c36f208e5c3ab88d6e119daebdba83161d4ac28db0573d7a1a9af3cb08"}
Jul 16 22:46:26.125751 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:26.125719    2556 generic.go:356] "Generic (PLEG): container finished" podID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerID="1e4aa4c36f208e5c3ab88d6e119daebdba83161d4ac28db0573d7a1a9af3cb08" exitCode=0
Jul 16 22:46:26.126133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:26.125795    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerDied","Data":"1e4aa4c36f208e5c3ab88d6e119daebdba83161d4ac28db0573d7a1a9af3cb08"}
Jul 16 22:46:26.126133 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:26.125853    2556 scope.go:117] "RemoveContainer" containerID="023df81b6f61d02d4763bdff9277d51898f9ef4df4c1fa384d8fe0679c51d35f"
Jul 16 22:46:26.153467 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:26.153441    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:27.265128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:27.265102    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:27.351981 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:27.351949    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca\") pod \"efe5aef4-a40c-4163-a5c7-7549c42332cd\" (UID: \"efe5aef4-a40c-4163-a5c7-7549c42332cd\") "
Jul 16 22:46:27.352338 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:27.352316    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efe5aef4-a40c-4163-a5c7-7549c42332cd" (UID: "efe5aef4-a40c-4163-a5c7-7549c42332cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:46:27.452594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:27.452564    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efe5aef4-a40c-4163-a5c7-7549c42332cd-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:46:28.138492 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.138467    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"
Jul 16 22:46:28.138694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.138491    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8" event={"ID":"efe5aef4-a40c-4163-a5c7-7549c42332cd","Type":"ContainerDied","Data":"07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9"}
Jul 16 22:46:28.138694 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.138537    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07aeca98b4b9f5de4648c9f99892d65f8fae2d6b26a164f711a43f8ed38ba3a9"
Jul 16 22:46:28.172856 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.172827    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:28.178956 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.178934    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-hndk8"]
Jul 16 22:46:28.829375 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:46:28.829343    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" path="/var/lib/kubelet/pods/efe5aef4-a40c-4163-a5c7-7549c42332cd/volumes"
Jul 16 22:47:06.696145 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696082    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"]
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696444    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696454    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696470    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696475    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696532    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696541    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.696578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.696548    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:47:06.698994 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.698978    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:06.702047 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.702026    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:47:06.702957 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.702942    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:47:06.715033 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.715012    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"]
Jul 16 22:47:06.887280 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.887243    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw\" (UID: \"6f5711d3-2abd-4e83-b725-89dd563f67ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:06.988075 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.987997    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw\" (UID: \"6f5711d3-2abd-4e83-b725-89dd563f67ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:06.988629 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:06.988610    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw\" (UID: \"6f5711d3-2abd-4e83-b725-89dd563f67ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:07.009026 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:07.009001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:07.154533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:07.154508    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"]
Jul 16 22:47:07.156053 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:47:07.156026    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5711d3_2abd_4e83_b725_89dd563f67ba.slice/crio-01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3 WatchSource:0}: Error finding container 01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3: Status 404 returned error can't find the container with id 01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3
Jul 16 22:47:07.320124 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:07.320037    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw" event={"ID":"6f5711d3-2abd-4e83-b725-89dd563f67ba","Type":"ContainerStarted","Data":"01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3"}
Jul 16 22:47:08.331791 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:08.331754    2556 generic.go:356] "Generic (PLEG): container finished" podID="6f5711d3-2abd-4e83-b725-89dd563f67ba" containerID="84658cc1151b19b39013dd6832ae865418c72b1ba8a0ad6cb5e9b4f2981c8fe7" exitCode=0
Jul 16 22:47:08.332210 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:08.331836    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw" event={"ID":"6f5711d3-2abd-4e83-b725-89dd563f67ba","Type":"ContainerDied","Data":"84658cc1151b19b39013dd6832ae865418c72b1ba8a0ad6cb5e9b4f2981c8fe7"}
Jul 16 22:47:09.472985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.472959    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:09.610044 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.609964    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca\") pod \"6f5711d3-2abd-4e83-b725-89dd563f67ba\" (UID: \"6f5711d3-2abd-4e83-b725-89dd563f67ba\") "
Jul 16 22:47:09.610384 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.610351    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f5711d3-2abd-4e83-b725-89dd563f67ba" (UID: "6f5711d3-2abd-4e83-b725-89dd563f67ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:47:09.710993 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.710959    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5711d3-2abd-4e83-b725-89dd563f67ba-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:47:09.885730 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.885624    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:09.988124 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:09.988085    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:10.343521 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:10.343488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw" event={"ID":"6f5711d3-2abd-4e83-b725-89dd563f67ba","Type":"ContainerDied","Data":"01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3"}
Jul 16 22:47:10.343521 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:10.343524    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw"
Jul 16 22:47:10.343732 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:10.343541    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01248daa5470b644826972ace380cfcd237c407e0276ea522b315b4bb8a4d7e3"
Jul 16 22:47:17.552666 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:17.552621    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:20.539125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:20.539078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:47:21.787921 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:21.787889    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:22.195515 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:22.195484    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:22.353178 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:22.353143    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.038313 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.038279    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.235600 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.235565    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.400296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.400270    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.562725 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.562693    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.659281 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.659200    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:28.755883 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:28.755850    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:30.241880 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:30.241847    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:36.223891 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:36.223860    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:36.332025 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:36.331991    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:36.426864 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:36.426831    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:36.527803 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:36.527718    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:47:37.044312 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:47:37.044280    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:03.023611 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:03.023583    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:08.985933 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:08.985899    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:09.090902 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:09.090874    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:09.202498 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:09.202471    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:24.484789 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:24.484758    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:24.587375 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:24.587332    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:25.150402 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:25.150336    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:30.934476 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:30.934445    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:31.045475 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:31.045446    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:31.135140 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:31.135086    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:41.689438 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:41.689399    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:42.529606 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:42.529578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:49.578267 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:49.578227    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:49.692907 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:49.692876    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:49.897995 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:49.897916    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:50.013748 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:50.013719    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:50.707444 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:50.707416    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:50.807645 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:50.807609    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:50.907122 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:50.907077    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:52.694062 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:52.694030    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:52.944411 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:52.944333    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:48:59.001298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:59.001259    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:59.100326 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:59.100295    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:48:59.888367 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:48:59.888337    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:02.936342 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:02.936310    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:49:04.836806 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:04.836776    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:06.115846 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:06.115815    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:06.215922 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:06.215892    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:06.393616 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:06.393543    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:06.747497 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:06.747465    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:06.850389 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:06.850362    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:09.832974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:09.832945    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:09.927140 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:09.927103    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:10.000107 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:10.000066    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:10.245001 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:10.244976    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:10.552439 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:10.552361    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:10.659111 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:10.659071    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:12.942699 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:12.942655    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:49:14.260182 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:14.260151    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:15.062898 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:15.062869    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:15.152583 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:15.152547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:15.420445 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:15.420363    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:21.135455 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.135419    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:21.206482 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.206445    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:21.503677 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.503646    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"]
Jul 16 22:49:21.503973 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.503961    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:49:21.504017 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.503974    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe5aef4-a40c-4163-a5c7-7549c42332cd" containerName="revoke-keys"
Jul 16 22:49:21.504017 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.503996    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5711d3-2abd-4e83-b725-89dd563f67ba" containerName="revoke-keys"
Jul 16 22:49:21.504017 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.504001    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5711d3-2abd-4e83-b725-89dd563f67ba" containerName="revoke-keys"
Jul 16 22:49:21.504135 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.504057    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5711d3-2abd-4e83-b725-89dd563f67ba" containerName="revoke-keys"
Jul 16 22:49:21.506938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.506923    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:21.509158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.509139    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:49:21.509158 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.509155    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:49:21.513065 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.513042    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"]
Jul 16 22:49:21.621027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.620988    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx\" (UID: \"01d3686b-43d3-4103-afbb-f1757a038045\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:21.722533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.722496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx\" (UID: \"01d3686b-43d3-4103-afbb-f1757a038045\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:21.723197 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.723171    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx\" (UID: \"01d3686b-43d3-4103-afbb-f1757a038045\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:21.819298 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.819226    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:21.944026 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.944003    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"]
Jul 16 22:49:21.946134 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:49:21.946111    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d3686b_43d3_4103_afbb_f1757a038045.slice/crio-59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e WatchSource:0}: Error finding container 59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e: Status 404 returned error can't find the container with id 59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e
Jul 16 22:49:21.953282 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:21.953255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx" event={"ID":"01d3686b-43d3-4103-afbb-f1757a038045","Type":"ContainerStarted","Data":"59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e"}
Jul 16 22:49:22.958794 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:22.958764    2556 generic.go:356] "Generic (PLEG): container finished" podID="01d3686b-43d3-4103-afbb-f1757a038045" containerID="0c309b9f88b296aac0a830ec5c5209cd0f6edcb7d87f5e67b0538886c2f9ced7" exitCode=0
Jul 16 22:49:22.959180 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:22.958856    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx" event={"ID":"01d3686b-43d3-4103-afbb-f1757a038045","Type":"ContainerDied","Data":"0c309b9f88b296aac0a830ec5c5209cd0f6edcb7d87f5e67b0538886c2f9ced7"}
Jul 16 22:49:24.086965 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.086945    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:24.142027 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.141999    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca\") pod \"01d3686b-43d3-4103-afbb-f1757a038045\" (UID: \"01d3686b-43d3-4103-afbb-f1757a038045\") "
Jul 16 22:49:24.142345 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.142327    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01d3686b-43d3-4103-afbb-f1757a038045" (UID: "01d3686b-43d3-4103-afbb-f1757a038045"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:49:24.242936 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.242862    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d3686b-43d3-4103-afbb-f1757a038045-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:49:24.969788 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.969758    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx"
Jul 16 22:49:24.969952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.969765    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx" event={"ID":"01d3686b-43d3-4103-afbb-f1757a038045","Type":"ContainerDied","Data":"59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e"}
Jul 16 22:49:24.969952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:24.969880    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59ca7a9eb05ebff167926b49f20ce2a427902e55973ad9d58475121314ef712e"
Jul 16 22:49:32.169722 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:32.169687    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:34.950110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:34.950052    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:49:36.665992 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:36.665960    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:37.062331 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:37.062303    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.017801 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.017713    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.137550 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.137520    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.297528 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.297449    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.540075 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.540046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.674131 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.674043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.771315 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.771289    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:43.915853 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:43.915825    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:49.848144 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:49.848112    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:49.955525 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:49.955492    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:50.066978 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:50.066949    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:50.249899 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:50.249871    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:50.920644 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:50.920603    2556 scope.go:117] "RemoveContainer" containerID="b00d9309384ce5aec126dd77d2b1a0ce9a3301636fcec272d49c32a05e57e470"
Jul 16 22:49:55.878580 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:55.878550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:49:55.954798 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:49:55.954766    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:02.890179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:02.890147    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:03.190952 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:03.190925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:03.313563 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:03.313530    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:05.976157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:05.976126    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:06.083317 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:06.083287    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:06.177256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:06.177224    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:06.496288 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:06.496259    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:06.723999 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:06.723969    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:06.827261 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:06.827172    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:10.467328 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:10.467298    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:11.411245 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:11.411212    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:50:30.342204 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.342175    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:50:30.342607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.342503    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01d3686b-43d3-4103-afbb-f1757a038045" containerName="revoke-keys"
Jul 16 22:50:30.342607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.342513    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="01d3686b-43d3-4103-afbb-f1757a038045" containerName="revoke-keys"
Jul 16 22:50:30.342607 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.342565    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="01d3686b-43d3-4103-afbb-f1757a038045" containerName="revoke-keys"
Jul 16 22:50:30.345513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.345498    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.347887 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.347851    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8a492916\"/\"e2e-lifecycle-model-8a492916-kserve-self-signed-certs\""
Jul 16 22:50:30.348178 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.348159    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8a492916\"/\"kube-root-ca.crt\""
Jul 16 22:50:30.348466 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.348447    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8a492916\"/\"openshift-service-ca.crt\""
Jul 16 22:50:30.348533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.348520    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8a492916\"/\"default-dockercfg-k9zjl\""
Jul 16 22:50:30.357626 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.357606    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:50:30.368820 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.368798    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:50:30.519472 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519438    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.519472 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519476    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.519689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519495    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.519689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519528    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q6rf\" (UniqueName: \"kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.519689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.519689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.519610    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.620937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.620866    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.620937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.620896    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.620937 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.620925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q6rf\" (UniqueName: \"kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621044    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621173 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621215    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621313 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621369 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.621484 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.621463    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.623155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.623131    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.623551 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.623532    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.629243 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.629227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q6rf\" (UniqueName: \"kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf\") pod \"e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") " pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.654408 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.654388    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:30.781054 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:30.781028    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:50:30.783720 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:50:30.783696    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode19d4827_80f6_435f_a982_4f769e419ceb.slice/crio-fbd0dbb737b324077015cc54fab7b02dae04299b1f29ba9bd80fd6fe83b72b23 WatchSource:0}: Error finding container fbd0dbb737b324077015cc54fab7b02dae04299b1f29ba9bd80fd6fe83b72b23: Status 404 returned error can't find the container with id fbd0dbb737b324077015cc54fab7b02dae04299b1f29ba9bd80fd6fe83b72b23
Jul 16 22:50:31.284908 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:31.284871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" event={"ID":"e19d4827-80f6-435f-a982-4f769e419ceb","Type":"ContainerStarted","Data":"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"}
Jul 16 22:50:31.285119 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:31.284924    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" event={"ID":"e19d4827-80f6-435f-a982-4f769e419ceb","Type":"ContainerStarted","Data":"fbd0dbb737b324077015cc54fab7b02dae04299b1f29ba9bd80fd6fe83b72b23"}
Jul 16 22:50:31.328019 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:31.327964    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" podStartSLOduration=1.327948636 podStartE2EDuration="1.327948636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:50:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:50:31.318134821 +0000 UTC m=+2201.012716681" watchObservedRunningTime="2026-07-16 22:50:31.327948636 +0000 UTC m=+2201.022530489"
Jul 16 22:50:33.735969 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:33.735923    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:50:40.654619 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:40.654578    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:40.655032 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:40.654646    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:40.667259 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:40.667227    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:41.346043 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:41.346015    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:50:50.137049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:50.137011    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:50:50.941500 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:50.941464    2556 scope.go:117] "RemoveContainer" containerID="98e1514b4fa8dd5c739d20d9fc7b4c244827223b3d63bf08e4c04a8a4f9f1c59"
Jul 16 22:50:54.275006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.274967    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"]
Jul 16 22:50:54.280005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.279985    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:54.281951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.281932    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:50:54.282023 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.281978    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:50:54.286319 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.286296    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"]
Jul 16 22:50:54.317246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.317208    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8a492916-wmbwr\" (UID: \"37040a06-662d-4799-9fed-8774764b4bdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:54.417954 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.417913    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8a492916-wmbwr\" (UID: \"37040a06-662d-4799-9fed-8774764b4bdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:54.418625 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.418606    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8a492916-wmbwr\" (UID: \"37040a06-662d-4799-9fed-8774764b4bdf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:54.592782 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.592692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:54.728704 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:54.728679    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"]
Jul 16 22:50:54.730348 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:50:54.730317    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37040a06_662d_4799_9fed_8774764b4bdf.slice/crio-665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617 WatchSource:0}: Error finding container 665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617: Status 404 returned error can't find the container with id 665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617
Jul 16 22:50:55.403781 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:55.403679    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr" event={"ID":"37040a06-662d-4799-9fed-8774764b4bdf","Type":"ContainerStarted","Data":"ac14a8bda1345a0de176b0f2ddaf4e786a7f5946c9dc72004139fcd1bb53ddc7"}
Jul 16 22:50:55.403781 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:55.403745    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr" event={"ID":"37040a06-662d-4799-9fed-8774764b4bdf","Type":"ContainerStarted","Data":"665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617"}
Jul 16 22:50:55.419354 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:55.419302    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr" podStartSLOduration=1.419286024 podStartE2EDuration="1.419286024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:50:55.418347185 +0000 UTC m=+2225.112929058" watchObservedRunningTime="2026-07-16 22:50:55.419286024 +0000 UTC m=+2225.113867878"
Jul 16 22:50:56.409671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:56.409636    2556 generic.go:356] "Generic (PLEG): container finished" podID="37040a06-662d-4799-9fed-8774764b4bdf" containerID="ac14a8bda1345a0de176b0f2ddaf4e786a7f5946c9dc72004139fcd1bb53ddc7" exitCode=0
Jul 16 22:50:56.410071 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:56.409730    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr" event={"ID":"37040a06-662d-4799-9fed-8774764b4bdf","Type":"ContainerDied","Data":"ac14a8bda1345a0de176b0f2ddaf4e786a7f5946c9dc72004139fcd1bb53ddc7"}
Jul 16 22:50:57.547986 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:57.547960    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:57.647829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:57.647789    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca\") pod \"37040a06-662d-4799-9fed-8774764b4bdf\" (UID: \"37040a06-662d-4799-9fed-8774764b4bdf\") "
Jul 16 22:50:57.648239 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:57.648214    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37040a06-662d-4799-9fed-8774764b4bdf" (UID: "37040a06-662d-4799-9fed-8774764b4bdf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:50:57.748430 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:57.748391    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37040a06-662d-4799-9fed-8774764b4bdf-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:50:58.420070 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:58.420043    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr"
Jul 16 22:50:58.420070 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:58.420064    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8a492916-wmbwr" event={"ID":"37040a06-662d-4799-9fed-8774764b4bdf","Type":"ContainerDied","Data":"665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617"}
Jul 16 22:50:58.420319 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:50:58.420138    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="665edafbf94deeefcf18efb63bc27802f9c188b00c756beb2d218c92bf689617"
Jul 16 22:51:01.482181 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:01.482146    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:01.586941 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:01.586888    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:04.491976 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:04.491940    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:05.039392 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:05.039352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:05.138482 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:05.138445    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:08.937420 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:08.937382    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:51:10.333613 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:10.333578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:14.373897 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:14.373804    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:16.695766 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:16.695732    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:16.760988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:16.760952    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:51:21.663854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:21.663819    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:51:21.760623 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:21.760564    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:36.685011 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:36.684951    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" podUID="e19d4827-80f6-435f-a982-4f769e419ceb" containerName="main" containerID="cri-o://6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42" gracePeriod=45
Jul 16 22:51:36.939110 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:36.939065    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:51:37.091555 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091517    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.091555 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091564    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.091827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091595    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.091827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091615    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q6rf\" (UniqueName: \"kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.091827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091676    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.091827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091701    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache\") pod \"e19d4827-80f6-435f-a982-4f769e419ceb\" (UID: \"e19d4827-80f6-435f-a982-4f769e419ceb\") "
Jul 16 22:51:37.092031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091887    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:51:37.092031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.091931    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home" (OuterVolumeSpecName: "home") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:51:37.092031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.092002    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache" (OuterVolumeSpecName: "model-cache") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:51:37.093987 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.093966    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm" (OuterVolumeSpecName: "dshm") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:51:37.093987 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.093971    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:51:37.094128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.094037    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf" (OuterVolumeSpecName: "kube-api-access-6q6rf") pod "e19d4827-80f6-435f-a982-4f769e419ceb" (UID: "e19d4827-80f6-435f-a982-4f769e419ceb"). InnerVolumeSpecName "kube-api-access-6q6rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192870    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192925    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192934    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e19d4827-80f6-435f-a982-4f769e419ceb-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192942    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192953    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q6rf\" (UniqueName: \"kubernetes.io/projected/e19d4827-80f6-435f-a982-4f769e419ceb-kube-api-access-6q6rf\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.192960 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.192963    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e19d4827-80f6-435f-a982-4f769e419ceb-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:51:37.611565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.611474    2556 generic.go:356] "Generic (PLEG): container finished" podID="e19d4827-80f6-435f-a982-4f769e419ceb" containerID="6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42" exitCode=0
Jul 16 22:51:37.611565 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.611520    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"
Jul 16 22:51:37.611753 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.611557    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" event={"ID":"e19d4827-80f6-435f-a982-4f769e419ceb","Type":"ContainerDied","Data":"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"}
Jul 16 22:51:37.611753 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.611608    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6" event={"ID":"e19d4827-80f6-435f-a982-4f769e419ceb","Type":"ContainerDied","Data":"fbd0dbb737b324077015cc54fab7b02dae04299b1f29ba9bd80fd6fe83b72b23"}
Jul 16 22:51:37.611753 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.611625    2556 scope.go:117] "RemoveContainer" containerID="6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"
Jul 16 22:51:37.620599 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.620581    2556 scope.go:117] "RemoveContainer" containerID="6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"
Jul 16 22:51:37.620880 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:51:37.620862    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42\": container with ID starting with 6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42 not found: ID does not exist" containerID="6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"
Jul 16 22:51:37.620928 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.620890    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42"} err="failed to get container status \"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42\": rpc error: code = NotFound desc = could not find container \"6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42\": container with ID starting with 6b6b086495adf7a82c9565ee0ec3543606ee418099101c447f6a65b99ea20c42 not found: ID does not exist"
Jul 16 22:51:37.631399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.631367    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:51:37.637555 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:37.637530    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8a492916/e2e-lifecycle-model-8a492916-kserve-846c76799-n7mb6"]
Jul 16 22:51:38.828943 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:38.828904    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e19d4827-80f6-435f-a982-4f769e419ceb" path="/var/lib/kubelet/pods/e19d4827-80f6-435f-a982-4f769e419ceb/volumes"
Jul 16 22:51:50.963557 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:50.963526    2556 scope.go:117] "RemoveContainer" containerID="45f142383706b3fd97a8aaf3424ef3abe7f13667bf83fbdc622930c8b70fe450"
Jul 16 22:51:58.294341 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:58.294305    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:51:59.048680 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:51:59.048650    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:04.881834 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:04.881798    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:04.960128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:04.960085    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:05.192293 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:05.192256    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:05.405900 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:05.405863    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:06.548231 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:06.548196    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:06.639255 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:06.639219    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:06.741545 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:06.741502    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:07.528145 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:07.528112    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:07.616538 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:07.616503    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:07.734925 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:07.734894    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:08.970622 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:08.970585    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:52:09.266235 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:09.266151    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:15.482737 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:15.482705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:15.554388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:15.554357    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:16.180496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:16.180457    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:19.451555 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:19.451519    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:52:22.648326 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:22.648291    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:22.751777 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:22.751742    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:22.856792 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:22.856754    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:33.424207 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:33.424171    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:33.790126 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:33.790080    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:36.852972 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:36.852940    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:52:50.986333 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:50.986253    2556 scope.go:117] "RemoveContainer" containerID="1e4aa4c36f208e5c3ab88d6e119daebdba83161d4ac28db0573d7a1a9af3cb08"
Jul 16 22:52:54.847319 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:52:54.847284    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:01.475219 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:01.475178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:01.575274 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:01.575241    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:01.851640 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:01.851519    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:02.236780 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:02.236748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:02.507174 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:02.507066    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:04.815003 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:04.814968    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:06.109698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:06.109656    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:09.370985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:09.370951    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:09.828212 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:09.828184    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:10.146031 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:10.145946    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:10.203970 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:10.203939    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:10.649578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:10.649541    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:53:12.882229 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:12.882194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:12.998395 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:12.998362    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:13.199431 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:13.199395    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:13.682778 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:13.682667    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:14.123505 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:14.123464    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:14.221439 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:14.221392    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:17.414408 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:17.414361    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:19.646017 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:19.645984    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:53:27.870959 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:27.870922    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:27.970454 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:27.970413    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:30.339054 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339022    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339370    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37040a06-662d-4799-9fed-8774764b4bdf" containerName="revoke-keys"
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339381    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="37040a06-662d-4799-9fed-8774764b4bdf" containerName="revoke-keys"
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339397    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e19d4827-80f6-435f-a982-4f769e419ceb" containerName="main"
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339402    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e19d4827-80f6-435f-a982-4f769e419ceb" containerName="main"
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339459    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e19d4827-80f6-435f-a982-4f769e419ceb" containerName="main"
Jul 16 22:53:30.339469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.339467    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="37040a06-662d-4799-9fed-8774764b4bdf" containerName="revoke-keys"
Jul 16 22:53:30.342496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.342480    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.345102 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.345066    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-acf29b\"/\"kube-root-ca.crt\""
Jul 16 22:53:30.345242 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.345069    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-acf29b\"/\"default-dockercfg-7fljr\""
Jul 16 22:53:30.345242 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.345119    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-acf29b\"/\"openshift-service-ca.crt\""
Jul 16 22:53:30.345242 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.345191    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-acf29b\"/\"test-model-acf29b-kserve-self-signed-certs\""
Jul 16 22:53:30.351631 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.351597    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:53:30.375729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.375702    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:30.378835 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.378811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.378962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.378849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrt8h\" (UniqueName: \"kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.378962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.378884    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.378962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.378920    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.378962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.378936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.379157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.379001    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480402 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480361    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480584 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480434    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480584 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480464    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrt8h\" (UniqueName: \"kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480584 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480584 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480584 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480539    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480863 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480840    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480907 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480858    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.480942 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.480910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.482858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.482826    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.482973 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.482962    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.488516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.488497    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrt8h\" (UniqueName: \"kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h\") pod \"test-model-acf29b-kserve-5687cb69c5-k662j\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") " pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.652671 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.652571    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:30.784968 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:30.784940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:53:30.787213 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:53:30.787185    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63f9d42b_cf06_4d95_861c_add1a84c7fcf.slice/crio-28d22205ace1551363b41105d19d53482ad2ec6c10c0036b487778fda27382cd WatchSource:0}: Error finding container 28d22205ace1551363b41105d19d53482ad2ec6c10c0036b487778fda27382cd: Status 404 returned error can't find the container with id 28d22205ace1551363b41105d19d53482ad2ec6c10c0036b487778fda27382cd
Jul 16 22:53:31.156875 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:31.156840    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" event={"ID":"63f9d42b-cf06-4d95-861c-add1a84c7fcf","Type":"ContainerStarted","Data":"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"}
Jul 16 22:53:31.157059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:31.156888    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" event={"ID":"63f9d42b-cf06-4d95-861c-add1a84c7fcf","Type":"ContainerStarted","Data":"28d22205ace1551363b41105d19d53482ad2ec6c10c0036b487778fda27382cd"}
Jul 16 22:53:31.175393 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:31.175343    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" podStartSLOduration=1.175329691 podStartE2EDuration="1.175329691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:53:31.174020773 +0000 UTC m=+2380.868602627" watchObservedRunningTime="2026-07-16 22:53:31.175329691 +0000 UTC m=+2380.869911544"
Jul 16 22:53:33.653442 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:33.653400    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:53:35.113518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:35.113478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:35.209223 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:35.209195    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:40.653594 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:40.653557    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:40.654070 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:40.653617    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:40.666486 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:40.666456    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:41.215469 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:41.215444    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:53:47.747409 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:47.747375    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:53:52.081577 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:52.081545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:54.443587 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:54.443550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:53:58.233164 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:53:58.233129    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:54:11.368163 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:11.368130    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:54:11.385076 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:11.385045    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:12.385174 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.385061    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:12.457838 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.457805    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"]
Jul 16 22:54:12.461358 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.461337    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:12.463286 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.463269    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:54:12.463394 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.463267    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:54:12.468240 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.468215    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"]
Jul 16 22:54:12.555709 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.555669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj\" (UID: \"ed56dcf1-401e-4015-b67c-f3852e448a93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:12.656446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.656344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj\" (UID: \"ed56dcf1-401e-4015-b67c-f3852e448a93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:12.656974 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.656954    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj\" (UID: \"ed56dcf1-401e-4015-b67c-f3852e448a93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:12.773152 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.773113    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:12.907276 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.907184    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"]
Jul 16 22:54:12.909928 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:54:12.909902    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded56dcf1_401e_4015_b67c_f3852e448a93.slice/crio-2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f WatchSource:0}: Error finding container 2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f: Status 404 returned error can't find the container with id 2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f
Jul 16 22:54:12.990049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:12.990018    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:13.090639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:13.090608    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:13.355989 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:13.355953    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj" event={"ID":"ed56dcf1-401e-4015-b67c-f3852e448a93","Type":"ContainerStarted","Data":"4513e2bb67c690e51e37e468b1882ce6a0aa4949a172e5d5d5e2489611872c9c"}
Jul 16 22:54:13.356172 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:13.356015    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj" event={"ID":"ed56dcf1-401e-4015-b67c-f3852e448a93","Type":"ContainerStarted","Data":"2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f"}
Jul 16 22:54:13.372630 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:13.372579    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj" podStartSLOduration=1.3725600660000001 podStartE2EDuration="1.372560066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:13.370828843 +0000 UTC m=+2423.065410712" watchObservedRunningTime="2026-07-16 22:54:13.372560066 +0000 UTC m=+2423.067141920"
Jul 16 22:54:14.362304 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:14.362268    2556 generic.go:356] "Generic (PLEG): container finished" podID="ed56dcf1-401e-4015-b67c-f3852e448a93" containerID="4513e2bb67c690e51e37e468b1882ce6a0aa4949a172e5d5d5e2489611872c9c" exitCode=0
Jul 16 22:54:14.362673 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:14.362316    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj" event={"ID":"ed56dcf1-401e-4015-b67c-f3852e448a93","Type":"ContainerDied","Data":"4513e2bb67c690e51e37e468b1882ce6a0aa4949a172e5d5d5e2489611872c9c"}
Jul 16 22:54:15.499728 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:15.499704    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:15.580310 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:15.580278    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca\") pod \"ed56dcf1-401e-4015-b67c-f3852e448a93\" (UID: \"ed56dcf1-401e-4015-b67c-f3852e448a93\") "
Jul 16 22:54:15.580689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:15.580656    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed56dcf1-401e-4015-b67c-f3852e448a93" (UID: "ed56dcf1-401e-4015-b67c-f3852e448a93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:15.681678 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:15.681591    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed56dcf1-401e-4015-b67c-f3852e448a93-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:16.141260 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:16.141227    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:54:16.373129 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:16.373081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj" event={"ID":"ed56dcf1-401e-4015-b67c-f3852e448a93","Type":"ContainerDied","Data":"2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f"}
Jul 16 22:54:16.373129 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:16.373123    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj"
Jul 16 22:54:16.373336 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:16.373143    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2077286dc2a612de3eb3c93bc9d45ade1b35986788c1bb14aede520becd05d8f"
Jul 16 22:54:22.967653 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:22.967625    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:26.389318 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.389276    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" podUID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" containerName="main" containerID="cri-o://969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d" gracePeriod=45
Jul 16 22:54:26.648556 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.648488    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:54:26.776320 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776279    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776334    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776349    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776370    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrt8h\" (UniqueName: \"kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776423    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776442    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home\") pod \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\" (UID: \"63f9d42b-cf06-4d95-861c-add1a84c7fcf\") "
Jul 16 22:54:26.776812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776646    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:54:26.776812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776733    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache" (OuterVolumeSpecName: "model-cache") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:54:26.776812 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776783    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:26.776955 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.776823    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home" (OuterVolumeSpecName: "home") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:54:26.778766 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.778742    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h" (OuterVolumeSpecName: "kube-api-access-zrt8h") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "kube-api-access-zrt8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:54:26.778855 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.778747    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm" (OuterVolumeSpecName: "dshm") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:54:26.778855 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.778785    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63f9d42b-cf06-4d95-861c-add1a84c7fcf" (UID: "63f9d42b-cf06-4d95-861c-add1a84c7fcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:54:26.877516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.877481    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63f9d42b-cf06-4d95-861c-add1a84c7fcf-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:26.877516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.877508    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:26.877516 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.877519    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrt8h\" (UniqueName: \"kubernetes.io/projected/63f9d42b-cf06-4d95-861c-add1a84c7fcf-kube-api-access-zrt8h\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:26.877755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.877529    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:26.877755 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:26.877538    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63f9d42b-cf06-4d95-861c-add1a84c7fcf-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:27.429190 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.429152    2556 generic.go:356] "Generic (PLEG): container finished" podID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" containerID="969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d" exitCode=0
Jul 16 22:54:27.429681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.429201    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"
Jul 16 22:54:27.429681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.429244    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" event={"ID":"63f9d42b-cf06-4d95-861c-add1a84c7fcf","Type":"ContainerDied","Data":"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"}
Jul 16 22:54:27.429681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.429288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j" event={"ID":"63f9d42b-cf06-4d95-861c-add1a84c7fcf","Type":"ContainerDied","Data":"28d22205ace1551363b41105d19d53482ad2ec6c10c0036b487778fda27382cd"}
Jul 16 22:54:27.429681 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.429307    2556 scope.go:117] "RemoveContainer" containerID="969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"
Jul 16 22:54:27.438789 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.438766    2556 scope.go:117] "RemoveContainer" containerID="969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"
Jul 16 22:54:27.439107 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:54:27.439067    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d\": container with ID starting with 969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d not found: ID does not exist" containerID="969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"
Jul 16 22:54:27.439184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.439113    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d"} err="failed to get container status \"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d\": rpc error: code = NotFound desc = could not find container \"969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d\": container with ID starting with 969024928ce93cf683c911b5b5f3cbee0ac994607abc1c52bb9ae95abdbec87d not found: ID does not exist"
Jul 16 22:54:27.444442 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.444412    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:54:27.448104 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.448072    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-acf29b/test-model-acf29b-kserve-5687cb69c5-k662j"]
Jul 16 22:54:27.552759 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.552729    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:27.958535 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:27.958504    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:28.152645 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:28.152604    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:54:28.829769 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:28.829734    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" path="/var/lib/kubelet/pods/63f9d42b-cf06-4d95-861c-add1a84c7fcf/volumes"
Jul 16 22:54:33.841450 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:33.841413    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:33.941571 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:33.941545    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:34.225315 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.225277    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:34.419660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.419628    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:34.506141 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506020    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"]
Jul 16 22:54:34.506490 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506469    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" containerName="main"
Jul 16 22:54:34.506578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506492    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" containerName="main"
Jul 16 22:54:34.506578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506527    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed56dcf1-401e-4015-b67c-f3852e448a93" containerName="revoke-keys"
Jul 16 22:54:34.506578 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506536    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed56dcf1-401e-4015-b67c-f3852e448a93" containerName="revoke-keys"
Jul 16 22:54:34.506735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506633    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="63f9d42b-cf06-4d95-861c-add1a84c7fcf" containerName="main"
Jul 16 22:54:34.506735 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.506647    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed56dcf1-401e-4015-b67c-f3852e448a93" containerName="revoke-keys"
Jul 16 22:54:34.511116 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.511069    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:34.513188 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.513166    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:54:34.513296 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.513221    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:54:34.517293 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.517257    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"]
Jul 16 22:54:34.647305 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.647265    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv\" (UID: \"d7cf5d4d-8f52-4262-84d0-ea47251587b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:34.748771 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.748726    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv\" (UID: \"d7cf5d4d-8f52-4262-84d0-ea47251587b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:34.749474 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.749454    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv\" (UID: \"d7cf5d4d-8f52-4262-84d0-ea47251587b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:34.824016 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.823929    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:34.960712 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:34.960662    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"]
Jul 16 22:54:34.963888 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:54:34.963862    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7cf5d4d_8f52_4262_84d0_ea47251587b2.slice/crio-5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9 WatchSource:0}: Error finding container 5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9: Status 404 returned error can't find the container with id 5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9
Jul 16 22:54:35.178049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:35.177999    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:35.272399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:35.272363    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:35.470034 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:35.469994    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv" event={"ID":"d7cf5d4d-8f52-4262-84d0-ea47251587b2","Type":"ContainerStarted","Data":"e336e994fdf165e2244a57be8b1c079be9ea760b544e1e96abbd2f06e8cfd08f"}
Jul 16 22:54:35.470227 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:35.470082    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv" event={"ID":"d7cf5d4d-8f52-4262-84d0-ea47251587b2","Type":"ContainerStarted","Data":"5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9"}
Jul 16 22:54:35.485005 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:35.484945    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv" podStartSLOduration=1.48492663 podStartE2EDuration="1.48492663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:54:35.48374489 +0000 UTC m=+2445.178326763" watchObservedRunningTime="2026-07-16 22:54:35.48492663 +0000 UTC m=+2445.179508558"
Jul 16 22:54:36.476466 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:36.476425    2556 generic.go:356] "Generic (PLEG): container finished" podID="d7cf5d4d-8f52-4262-84d0-ea47251587b2" containerID="e336e994fdf165e2244a57be8b1c079be9ea760b544e1e96abbd2f06e8cfd08f" exitCode=0
Jul 16 22:54:36.476899 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:36.476513    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv" event={"ID":"d7cf5d4d-8f52-4262-84d0-ea47251587b2","Type":"ContainerDied","Data":"e336e994fdf165e2244a57be8b1c079be9ea760b544e1e96abbd2f06e8cfd08f"}
Jul 16 22:54:37.613586 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:37.613562    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:37.676858 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:37.676818    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca\") pod \"d7cf5d4d-8f52-4262-84d0-ea47251587b2\" (UID: \"d7cf5d4d-8f52-4262-84d0-ea47251587b2\") "
Jul 16 22:54:37.677208 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:37.677187    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7cf5d4d-8f52-4262-84d0-ea47251587b2" (UID: "d7cf5d4d-8f52-4262-84d0-ea47251587b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:54:37.778067 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:37.777972    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7cf5d4d-8f52-4262-84d0-ea47251587b2-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:54:38.487006 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:38.486976    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv"
Jul 16 22:54:38.487179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:38.487024    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv" event={"ID":"d7cf5d4d-8f52-4262-84d0-ea47251587b2","Type":"ContainerDied","Data":"5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9"}
Jul 16 22:54:38.487179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:38.487069    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fd3e78d8435cbe63e083d73a5177d9bad79af5ce3c997e5efec41da4cc6e2f9"
Jul 16 22:54:45.173609 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:45.173580    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:48.236983 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:48.236941    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:54:49.613125 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:49.613046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:50.046076 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:50.046043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:55.916800 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:55.916762    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:55.985933 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:55.985898    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:56.223161 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:56.223129    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:56.484984 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:56.484887    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:56.851613 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:56.851520    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:57.202422 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:57.202391    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:54:57.402979 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:54:57.402949    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:00.538145 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:00.538112    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:00.663705 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:00.663670    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:00.751673 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:00.751636    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:00.853399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:00.853314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:01.365347 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:01.365312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:01.467582 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:01.467549    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:05.024867 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:05.024819    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:05.586652 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:05.586618    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:05.865985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:05.865891    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:55:05.938066 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:05.938035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:06.706105 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:06.706068    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:08.611628 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:08.611589    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:08.707576 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:08.707540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:08.806779 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:08.806746    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:09.039137 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:09.039105    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:09.451558 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:09.451529    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:09.553737 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:09.553705    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:13.107135 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:13.107080    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:16.342640 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:16.342602    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:55:21.321365 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.321329    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"]
Jul 16 22:55:21.321770 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.321752    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7cf5d4d-8f52-4262-84d0-ea47251587b2" containerName="revoke-keys"
Jul 16 22:55:21.321814 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.321771    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7cf5d4d-8f52-4262-84d0-ea47251587b2" containerName="revoke-keys"
Jul 16 22:55:21.321849 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.321829    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7cf5d4d-8f52-4262-84d0-ea47251587b2" containerName="revoke-keys"
Jul 16 22:55:21.325155 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.325139    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:21.327033 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.327010    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:55:21.327033 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.327024    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:55:21.330886 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.330855    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"]
Jul 16 22:55:21.370613 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.370578    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb113-745pt\" (UID: \"e661cab8-46cc-4750-aec2-5988bdccb56d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:21.471326 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.471291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb113-745pt\" (UID: \"e661cab8-46cc-4750-aec2-5988bdccb56d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:21.471966 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.471943    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddb113-745pt\" (UID: \"e661cab8-46cc-4750-aec2-5988bdccb56d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:21.636362 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.636274    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:21.768714 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.768683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"]
Jul 16 22:55:21.772032 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:55:21.771992    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode661cab8_46cc_4750_aec2_5988bdccb56d.slice/crio-5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc WatchSource:0}: Error finding container 5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc: Status 404 returned error can't find the container with id 5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc
Jul 16 22:55:21.858964 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.858932    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:21.959535 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:21.959503    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:22.705799 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:22.705701    2556 generic.go:356] "Generic (PLEG): container finished" podID="e661cab8-46cc-4750-aec2-5988bdccb56d" containerID="cc418a25d4405884305bfc407af7f3b7c5a0f8e88eacb198b575daf1a24dc4fb" exitCode=0
Jul 16 22:55:22.705799 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:22.705781    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt" event={"ID":"e661cab8-46cc-4750-aec2-5988bdccb56d","Type":"ContainerDied","Data":"cc418a25d4405884305bfc407af7f3b7c5a0f8e88eacb198b575daf1a24dc4fb"}
Jul 16 22:55:22.706234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:22.705828    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt" event={"ID":"e661cab8-46cc-4750-aec2-5988bdccb56d","Type":"ContainerStarted","Data":"5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc"}
Jul 16 22:55:23.836715 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:23.836684    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:23.892610 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:23.892575    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca\") pod \"e661cab8-46cc-4750-aec2-5988bdccb56d\" (UID: \"e661cab8-46cc-4750-aec2-5988bdccb56d\") "
Jul 16 22:55:23.892947 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:23.892925    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e661cab8-46cc-4750-aec2-5988bdccb56d" (UID: "e661cab8-46cc-4750-aec2-5988bdccb56d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:23.993504 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:23.993410    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e661cab8-46cc-4750-aec2-5988bdccb56d-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:55:24.716363 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:24.716329    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt" event={"ID":"e661cab8-46cc-4750-aec2-5988bdccb56d","Type":"ContainerDied","Data":"5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc"}
Jul 16 22:55:24.716513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:24.716380    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c8dc3057fcfbef7595b5f9e38828989de80d72bb6744ef3637b91efa56a26dc"
Jul 16 22:55:24.716513 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:24.716337    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddb113-745pt"
Jul 16 22:55:30.424818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:30.424785    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:30.521946 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:30.521914    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:31.931490 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:31.931458    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:32.032071 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:32.032038    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:37.306471 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:37.306436    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:41.440157 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:41.440123    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:41.852357 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:41.852271    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:43.835868 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:43.835836    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:55:47.706548 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:47.706500    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:47.772356 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:47.772321    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:47.996215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:47.996129    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:48.240461 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.240410    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:48.366639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.366557    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"]
Jul 16 22:55:48.366929 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.366916    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e661cab8-46cc-4750-aec2-5988bdccb56d" containerName="revoke-keys"
Jul 16 22:55:48.366969 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.366931    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e661cab8-46cc-4750-aec2-5988bdccb56d" containerName="revoke-keys"
Jul 16 22:55:48.367025 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.367016    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e661cab8-46cc-4750-aec2-5988bdccb56d" containerName="revoke-keys"
Jul 16 22:55:48.370339 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.370323    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:48.372255 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.372234    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 22:55:48.372371 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.372286    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 22:55:48.377307 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.377282    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"]
Jul 16 22:55:48.508210 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.508167    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ef186c-zbg42\" (UID: \"c070bfa7-207f-4007-b063-2fabaca903b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:48.609083 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.609040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ef186c-zbg42\" (UID: \"c070bfa7-207f-4007-b063-2fabaca903b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:48.609666 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.609645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ef186c-zbg42\" (UID: \"c070bfa7-207f-4007-b063-2fabaca903b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:48.682464 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.682374    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:48.818377 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.818352    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"]
Jul 16 22:55:48.820901 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:55:48.820871    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc070bfa7_207f_4007_b063_2fabaca903b6.slice/crio-48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11 WatchSource:0}: Error finding container 48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11: Status 404 returned error can't find the container with id 48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11
Jul 16 22:55:48.840079 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.840047    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42" event={"ID":"c070bfa7-207f-4007-b063-2fabaca903b6","Type":"ContainerStarted","Data":"48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11"}
Jul 16 22:55:48.910698 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:48.910661    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:49.011664 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:49.011626    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:55:49.846636 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:49.846548    2556 generic.go:356] "Generic (PLEG): container finished" podID="c070bfa7-207f-4007-b063-2fabaca903b6" containerID="7efaec8cd1886dd703c6fabcc178fda9176585f41e6fc854605ad7abb785aacf" exitCode=0
Jul 16 22:55:49.846636 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:49.846616    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42" event={"ID":"c070bfa7-207f-4007-b063-2fabaca903b6","Type":"ContainerDied","Data":"7efaec8cd1886dd703c6fabcc178fda9176585f41e6fc854605ad7abb785aacf"}
Jul 16 22:55:50.984689 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:50.984656    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:51.129831 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.129738    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca\") pod \"c070bfa7-207f-4007-b063-2fabaca903b6\" (UID: \"c070bfa7-207f-4007-b063-2fabaca903b6\") "
Jul 16 22:55:51.130179 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.130159    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c070bfa7-207f-4007-b063-2fabaca903b6" (UID: "c070bfa7-207f-4007-b063-2fabaca903b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 22:55:51.230785 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.230749    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c070bfa7-207f-4007-b063-2fabaca903b6-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:55:51.858386 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.858350    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42"
Jul 16 22:55:51.858560 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.858393    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ef186c-zbg42" event={"ID":"c070bfa7-207f-4007-b063-2fabaca903b6","Type":"ContainerDied","Data":"48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11"}
Jul 16 22:55:51.858560 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:55:51.858437    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48b4d123388232b271360696a698d05de17c423fe2c3bed152aa08dbe6e5db11"
Jul 16 22:56:02.255379 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:02.255344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:02.474854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:02.474824    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:03.869141 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:03.869079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:08.469360 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:08.469328    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:08.639473 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:08.639438    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:56:08.916322 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:08.916232    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:14.823020 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:14.822983    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:14.927085 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:14.927051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:15.166140 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:15.166037    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:15.384639 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:15.384596    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:15.742892 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:15.742864    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:16.073399 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:16.073278    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:16.376058 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:16.375976    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:18.899129 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:18.899073    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:18.983417 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:18.983389    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:19.076047 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:19.076019    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:19.391378 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:19.391348    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:19.574148 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:19.574074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:19.675008 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:19.674918    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:23.416128 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:23.416083    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:23.753215 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:23.753177    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:56:23.774825 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:23.774787    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:24.129692 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:24.129575    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:24.325434 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:24.325386    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:26.917109 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:26.917025    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:27.011902 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:27.011874    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:27.081474 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:27.081438    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:27.461049 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:27.461021    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:27.694552 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:27.694520    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:27.795727 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:27.795642    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:31.431660 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:31.431627    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:32.547847 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:32.547811    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:56:40.419164 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:40.419113    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:40.515452 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:40.515414    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:45.453496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.453463    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:45.847836 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.847752    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:56:45.848116 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.848104    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c070bfa7-207f-4007-b063-2fabaca903b6" containerName="revoke-keys"
Jul 16 22:56:45.848165 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.848118    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c070bfa7-207f-4007-b063-2fabaca903b6" containerName="revoke-keys"
Jul 16 22:56:45.848203 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.848182    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c070bfa7-207f-4007-b063-2fabaca903b6" containerName="revoke-keys"
Jul 16 22:56:45.851290 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.851272    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:45.853871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.853846    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"openshift-service-ca.crt\""
Jul 16 22:56:45.853871 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.853865    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"default-dockercfg-l8g85\""
Jul 16 22:56:45.854061 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.853879    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"auth-test-model-35a77a-kserve-self-signed-certs\""
Jul 16 22:56:45.854061 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.853866    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"kube-root-ca.crt\""
Jul 16 22:56:45.862444 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.862423    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:56:45.881176 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:45.881145    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:46.005211 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.005396 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.005451 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.005496 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005455    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.005533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005497    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.005533 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.005515    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxgmh\" (UniqueName: \"kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106593    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106637    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106678    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxgmh\" (UniqueName: \"kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.106854 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.106776    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.107118 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.107065    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.107261 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.107173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.107261 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.107242    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.108888 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.108871    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.109174 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.109157    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.114421 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.114393    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxgmh\" (UniqueName: \"kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh\") pod \"auth-test-model-35a77a-kserve-76d84b8487-njztj\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") " pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.160750 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.160707    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:46.302577 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:46.302538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:56:46.305559 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:56:46.305526    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod673d55d5_11be_4aed_9a29_2c6a10fcf4b1.slice/crio-d2cbbe4e716a4f4e598b79716f47ccbba7dfab0345b223f9a7877a37bd9c6ca3 WatchSource:0}: Error finding container d2cbbe4e716a4f4e598b79716f47ccbba7dfab0345b223f9a7877a37bd9c6ca3: Status 404 returned error can't find the container with id d2cbbe4e716a4f4e598b79716f47ccbba7dfab0345b223f9a7877a37bd9c6ca3
Jul 16 22:56:47.144997 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:47.144960    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" event={"ID":"673d55d5-11be-4aed-9a29-2c6a10fcf4b1","Type":"ContainerStarted","Data":"83a6d6aad2118a57e31b161b872f28205a5670c8ff0ff77c91421f93f6700039"}
Jul 16 22:56:47.145424 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:47.145007    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" event={"ID":"673d55d5-11be-4aed-9a29-2c6a10fcf4b1","Type":"ContainerStarted","Data":"d2cbbe4e716a4f4e598b79716f47ccbba7dfab0345b223f9a7877a37bd9c6ca3"}
Jul 16 22:56:47.163002 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:47.162943    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" podStartSLOduration=2.162925731 podStartE2EDuration="2.162925731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:56:47.161771467 +0000 UTC m=+2576.856353364" watchObservedRunningTime="2026-07-16 22:56:47.162925731 +0000 UTC m=+2576.857507584"
Jul 16 22:56:48.942415 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:48.941684    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:56:49.917644 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:49.917610    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:50.030817 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:50.030791    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:56:56.161363 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:56.161324    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:56.161907 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:56.161612    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:56.173998 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:56.173969    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:56:56.199039 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:56:56.199011    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:57:16.771051 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:16.771023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:25.951995 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.951957    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:57:25.955551 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.955534    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:25.957962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.957932    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"kube-root-ca.crt\""
Jul 16 22:57:25.957962 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.957956    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"default-dockercfg-z8k9t\""
Jul 16 22:57:25.958196 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.957956    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"openshift-service-ca.crt\""
Jul 16 22:57:25.958196 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.957933    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"auth-test-model-5159e9-kserve-self-signed-certs\""
Jul 16 22:57:25.964117 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:25.964068    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:57:26.034404 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.034372    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:57:26.066951 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.066908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.067122 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.066957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.067122 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.067004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.067122 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.067024    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt74k\" (UniqueName: \"kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.067234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.067165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.067234 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.067194    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168146 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt74k\" (UniqueName: \"kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168206    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168288    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168361 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168313    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168609 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168560    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168760 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168738    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.168828 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.168761    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.170495 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.170478    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.170805 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.170789    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.185913 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.185889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt74k\" (UniqueName: \"kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k\") pod \"auth-test-model-5159e9-kserve-8b88d55d7-szljc\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") " pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.265844 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.265744    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:26.412446 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:26.412411    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:57:26.415755 ip-10-0-133-196 kubenswrapper[2556]: W0716 22:57:26.415726    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff231ef1_0189_4cf7_9714_762523616951.slice/crio-72b7c3e06edfa8d038c518b6cca97e01b6e99b2ee3c1c8eff84eedd67e19498d WatchSource:0}: Error finding container 72b7c3e06edfa8d038c518b6cca97e01b6e99b2ee3c1c8eff84eedd67e19498d: Status 404 returned error can't find the container with id 72b7c3e06edfa8d038c518b6cca97e01b6e99b2ee3c1c8eff84eedd67e19498d
Jul 16 22:57:27.341134 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:27.341086    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" event={"ID":"ff231ef1-0189-4cf7-9714-762523616951","Type":"ContainerStarted","Data":"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"}
Jul 16 22:57:27.341134 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:27.341139    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" event={"ID":"ff231ef1-0189-4cf7-9714-762523616951","Type":"ContainerStarted","Data":"72b7c3e06edfa8d038c518b6cca97e01b6e99b2ee3c1c8eff84eedd67e19498d"}
Jul 16 22:57:27.358779 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:27.358717    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" podStartSLOduration=2.358697598 podStartE2EDuration="2.358697598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:57:27.358313929 +0000 UTC m=+2617.052895785" watchObservedRunningTime="2026-07-16 22:57:27.358697598 +0000 UTC m=+2617.053279454"
Jul 16 22:57:29.542232 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:29.542195    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:57:36.266527 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:36.266492    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:36.267040 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:36.266570    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:36.279216 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:36.279192    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:36.397602 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:36.397576    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:57:43.244047 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:43.244007    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:57:59.235098 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:57:59.235049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:58:07.570475 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:58:07.570442    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:08.610059 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:58:08.610025    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:58:16.744827 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:58:16.744789    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:58:34.442246 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:58:34.442212    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:58:50.859190 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:58:50.859112    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:59:09.743986 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:09.743950    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:59:24.816233 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:24.816200    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:59:24.842432 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:24.842396    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:25.575018 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:25.574985    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:59:25.614802 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:25.614762    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:27.178035 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:27.178006    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 22:59:28.850252 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:28.850216    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:59:39.836870 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:39.836828    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" podUID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" containerName="main" containerID="cri-o://83a6d6aad2118a57e31b161b872f28205a5670c8ff0ff77c91421f93f6700039" gracePeriod=45
Jul 16 22:59:40.008729 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.008695    2556 generic.go:356] "Generic (PLEG): container finished" podID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" containerID="83a6d6aad2118a57e31b161b872f28205a5670c8ff0ff77c91421f93f6700039" exitCode=0
Jul 16 22:59:40.008859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.008763    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" event={"ID":"673d55d5-11be-4aed-9a29-2c6a10fcf4b1","Type":"ContainerDied","Data":"83a6d6aad2118a57e31b161b872f28205a5670c8ff0ff77c91421f93f6700039"}
Jul 16 22:59:40.087732 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.087671    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:59:40.175256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175216    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxgmh\" (UniqueName: \"kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175256 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175258    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175320    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175518 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175425    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175536    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.175634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175552    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175634 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175594    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir\") pod \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\" (UID: \"673d55d5-11be-4aed-9a29-2c6a10fcf4b1\") "
Jul 16 22:59:40.175777 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175719    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home" (OuterVolumeSpecName: "home") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.175825 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175799    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.175928 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175910    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.175985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175934    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.175985 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.175948    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.177566 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.177542    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm" (OuterVolumeSpecName: "dshm") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.177666 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.177617    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:59:40.177666 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.177637    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh" (OuterVolumeSpecName: "kube-api-access-cxgmh") pod "673d55d5-11be-4aed-9a29-2c6a10fcf4b1" (UID: "673d55d5-11be-4aed-9a29-2c6a10fcf4b1"). InnerVolumeSpecName "kube-api-access-cxgmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:59:40.277184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.277143    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxgmh\" (UniqueName: \"kubernetes.io/projected/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-kube-api-access-cxgmh\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.277184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.277176    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.277184 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.277186    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/673d55d5-11be-4aed-9a29-2c6a10fcf4b1-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.597988 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.597945    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" podUID="ff231ef1-0189-4cf7-9714-762523616951" containerName="main" containerID="cri-o://bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269" gracePeriod=45
Jul 16 22:59:40.837623 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.837603    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:59:40.984342 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984299    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984342 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984336    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984358    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt74k\" (UniqueName: \"kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984414    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984430    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984612 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984474    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir\") pod \"ff231ef1-0189-4cf7-9714-762523616951\" (UID: \"ff231ef1-0189-4cf7-9714-762523616951\") "
Jul 16 22:59:40.984829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984633    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home" (OuterVolumeSpecName: "home") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.984829 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984750    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.984936 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984869    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.984936 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.984887    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:40.985039 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.985009    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.986818 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.986791    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm" (OuterVolumeSpecName: "dshm") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 22:59:40.986938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.986844    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 22:59:40.986938 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:40.986856    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k" (OuterVolumeSpecName: "kube-api-access-jt74k") pod "ff231ef1-0189-4cf7-9714-762523616951" (UID: "ff231ef1-0189-4cf7-9714-762523616951"). InnerVolumeSpecName "kube-api-access-jt74k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 22:59:41.014663 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.014629    2556 generic.go:356] "Generic (PLEG): container finished" podID="ff231ef1-0189-4cf7-9714-762523616951" containerID="bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269" exitCode=0
Jul 16 22:59:41.014859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.014671    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"
Jul 16 22:59:41.014859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.014666    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" event={"ID":"ff231ef1-0189-4cf7-9714-762523616951","Type":"ContainerDied","Data":"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"}
Jul 16 22:59:41.014859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.014731    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc" event={"ID":"ff231ef1-0189-4cf7-9714-762523616951","Type":"ContainerDied","Data":"72b7c3e06edfa8d038c518b6cca97e01b6e99b2ee3c1c8eff84eedd67e19498d"}
Jul 16 22:59:41.014859 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.014753    2556 scope.go:117] "RemoveContainer" containerID="bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"
Jul 16 22:59:41.016245 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.016111    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj" event={"ID":"673d55d5-11be-4aed-9a29-2c6a10fcf4b1","Type":"ContainerDied","Data":"d2cbbe4e716a4f4e598b79716f47ccbba7dfab0345b223f9a7877a37bd9c6ca3"}
Jul 16 22:59:41.016245 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.016142    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"
Jul 16 22:59:41.024630 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.024610    2556 scope.go:117] "RemoveContainer" containerID="bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"
Jul 16 22:59:41.024951 ip-10-0-133-196 kubenswrapper[2556]: E0716 22:59:41.024931    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269\": container with ID starting with bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269 not found: ID does not exist" containerID="bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"
Jul 16 22:59:41.024996 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.024960    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269"} err="failed to get container status \"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269\": rpc error: code = NotFound desc = could not find container \"bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269\": container with ID starting with bc2caa6cb3d68f0f59407000c319975095344934e8af143575cbd3955dd2f269 not found: ID does not exist"
Jul 16 22:59:41.024996 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.024979    2556 scope.go:117] "RemoveContainer" containerID="83a6d6aad2118a57e31b161b872f28205a5670c8ff0ff77c91421f93f6700039"
Jul 16 22:59:41.044791 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.044761    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:59:41.053696 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.053670    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/auth-test-model-35a77a-kserve-76d84b8487-njztj"]
Jul 16 22:59:41.064170 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.064142    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:59:41.068932 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.068909    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/auth-test-model-5159e9-kserve-8b88d55d7-szljc"]
Jul 16 22:59:41.086261 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.086231    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:41.086261 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.086258    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff231ef1-0189-4cf7-9714-762523616951-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:41.086388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.086267    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt74k\" (UniqueName: \"kubernetes.io/projected/ff231ef1-0189-4cf7-9714-762523616951-kube-api-access-jt74k\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:41.086388 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:41.086275    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff231ef1-0189-4cf7-9714-762523616951-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 22:59:42.648100 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:42.648061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 22:59:42.828894 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:42.828860    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" path="/var/lib/kubelet/pods/673d55d5-11be-4aed-9a29-2c6a10fcf4b1/volumes"
Jul 16 22:59:42.829311 ip-10-0-133-196 kubenswrapper[2556]: I0716 22:59:42.829296    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff231ef1-0189-4cf7-9714-762523616951" path="/var/lib/kubelet/pods/ff231ef1-0189-4cf7-9714-762523616951/volumes"
Jul 16 23:00:00.186501 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186466    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"]
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186832    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff231ef1-0189-4cf7-9714-762523616951" containerName="main"
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186848    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff231ef1-0189-4cf7-9714-762523616951" containerName="main"
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186869    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" containerName="main"
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186878    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" containerName="main"
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186980    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="673d55d5-11be-4aed-9a29-2c6a10fcf4b1" containerName="main"
Jul 16 23:00:00.186997 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.186998    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff231ef1-0189-4cf7-9714-762523616951" containerName="main"
Jul 16 23:00:00.190077 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.190061    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"
Jul 16 23:00:00.192156 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.192132    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 23:00:00.199683 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.199662    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"
Jul 16 23:00:00.210927 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.210874    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"]
Jul 16 23:00:00.214996 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.214973    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"]
Jul 16 23:00:00.215518 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.215495    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"
Jul 16 23:00:00.218539 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.218516    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"]
Jul 16 23:00:00.218647 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.218587    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"
Jul 16 23:00:00.240245 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.240219    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"
Jul 16 23:00:00.256248 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.256215    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"
Jul 16 23:00:00.256493 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.256472    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"]
Jul 16 23:00:00.263273 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.263232    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"]
Jul 16 23:00:00.370866 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.370353    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"]
Jul 16 23:00:00.374396 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:00:00.374315    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ebb9e22_4193_4066_9f9d_5671ea7f5f09.slice/crio-7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9 WatchSource:0}: Error finding container 7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9: Status 404 returned error can't find the container with id 7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9
Jul 16 23:00:00.445794 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.445768    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"]
Jul 16 23:00:00.447577 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:00:00.447543    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf358b256_5369_4032_a4a6_0883e93c2b51.slice/crio-b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9 WatchSource:0}: Error finding container b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9: Status 404 returned error can't find the container with id b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9
Jul 16 23:00:00.633177 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:00.633153    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"]
Jul 16 23:00:00.635683 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:00:00.635657    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d40d19b_3349_418e_a01a_fff6f15288cb.slice/crio-ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a WatchSource:0}: Error finding container ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a: Status 404 returned error can't find the container with id ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a
Jul 16 23:00:01.121830 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.121732    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x" event={"ID":"f358b256-5369-4032-a4a6-0883e93c2b51","Type":"ContainerStarted","Data":"c1c07f9da5bcbe85b3e498ed87cec8278ae0517219923ca5a5541840362c7c1c"}
Jul 16 23:00:01.122009 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.121832    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x" event={"ID":"f358b256-5369-4032-a4a6-0883e93c2b51","Type":"ContainerStarted","Data":"b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9"}
Jul 16 23:00:01.123327 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.123302    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq" event={"ID":"5ebb9e22-4193-4066-9f9d-5671ea7f5f09","Type":"ContainerStarted","Data":"0237b41da952ec36898e977cab17a372d87b934b679a3909d92b391e5f8c981c"}
Jul 16 23:00:01.123436 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.123342    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq" event={"ID":"5ebb9e22-4193-4066-9f9d-5671ea7f5f09","Type":"ContainerStarted","Data":"7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9"}
Jul 16 23:00:01.124858 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.124827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c" event={"ID":"6d40d19b-3349-418e-a01a-fff6f15288cb","Type":"ContainerStarted","Data":"307bdb15cc2877d59e99b8589771b42aacbaaee06406ea1b0221944a13acbdc5"}
Jul 16 23:00:01.124941 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.124900    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c" event={"ID":"6d40d19b-3349-418e-a01a-fff6f15288cb","Type":"ContainerStarted","Data":"ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a"}
Jul 16 23:00:01.141392 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.141341    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x" podStartSLOduration=1.141321411 podStartE2EDuration="1.141321411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:01.13857182 +0000 UTC m=+2770.833153674" watchObservedRunningTime="2026-07-16 23:00:01.141321411 +0000 UTC m=+2770.835903299"
Jul 16 23:00:01.155159 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.155107    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq" podStartSLOduration=1.1550723139999999 podStartE2EDuration="1.155072314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:01.152831315 +0000 UTC m=+2770.847413197" watchObservedRunningTime="2026-07-16 23:00:01.155072314 +0000 UTC m=+2770.849654168"
Jul 16 23:00:01.169795 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.169737    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c" podStartSLOduration=1.169720744 podStartE2EDuration="1.169720744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:01.167712794 +0000 UTC m=+2770.862294647" watchObservedRunningTime="2026-07-16 23:00:01.169720744 +0000 UTC m=+2770.864302598"
Jul 16 23:00:01.527219 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:01.527187    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:02.131398 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.131364    2556 generic.go:356] "Generic (PLEG): container finished" podID="f358b256-5369-4032-a4a6-0883e93c2b51" containerID="c1c07f9da5bcbe85b3e498ed87cec8278ae0517219923ca5a5541840362c7c1c" exitCode=0
Jul 16 23:00:02.131590 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.131452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x" event={"ID":"f358b256-5369-4032-a4a6-0883e93c2b51","Type":"ContainerDied","Data":"c1c07f9da5bcbe85b3e498ed87cec8278ae0517219923ca5a5541840362c7c1c"}
Jul 16 23:00:02.132922 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.132902    2556 generic.go:356] "Generic (PLEG): container finished" podID="5ebb9e22-4193-4066-9f9d-5671ea7f5f09" containerID="0237b41da952ec36898e977cab17a372d87b934b679a3909d92b391e5f8c981c" exitCode=0
Jul 16 23:00:02.133045 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.132930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq" event={"ID":"5ebb9e22-4193-4066-9f9d-5671ea7f5f09","Type":"ContainerDied","Data":"0237b41da952ec36898e977cab17a372d87b934b679a3909d92b391e5f8c981c"}
Jul 16 23:00:02.134464 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.134443    2556 generic.go:356] "Generic (PLEG): container finished" podID="6d40d19b-3349-418e-a01a-fff6f15288cb" containerID="307bdb15cc2877d59e99b8589771b42aacbaaee06406ea1b0221944a13acbdc5" exitCode=0
Jul 16 23:00:02.134549 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:02.134526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c" event={"ID":"6d40d19b-3349-418e-a01a-fff6f15288cb","Type":"ContainerDied","Data":"307bdb15cc2877d59e99b8589771b42aacbaaee06406ea1b0221944a13acbdc5"}
Jul 16 23:00:03.292896 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:03.292859    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"
Jul 16 23:00:03.335133 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:03.335108    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"
Jul 16 23:00:03.338253 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:03.338230    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"
Jul 16 23:00:04.146703 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.146668    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"
Jul 16 23:00:04.146703 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.146679    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x" event={"ID":"f358b256-5369-4032-a4a6-0883e93c2b51","Type":"ContainerDied","Data":"b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9"}
Jul 16 23:00:04.146940 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.146725    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5acf15967c1ec86e6fec0146452c218dc3a0b30bde0710af743f396738657c9"
Jul 16 23:00:04.147967 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.147946    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq"
Jul 16 23:00:04.148068 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.147968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737380-rfwbq" event={"ID":"5ebb9e22-4193-4066-9f9d-5671ea7f5f09","Type":"ContainerDied","Data":"7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9"}
Jul 16 23:00:04.148068 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.148012    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7501d12464e5407dc0a8b21b9dcb08d24971a142479bedc7ca193d8e5d9e06a9"
Jul 16 23:00:04.149295 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.149274    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"
Jul 16 23:00:04.149407 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.149301    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c" event={"ID":"6d40d19b-3349-418e-a01a-fff6f15288cb","Type":"ContainerDied","Data":"ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a"}
Jul 16 23:00:04.149407 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:04.149338    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce688bd0e888a0e3e13524af0f0ec86190e446b473ed8af5feaa8d52df5d819a"
Jul 16 23:00:19.049321 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049287    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049628    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f358b256-5369-4032-a4a6-0883e93c2b51" containerName="cleanup"
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049638    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f358b256-5369-4032-a4a6-0883e93c2b51" containerName="cleanup"
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049655    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d40d19b-3349-418e-a01a-fff6f15288cb" containerName="cleanup"
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049661    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d40d19b-3349-418e-a01a-fff6f15288cb" containerName="cleanup"
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049677    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebb9e22-4193-4066-9f9d-5671ea7f5f09" containerName="cleanup"
Jul 16 23:00:19.049725 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049683    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebb9e22-4193-4066-9f9d-5671ea7f5f09" containerName="cleanup"
Jul 16 23:00:19.049912 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049732    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f358b256-5369-4032-a4a6-0883e93c2b51" containerName="cleanup"
Jul 16 23:00:19.049912 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049740    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d40d19b-3349-418e-a01a-fff6f15288cb" containerName="cleanup"
Jul 16 23:00:19.049912 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.049747    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ebb9e22-4193-4066-9f9d-5671ea7f5f09" containerName="cleanup"
Jul 16 23:00:19.052901 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.052880    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.055024 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.055003    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"openshift-service-ca.crt\""
Jul 16 23:00:19.055490 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.055469    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"default-dockercfg-z8k9t\""
Jul 16 23:00:19.055649 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.055486    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"sub-test-model-5159e9-kserve-self-signed-certs\""
Jul 16 23:00:19.055649 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.055505    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"kube-root-ca.crt\""
Jul 16 23:00:19.062965 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.062940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:00:19.079757 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.079730    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:00:19.112959 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.112921    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.113166 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.112962    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.113249 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.113142    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.113365 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.113331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.113496 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.113468    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.113632 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.113515    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld2v4\" (UniqueName: \"kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214390 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214351    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214390 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214651 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214432    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214651 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214557    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld2v4\" (UniqueName: \"kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214651 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214651 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214643    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214863 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214794    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.214863 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.214837    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.215033 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.215007    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.216830 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.216810    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.217153 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.217135    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.223608 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.223587    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld2v4\" (UniqueName: \"kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4\") pod \"sub-test-model-5159e9-kserve-6768dcfddc-qhcks\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") " pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.363909 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.363819    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:19.499808 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:19.499785    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:00:19.501936 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:00:19.501913    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod180b3a0c_dc7e_4755_9cc1_e045f6774d9e.slice/crio-fd69afee116e013ebb75da6ba89a8a8c90948f0cf9c2632dab9bd585467ca5d7 WatchSource:0}: Error finding container fd69afee116e013ebb75da6ba89a8a8c90948f0cf9c2632dab9bd585467ca5d7: Status 404 returned error can't find the container with id fd69afee116e013ebb75da6ba89a8a8c90948f0cf9c2632dab9bd585467ca5d7
Jul 16 23:00:20.235826 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:20.235790    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" event={"ID":"180b3a0c-dc7e-4755-9cc1-e045f6774d9e","Type":"ContainerStarted","Data":"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"}
Jul 16 23:00:20.236245 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:20.235835    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" event={"ID":"180b3a0c-dc7e-4755-9cc1-e045f6774d9e","Type":"ContainerStarted","Data":"fd69afee116e013ebb75da6ba89a8a8c90948f0cf9c2632dab9bd585467ca5d7"}
Jul 16 23:00:20.254628 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:20.254579    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" podStartSLOduration=1.254562663 podStartE2EDuration="1.254562663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:00:20.253500794 +0000 UTC m=+2789.948082685" watchObservedRunningTime="2026-07-16 23:00:20.254562663 +0000 UTC m=+2789.949144514"
Jul 16 23:00:23.350931 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:23.350895    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:00:29.364908 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:29.364867    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:29.365369 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:29.364926    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:29.377687 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:29.377662    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:30.299526 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:30.299498    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:00:43.117772 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:00:43.117734    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:01:00.723771 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:00.723736    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:01.651794 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:01.651758    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:01:01.671965 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:01.671937    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:03.418520 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:03.418486    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:03.587016 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:03.586981    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:05.849477 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:05.849432    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:01:16.671302 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:16.671259    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" podUID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" containerName="main" containerID="cri-o://c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3" gracePeriod=45
Jul 16 23:01:16.919394 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:16.919368    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:01:17.053512 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053475    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053700 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053526    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld2v4\" (UniqueName: \"kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053700 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053552    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053700 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053649    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053700 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053675    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053700 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053696    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm\") pod \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\" (UID: \"180b3a0c-dc7e-4755-9cc1-e045f6774d9e\") "
Jul 16 23:01:17.053977 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053758    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home" (OuterVolumeSpecName: "home") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:17.053977 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053957    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:17.054117 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.053975    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:17.054117 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.054048    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.054117 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.054065    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.054295 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.054120    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.055827 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.055798    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm" (OuterVolumeSpecName: "dshm") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:01:17.055827 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.055818    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4" (OuterVolumeSpecName: "kube-api-access-ld2v4") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "kube-api-access-ld2v4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:01:17.055986 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.055908    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "180b3a0c-dc7e-4755-9cc1-e045f6774d9e" (UID: "180b3a0c-dc7e-4755-9cc1-e045f6774d9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:01:17.154674 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.154635    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.154674 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.154668    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.154674 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.154683    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld2v4\" (UniqueName: \"kubernetes.io/projected/180b3a0c-dc7e-4755-9cc1-e045f6774d9e-kube-api-access-ld2v4\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:01:17.530306 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.530268    2556 generic.go:356] "Generic (PLEG): container finished" podID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" containerID="c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3" exitCode=0
Jul 16 23:01:17.530306 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.530309    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"
Jul 16 23:01:17.530533 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.530353    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" event={"ID":"180b3a0c-dc7e-4755-9cc1-e045f6774d9e","Type":"ContainerDied","Data":"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"}
Jul 16 23:01:17.530533 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.530390    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks" event={"ID":"180b3a0c-dc7e-4755-9cc1-e045f6774d9e","Type":"ContainerDied","Data":"fd69afee116e013ebb75da6ba89a8a8c90948f0cf9c2632dab9bd585467ca5d7"}
Jul 16 23:01:17.530533 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.530406    2556 scope.go:117] "RemoveContainer" containerID="c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"
Jul 16 23:01:17.540193 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.540176    2556 scope.go:117] "RemoveContainer" containerID="c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"
Jul 16 23:01:17.540501 ip-10-0-133-196 kubenswrapper[2556]: E0716 23:01:17.540480    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3\": container with ID starting with c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3 not found: ID does not exist" containerID="c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"
Jul 16 23:01:17.540545 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.540511    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3"} err="failed to get container status \"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3\": rpc error: code = NotFound desc = could not find container \"c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3\": container with ID starting with c0ba5286a47f2fd4e62d89d9ba7e92105e15d0d0f53c413d764d54727c3ac8b3 not found: ID does not exist"
Jul 16 23:01:17.554101 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.554064    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:01:17.557146 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.557121    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/sub-test-model-5159e9-kserve-6768dcfddc-qhcks"]
Jul 16 23:01:17.758969 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.758917    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:01:17.759513 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.759338    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" containerName="main"
Jul 16 23:01:17.759513 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.759351    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" containerName="main"
Jul 16 23:01:17.759513 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.759415    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" containerName="main"
Jul 16 23:01:17.763922 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.763901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.765964 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.765936    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"openshift-service-ca.crt\""
Jul 16 23:01:17.766102 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.765988    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"kube-root-ca.crt\""
Jul 16 23:01:17.766102 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.766004    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"rate-test-model-35a77a-kserve-self-signed-certs\""
Jul 16 23:01:17.766102 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.766079    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-35a77a\"/\"default-dockercfg-l8g85\""
Jul 16 23:01:17.773330 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.773312    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:01:17.840349 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.840272    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:17.861747 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.861841 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861755    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.861841 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.861841 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.861943 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861865    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.861943 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.861900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxtzh\" (UniqueName: \"kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963372 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963333    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxtzh\" (UniqueName: \"kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963372 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963378    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963609 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963609 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963736 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963622    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963736 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963717    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963842 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963762    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963842 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963824    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.963947 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.963898    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.965667 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.965646    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.966008 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.965991    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:17.970958 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:17.970935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxtzh\" (UniqueName: \"kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh\") pod \"rate-test-model-35a77a-kserve-5d99694df9-vrv6g\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") " pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:18.074848 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.074815    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:18.213622 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.213599    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:01:18.216562 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:01:18.216535    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d093b3d_a130_4ab7_8a31_33e5d68a02f1.slice/crio-80c2b7676f8764f3c6722d193d72898eb313897a67cd3cee853516344ba2685f WatchSource:0}: Error finding container 80c2b7676f8764f3c6722d193d72898eb313897a67cd3cee853516344ba2685f: Status 404 returned error can't find the container with id 80c2b7676f8764f3c6722d193d72898eb313897a67cd3cee853516344ba2685f
Jul 16 23:01:18.538353 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.538309    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" event={"ID":"3d093b3d-a130-4ab7-8a31-33e5d68a02f1","Type":"ContainerStarted","Data":"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"}
Jul 16 23:01:18.538353 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.538356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" event={"ID":"3d093b3d-a130-4ab7-8a31-33e5d68a02f1","Type":"ContainerStarted","Data":"80c2b7676f8764f3c6722d193d72898eb313897a67cd3cee853516344ba2685f"}
Jul 16 23:01:18.562903 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.562853    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" podStartSLOduration=1.562835599 podStartE2EDuration="1.562835599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:18.560529971 +0000 UTC m=+2848.255111825" watchObservedRunningTime="2026-07-16 23:01:18.562835599 +0000 UTC m=+2848.257417453"
Jul 16 23:01:18.829123 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:18.829030    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="180b3a0c-dc7e-4755-9cc1-e045f6774d9e" path="/var/lib/kubelet/pods/180b3a0c-dc7e-4755-9cc1-e045f6774d9e/volumes"
Jul 16 23:01:22.643940 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:22.643895    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:01:28.075610 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:28.075573    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:28.076025 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:28.075864    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:28.088342 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:28.088316    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:28.602479 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:28.602452    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:01:41.217713 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:41.217644    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:41.319017 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:41.318983    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:42.661572 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.661499    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:01:42.664934 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.664912    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.669219 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.669193    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"kube-root-ca.crt\""
Jul 16 23:01:42.669219 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.669219    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"default-dockercfg-z8k9t\""
Jul 16 23:01:42.669423 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.669241    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"openshift-service-ca.crt\""
Jul 16 23:01:42.669423 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.669266    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5159e9\"/\"rate-test-model-5159e9-kserve-self-signed-certs\""
Jul 16 23:01:42.679046 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.679022    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:01:42.688989 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.688953    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.689181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.688994    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.689181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.689028    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.689181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.689060    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.689181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.689076    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.689422 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.689200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp7lw\" (UniqueName: \"kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.690289 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.690263    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:42.790314 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790505 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790333    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790505 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790461    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790580 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790523    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790580 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790681 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp7lw\" (UniqueName: \"kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790836 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790803    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790926 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.790926 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.790886    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.792580 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.792546    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.792907 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.792891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.799325 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.799301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp7lw\" (UniqueName: \"kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw\") pod \"rate-test-model-5159e9-kserve-647468656c-vk2vt\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") " pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:42.974552 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:42.974519    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:43.110956 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:43.110931    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:01:43.113166 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:01:43.113137    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf613330_73eb_432c_9ce1_7e3141d3e0a7.slice/crio-27654351c6820dd683f9cd7cb4496d4405b4c1aed16058f5bad37a82a773b005 WatchSource:0}: Error finding container 27654351c6820dd683f9cd7cb4496d4405b4c1aed16058f5bad37a82a773b005: Status 404 returned error can't find the container with id 27654351c6820dd683f9cd7cb4496d4405b4c1aed16058f5bad37a82a773b005
Jul 16 23:01:43.668995 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:43.668961    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" event={"ID":"bf613330-73eb-432c-9ce1-7e3141d3e0a7","Type":"ContainerStarted","Data":"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"}
Jul 16 23:01:43.669406 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:43.669006    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" event={"ID":"bf613330-73eb-432c-9ce1-7e3141d3e0a7","Type":"ContainerStarted","Data":"27654351c6820dd683f9cd7cb4496d4405b4c1aed16058f5bad37a82a773b005"}
Jul 16 23:01:43.684860 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:43.684804    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" podStartSLOduration=1.6847880069999999 podStartE2EDuration="1.684788007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:01:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:01:43.684528365 +0000 UTC m=+2873.379110243" watchObservedRunningTime="2026-07-16 23:01:43.684788007 +0000 UTC m=+2873.379369896"
Jul 16 23:01:46.043709 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:46.043667    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:01:50.695716 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:50.695682    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:50.793310 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:50.793270    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:01:52.975010 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:52.974977    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:52.975010 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:52.975020    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:52.987818 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:52.987787    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:01:53.731346 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:01:53.731318    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:02:06.762009 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:06.761973    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:02:20.238366 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:20.238335    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:02:28.422919 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:28.422889    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:02:28.444130 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:28.444071    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:29.264549 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:29.264511    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:02:29.284989 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:29.284954    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:30.302517 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.302479    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7447-139797815063728"]
Jul 16 23:02:30.305786 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.305769    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:30.307801 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.307775    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-56nfd\""
Jul 16 23:02:30.309404 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.309374    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139797815063728"]
Jul 16 23:02:30.421498 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.421464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjvbt\" (UniqueName: \"kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt\") pod \"test-curl-7447-139797815063728\" (UID: \"5ec09ac6-3da0-4e6b-9a15-28773f9fad71\") " pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:30.522110 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.522066    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjvbt\" (UniqueName: \"kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt\") pod \"test-curl-7447-139797815063728\" (UID: \"5ec09ac6-3da0-4e6b-9a15-28773f9fad71\") " pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:30.529984 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.529968    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjvbt\" (UniqueName: \"kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt\") pod \"test-curl-7447-139797815063728\" (UID: \"5ec09ac6-3da0-4e6b-9a15-28773f9fad71\") " pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:30.616816 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.616743    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:30.746015 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.745985    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139797815063728"]
Jul 16 23:02:30.749395 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:02:30.749372    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ec09ac6_3da0_4e6b_9a15_28773f9fad71.slice/crio-1787a07a3c865ec38c5b08cbe914033508c074c2ea8544741d8570c3c8a93fd0 WatchSource:0}: Error finding container 1787a07a3c865ec38c5b08cbe914033508c074c2ea8544741d8570c3c8a93fd0: Status 404 returned error can't find the container with id 1787a07a3c865ec38c5b08cbe914033508c074c2ea8544741d8570c3c8a93fd0
Jul 16 23:02:30.750625 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.750604    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 23:02:30.922755 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:30.922676    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139797815063728" event={"ID":"5ec09ac6-3da0-4e6b-9a15-28773f9fad71","Type":"ContainerStarted","Data":"1787a07a3c865ec38c5b08cbe914033508c074c2ea8544741d8570c3c8a93fd0"}
Jul 16 23:02:31.929342 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:31.929255    2556 generic.go:356] "Generic (PLEG): container finished" podID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" containerID="a607400d6cc2d099078a713af6a29e87fdfe1581fc31f1777e6a4ec336f66750" exitCode=0
Jul 16 23:02:31.929702 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:31.929340    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139797815063728" event={"ID":"5ec09ac6-3da0-4e6b-9a15-28773f9fad71","Type":"ContainerDied","Data":"a607400d6cc2d099078a713af6a29e87fdfe1581fc31f1777e6a4ec336f66750"}
Jul 16 23:02:33.067949 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.067927    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:33.089054 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.088988    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7447-139797815063728_5ec09ac6-3da0-4e6b-9a15-28773f9fad71/test-curl-7447-139797815063728/0.log"
Jul 16 23:02:33.112021 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.111997    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7447-139797815063728"]
Jul 16 23:02:33.116965 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.116938    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7447-139797815063728"]
Jul 16 23:02:33.141652 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.141631    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjvbt\" (UniqueName: \"kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt\") pod \"5ec09ac6-3da0-4e6b-9a15-28773f9fad71\" (UID: \"5ec09ac6-3da0-4e6b-9a15-28773f9fad71\") "
Jul 16 23:02:33.143755 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.143722    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt" (OuterVolumeSpecName: "kube-api-access-fjvbt") pod "5ec09ac6-3da0-4e6b-9a15-28773f9fad71" (UID: "5ec09ac6-3da0-4e6b-9a15-28773f9fad71"). InnerVolumeSpecName "kube-api-access-fjvbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:02:33.242670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.242647    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjvbt\" (UniqueName: \"kubernetes.io/projected/5ec09ac6-3da0-4e6b-9a15-28773f9fad71-kube-api-access-fjvbt\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:33.941779 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.941753    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:33.941961 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:33.941795    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1787a07a3c865ec38c5b08cbe914033508c074c2ea8544741d8570c3c8a93fd0"
Jul 16 23:02:33.951003 ip-10-0-133-196 kubenswrapper[2556]: E0716 23:02:33.950978    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7447-139797815063728\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" podUID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" pod="opendatahub/test-curl-7447-139797815063728"
Jul 16 23:02:34.829298 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:34.829272    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" path="/var/lib/kubelet/pods/5ec09ac6-3da0-4e6b-9a15-28773f9fad71/volumes"
Jul 16 23:02:40.840144 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:40.840112    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:02:43.443876 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.443797    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" podUID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" containerName="main" containerID="cri-o://706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8" gracePeriod=45
Jul 16 23:02:43.700608 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.700547    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:02:43.721176 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721152    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721317 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721193    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721317 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721276    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721452 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721329    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721452 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721371    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721452 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721422    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxtzh\" (UniqueName: \"kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh\") pod \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\" (UID: \"3d093b3d-a130-4ab7-8a31-33e5d68a02f1\") "
Jul 16 23:02:43.721598 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721533    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:43.721936 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721707    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.721936 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721910    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home" (OuterVolumeSpecName: "home") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:43.721936 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.721919    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:43.723740 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.723709    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:02:43.724063 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.724041    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh" (OuterVolumeSpecName: "kube-api-access-rxtzh") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "kube-api-access-rxtzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:02:43.724153 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.724136    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm" (OuterVolumeSpecName: "dshm") pod "3d093b3d-a130-4ab7-8a31-33e5d68a02f1" (UID: "3d093b3d-a130-4ab7-8a31-33e5d68a02f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:43.822939 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.822912    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxtzh\" (UniqueName: \"kubernetes.io/projected/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-kube-api-access-rxtzh\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.822939 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.822936    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.823080 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.822945    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.823080 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.822952    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.823080 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.822978    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d093b3d-a130-4ab7-8a31-33e5d68a02f1-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:43.994951 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.994869    2556 generic.go:356] "Generic (PLEG): container finished" podID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" containerID="706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8" exitCode=0
Jul 16 23:02:43.994951 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.994907    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"
Jul 16 23:02:43.995162 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.994908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" event={"ID":"3d093b3d-a130-4ab7-8a31-33e5d68a02f1","Type":"ContainerDied","Data":"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"}
Jul 16 23:02:43.995162 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.995002    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g" event={"ID":"3d093b3d-a130-4ab7-8a31-33e5d68a02f1","Type":"ContainerDied","Data":"80c2b7676f8764f3c6722d193d72898eb313897a67cd3cee853516344ba2685f"}
Jul 16 23:02:43.995162 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:43.995031    2556 scope.go:117] "RemoveContainer" containerID="706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"
Jul 16 23:02:44.004421 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.004404    2556 scope.go:117] "RemoveContainer" containerID="706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"
Jul 16 23:02:44.004691 ip-10-0-133-196 kubenswrapper[2556]: E0716 23:02:44.004669    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8\": container with ID starting with 706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8 not found: ID does not exist" containerID="706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"
Jul 16 23:02:44.004742 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.004700    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8"} err="failed to get container status \"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8\": rpc error: code = NotFound desc = could not find container \"706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8\": container with ID starting with 706d60ad2cc7ad0a255e6b7960c3da46ca53396e34548ec79ae62e12634433c8 not found: ID does not exist"
Jul 16 23:02:44.015222 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.015196    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:02:44.017695 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.017674    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-35a77a/rate-test-model-35a77a-kserve-5d99694df9-vrv6g"]
Jul 16 23:02:44.284916 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.284839    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" podUID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" containerName="main" containerID="cri-o://9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3" gracePeriod=45
Jul 16 23:02:44.534263 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.534241    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:02:44.629821 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629760    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.629821 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629787    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.629821 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629814    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.630055 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629834    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.630055 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629855    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp7lw\" (UniqueName: \"kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.630055 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.629883    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs\") pod \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\" (UID: \"bf613330-73eb-432c-9ce1-7e3141d3e0a7\") "
Jul 16 23:02:44.630225 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.630119    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:44.630225 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.630133    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:44.630225 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.630078    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home" (OuterVolumeSpecName: "home") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:44.631775 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.631751    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm" (OuterVolumeSpecName: "dshm") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 23:02:44.631964 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.631943    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw" (OuterVolumeSpecName: "kube-api-access-jp7lw") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "kube-api-access-jp7lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:02:44.632032 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.631978    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf613330-73eb-432c-9ce1-7e3141d3e0a7" (UID: "bf613330-73eb-432c-9ce1-7e3141d3e0a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 23:02:44.731253 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731232    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-dshm\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.731253 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731251    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-model-cache\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.731393 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731261    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-home\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.731393 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731268    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tmp-dir\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.731393 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731276    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp7lw\" (UniqueName: \"kubernetes.io/projected/bf613330-73eb-432c-9ce1-7e3141d3e0a7-kube-api-access-jp7lw\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.731393 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.731284    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf613330-73eb-432c-9ce1-7e3141d3e0a7-tls-certs\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:44.829265 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.829233    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" path="/var/lib/kubelet/pods/3d093b3d-a130-4ab7-8a31-33e5d68a02f1/volumes"
Jul 16 23:02:44.943312 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943289    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7447-139797811996544"]
Jul 16 23:02:44.943615 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943602    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" containerName="test-curl-7447-139797815063728"
Jul 16 23:02:44.943670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943616    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" containerName="test-curl-7447-139797815063728"
Jul 16 23:02:44.943670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943634    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" containerName="main"
Jul 16 23:02:44.943670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943639    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" containerName="main"
Jul 16 23:02:44.943670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943647    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" containerName="main"
Jul 16 23:02:44.943670 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943652    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" containerName="main"
Jul 16 23:02:44.943839 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943700    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec09ac6-3da0-4e6b-9a15-28773f9fad71" containerName="test-curl-7447-139797815063728"
Jul 16 23:02:44.943839 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943711    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" containerName="main"
Jul 16 23:02:44.943839 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.943717    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d093b3d-a130-4ab7-8a31-33e5d68a02f1" containerName="main"
Jul 16 23:02:44.947058 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.947043    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:44.949323 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.949306    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-56nfd\""
Jul 16 23:02:44.952875 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:44.952854    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139797811996544"]
Jul 16 23:02:45.002855 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.002830    2556 generic.go:356] "Generic (PLEG): container finished" podID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" containerID="9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3" exitCode=0
Jul 16 23:02:45.002978 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.002875    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"
Jul 16 23:02:45.002978 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.002915    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" event={"ID":"bf613330-73eb-432c-9ce1-7e3141d3e0a7","Type":"ContainerDied","Data":"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"}
Jul 16 23:02:45.002978 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.002956    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt" event={"ID":"bf613330-73eb-432c-9ce1-7e3141d3e0a7","Type":"ContainerDied","Data":"27654351c6820dd683f9cd7cb4496d4405b4c1aed16058f5bad37a82a773b005"}
Jul 16 23:02:45.002978 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.002971    2556 scope.go:117] "RemoveContainer" containerID="9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"
Jul 16 23:02:45.011222 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.011204    2556 scope.go:117] "RemoveContainer" containerID="9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"
Jul 16 23:02:45.011472 ip-10-0-133-196 kubenswrapper[2556]: E0716 23:02:45.011452    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3\": container with ID starting with 9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3 not found: ID does not exist" containerID="9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"
Jul 16 23:02:45.011521 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.011481    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3"} err="failed to get container status \"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3\": rpc error: code = NotFound desc = could not find container \"9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3\": container with ID starting with 9f254057511c37d072d23409bc6aa23bdaf9b9975919a8f0617dc186313306a3 not found: ID does not exist"
Jul 16 23:02:45.018188 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.018168    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:02:45.021622 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.021604    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5159e9/rate-test-model-5159e9-kserve-647468656c-vk2vt"]
Jul 16 23:02:45.033902 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.033883    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgrmv\" (UniqueName: \"kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv\") pod \"test-curl-7447-139797811996544\" (UID: \"826c314e-6d87-44e9-a51a-ab29ef8948c3\") " pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:45.134523 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.134495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgrmv\" (UniqueName: \"kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv\") pod \"test-curl-7447-139797811996544\" (UID: \"826c314e-6d87-44e9-a51a-ab29ef8948c3\") " pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:45.142970 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.142945    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgrmv\" (UniqueName: \"kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv\") pod \"test-curl-7447-139797811996544\" (UID: \"826c314e-6d87-44e9-a51a-ab29ef8948c3\") " pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:45.258108 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.258004    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:45.591959 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:45.591878    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7447-139797811996544"]
Jul 16 23:02:45.594861 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:02:45.594833    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod826c314e_6d87_44e9_a51a_ab29ef8948c3.slice/crio-3556784daf852ed2cf297176915bb87ffa2aba3265e21921befda2695cdd1566 WatchSource:0}: Error finding container 3556784daf852ed2cf297176915bb87ffa2aba3265e21921befda2695cdd1566: Status 404 returned error can't find the container with id 3556784daf852ed2cf297176915bb87ffa2aba3265e21921befda2695cdd1566
Jul 16 23:02:46.011385 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:46.011354    2556 generic.go:356] "Generic (PLEG): container finished" podID="826c314e-6d87-44e9-a51a-ab29ef8948c3" containerID="bd79843eb3899cc1fb6155819aabccc5baedb0df31ee5e62c513e1ef934d17fd" exitCode=0
Jul 16 23:02:46.011501 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:46.011439    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139797811996544" event={"ID":"826c314e-6d87-44e9-a51a-ab29ef8948c3","Type":"ContainerDied","Data":"bd79843eb3899cc1fb6155819aabccc5baedb0df31ee5e62c513e1ef934d17fd"}
Jul 16 23:02:46.011501 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:46.011478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7447-139797811996544" event={"ID":"826c314e-6d87-44e9-a51a-ab29ef8948c3","Type":"ContainerStarted","Data":"3556784daf852ed2cf297176915bb87ffa2aba3265e21921befda2695cdd1566"}
Jul 16 23:02:46.829199 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:46.829167    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf613330-73eb-432c-9ce1-7e3141d3e0a7" path="/var/lib/kubelet/pods/bf613330-73eb-432c-9ce1-7e3141d3e0a7/volumes"
Jul 16 23:02:47.156129 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.156085    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:47.213498 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.213472    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7447-139797811996544_826c314e-6d87-44e9-a51a-ab29ef8948c3/test-curl-7447-139797811996544/0.log"
Jul 16 23:02:47.249228 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.249205    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgrmv\" (UniqueName: \"kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv\") pod \"826c314e-6d87-44e9-a51a-ab29ef8948c3\" (UID: \"826c314e-6d87-44e9-a51a-ab29ef8948c3\") "
Jul 16 23:02:47.251181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.251162    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv" (OuterVolumeSpecName: "kube-api-access-dgrmv") pod "826c314e-6d87-44e9-a51a-ab29ef8948c3" (UID: "826c314e-6d87-44e9-a51a-ab29ef8948c3"). InnerVolumeSpecName "kube-api-access-dgrmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 23:02:47.264818 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.264796    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7447-139797811996544"]
Jul 16 23:02:47.270066 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.270047    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7447-139797811996544"]
Jul 16 23:02:47.350048 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.350027    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgrmv\" (UniqueName: \"kubernetes.io/projected/826c314e-6d87-44e9-a51a-ab29ef8948c3-kube-api-access-dgrmv\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:47.489754 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.489726    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"]
Jul 16 23:02:47.490112 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.490076    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="826c314e-6d87-44e9-a51a-ab29ef8948c3" containerName="test-curl-7447-139797811996544"
Jul 16 23:02:47.490112 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.490103    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="826c314e-6d87-44e9-a51a-ab29ef8948c3" containerName="test-curl-7447-139797811996544"
Jul 16 23:02:47.490196 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.490169    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="826c314e-6d87-44e9-a51a-ab29ef8948c3" containerName="test-curl-7447-139797811996544"
Jul 16 23:02:47.493116 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.493086    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:47.495061 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.495042    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:02:47.495199 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.495085    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 23:02:47.501793 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.501765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"]
Jul 16 23:02:47.551956 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.551932    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr\" (UID: \"f3b0f0d3-0307-4d61-a178-2d5612eb36d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:47.652378 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.652343    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr\" (UID: \"f3b0f0d3-0307-4d61-a178-2d5612eb36d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:47.653149 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.653128    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr\" (UID: \"f3b0f0d3-0307-4d61-a178-2d5612eb36d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:47.804937 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:47.804848    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:48.011901 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.011872    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:48.029431 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.029404    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:48.029554 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.029455    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3556784daf852ed2cf297176915bb87ffa2aba3265e21921befda2695cdd1566"
Jul 16 23:02:48.038831 ip-10-0-133-196 kubenswrapper[2556]: E0716 23:02:48.038807    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7447-139797811996544\" is forbidden: User \"system:node:ip-10-0-133-196.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-196.ec2.internal' and this object" podUID="826c314e-6d87-44e9-a51a-ab29ef8948c3" pod="opendatahub/test-curl-7447-139797811996544"
Jul 16 23:02:48.113050 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.112972    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:48.144325 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.144248    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"]
Jul 16 23:02:48.145125 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:02:48.145075    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b0f0d3_0307_4d61_a178_2d5612eb36d1.slice/crio-9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026 WatchSource:0}: Error finding container 9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026: Status 404 returned error can't find the container with id 9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026
Jul 16 23:02:48.829241 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:48.829205    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="826c314e-6d87-44e9-a51a-ab29ef8948c3" path="/var/lib/kubelet/pods/826c314e-6d87-44e9-a51a-ab29ef8948c3/volumes"
Jul 16 23:02:49.035769 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:49.035739    2556 generic.go:356] "Generic (PLEG): container finished" podID="f3b0f0d3-0307-4d61-a178-2d5612eb36d1" containerID="39606593def81495cac582af884d552f5cd49066b1b5bfbba1e270078bf95ce7" exitCode=0
Jul 16 23:02:49.036124 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:49.035819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr" event={"ID":"f3b0f0d3-0307-4d61-a178-2d5612eb36d1","Type":"ContainerDied","Data":"39606593def81495cac582af884d552f5cd49066b1b5bfbba1e270078bf95ce7"}
Jul 16 23:02:49.036124 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:49.035868    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr" event={"ID":"f3b0f0d3-0307-4d61-a178-2d5612eb36d1","Type":"ContainerStarted","Data":"9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026"}
Jul 16 23:02:50.167168 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:50.167147    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:50.276187 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:50.276162    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca\") pod \"f3b0f0d3-0307-4d61-a178-2d5612eb36d1\" (UID: \"f3b0f0d3-0307-4d61-a178-2d5612eb36d1\") "
Jul 16 23:02:50.276518 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:50.276500    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f3b0f0d3-0307-4d61-a178-2d5612eb36d1" (UID: "f3b0f0d3-0307-4d61-a178-2d5612eb36d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:02:50.377070 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:50.376998    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b0f0d3-0307-4d61-a178-2d5612eb36d1-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:02:51.047557 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:51.047521    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr" event={"ID":"f3b0f0d3-0307-4d61-a178-2d5612eb36d1","Type":"ContainerDied","Data":"9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026"}
Jul 16 23:02:51.047557 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:51.047548    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr"
Jul 16 23:02:51.047756 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:51.047577    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b54c1539a7ff496d793022f2aaf828507c66a8d98e320cec8dfedb81567a026"
Jul 16 23:02:57.980601 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:57.980560    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"]
Jul 16 23:02:57.983672 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:57.983650    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-35a77a-29737380-hxh8x"]
Jul 16 23:02:57.986497 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:57.986474    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:02:58.829272 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:02:58.829237    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f358b256-5369-4032-a4a6-0883e93c2b51" path="/var/lib/kubelet/pods/f358b256-5369-4032-a4a6-0883e93c2b51/volumes"
Jul 16 23:03:01.042215 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:01.042181    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:03:02.597268 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:02.597237    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:03.000269 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:03.000235    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:08.850288 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:08.850260    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:09.480577 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.480544    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"]
Jul 16 23:03:09.480888 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.480875    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b0f0d3-0307-4d61-a178-2d5612eb36d1" containerName="revoke-keys"
Jul 16 23:03:09.480933 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.480889    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b0f0d3-0307-4d61-a178-2d5612eb36d1" containerName="revoke-keys"
Jul 16 23:03:09.480968 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.480944    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b0f0d3-0307-4d61-a178-2d5612eb36d1" containerName="revoke-keys"
Jul 16 23:03:09.486006 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.485990    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:09.487877 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.487857    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 23:03:09.487971 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.487924    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ppfww\""
Jul 16 23:03:09.491323 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.491301    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"]
Jul 16 23:03:09.633989 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.633956    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr\" (UID: \"54744640-4343-42c5-974a-76252c616ef0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:09.735472 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.735399    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr\" (UID: \"54744640-4343-42c5-974a-76252c616ef0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:09.735976 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.735958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr\" (UID: \"54744640-4343-42c5-974a-76252c616ef0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:09.798542 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.798517    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:09.927853 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:09.927829    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"]
Jul 16 23:03:09.930157 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:03:09.930131    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54744640_4343_42c5_974a_76252c616ef0.slice/crio-e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2 WatchSource:0}: Error finding container e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2: Status 404 returned error can't find the container with id e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2
Jul 16 23:03:10.144878 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:10.144842    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr" event={"ID":"54744640-4343-42c5-974a-76252c616ef0","Type":"ContainerStarted","Data":"e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2"}
Jul 16 23:03:10.676282 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:10.676257    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:11.151275 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:11.151238    2556 generic.go:356] "Generic (PLEG): container finished" podID="54744640-4343-42c5-974a-76252c616ef0" containerID="1b3479a81eebd3e67f940b2a8a7e9fee7999ee9ba8973516a1179d67a7081653" exitCode=0
Jul 16 23:03:11.151687 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:11.151317    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr" event={"ID":"54744640-4343-42c5-974a-76252c616ef0","Type":"ContainerDied","Data":"1b3479a81eebd3e67f940b2a8a7e9fee7999ee9ba8973516a1179d67a7081653"}
Jul 16 23:03:12.283614 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:12.283592    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:12.355143 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:12.355118    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca\") pod \"54744640-4343-42c5-974a-76252c616ef0\" (UID: \"54744640-4343-42c5-974a-76252c616ef0\") "
Jul 16 23:03:12.355442 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:12.355422    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54744640-4343-42c5-974a-76252c616ef0" (UID: "54744640-4343-42c5-974a-76252c616ef0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 23:03:12.456207 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:12.456176    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54744640-4343-42c5-974a-76252c616ef0-maas-api-service-ca\") on node \"ip-10-0-133-196.ec2.internal\" DevicePath \"\""
Jul 16 23:03:13.163921 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:13.163847    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr"
Jul 16 23:03:13.163921 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:13.163897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr" event={"ID":"54744640-4343-42c5-974a-76252c616ef0","Type":"ContainerDied","Data":"e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2"}
Jul 16 23:03:13.164125 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:13.163956    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e62f14f2bb25c54afbbf498d905b258807a5ed047821ad2b6d9de45862aa42f2"
Jul 16 23:03:19.948643 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:19.948613    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:19.949124 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:19.949085    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"]
Jul 16 23:03:19.953221 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:19.953201    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-5159e9-29737380-2nt9c"]
Jul 16 23:03:20.829343 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:20.829313    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d40d19b-3349-418e-a01a-fff6f15288cb" path="/var/lib/kubelet/pods/6d40d19b-3349-418e-a01a-fff6f15288cb/volumes"
Jul 16 23:03:22.847674 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:22.847643    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ggpdz"]
Jul 16 23:03:24.572577 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:24.572548    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:25.000369 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:25.000338    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:30.928288 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:30.928257    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:30.986431 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:30.986403    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:31.210438 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:31.210399    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:31.472136 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:31.472026    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:38.408308 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:38.408284    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7846b6849-dpdwb_c2766481-5285-4194-81e8-87a5aad77882/copy-manifests/0.log"
Jul 16 23:03:38.418108 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:38.418056    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7846b6849-dpdwb_c2766481-5285-4194-81e8-87a5aad77882/manager/0.log"
Jul 16 23:03:39.049384 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.049351    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737350-gqdzj_3eba2648-7ccf-424c-b8cb-ba41d6538415/cleanup/0.log"
Jul 16 23:03:39.165520 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.165496    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737365-m7tfh_b332d32a-dc62-4775-80d5-df7015f9009b/cleanup/0.log"
Jul 16 23:03:39.286233 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.286210    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737380-rfwbq_5ebb9e22-4193-4066-9f9d-5671ea7f5f09/cleanup/0.log"
Jul 16 23:03:39.400385 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.400307    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-667ee841-blb4v_136a456d-e8a7-49b6-8b7a-c6d3451a304a/revoke-keys/1.log"
Jul 16 23:03:39.516807 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.516779    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-f24d3bde-knx49_353369c8-0b84-4141-9c68-3cdeda9b48fc/revoke-keys/0.log"
Jul 16 23:03:39.635423 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.635392    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-ddb113-745pt_e661cab8-46cc-4750-aec2-5988bdccb56d/revoke-keys/0.log"
Jul 16 23:03:39.754151 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.754121    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-ef186c-zbg42_c070bfa7-207f-4007-b063-2fabaca903b6/revoke-keys/0.log"
Jul 16 23:03:39.870479 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.870455    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-67d0ce59-7xxlw_6f5711d3-2abd-4e83-b725-89dd563f67ba/revoke-keys/0.log"
Jul 16 23:03:39.991203 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:39.991174    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-acf29b-f59xj_ed56dcf1-401e-4015-b67c-f3852e448a93/revoke-keys/0.log"
Jul 16 23:03:40.106698 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.106631    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-db89db-wpmsv_d7cf5d4d-8f52-4262-84d0-ea47251587b2/revoke-keys/0.log"
Jul 16 23:03:40.220073 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.220041    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-907fee31-v75ck_554ddccd-4025-4b10-aa29-ab092027ddad/revoke-keys/0.log"
Jul 16 23:03:40.335404 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.335376    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-0f02bfcd-qvmfx_01d3686b-43d3-4103-afbb-f1757a038045/revoke-keys/0.log"
Jul 16 23:03:40.459018 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.458992    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-8a492916-wmbwr_37040a06-662d-4799-9fed-8774764b4bdf/revoke-keys/0.log"
Jul 16 23:03:40.576130 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.576081    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-35a77a-cspnr_f3b0f0d3-0307-4d61-a178-2d5612eb36d1/revoke-keys/0.log"
Jul 16 23:03:40.691601 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:40.691574    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-5159e9-7sblr_54744640-4343-42c5-974a-76252c616ef0/revoke-keys/0.log"
Jul 16 23:03:42.501488 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:42.501460    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d2p6f_1ae77ea2-37a4-43f8-9360-c6f8b747dc29/kuadrant-console-plugin/0.log"
Jul 16 23:03:42.890434 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:42.890365    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggpdz_3b45869b-1334-4e93-b0be-65955acd6202/limitador/0.log"
Jul 16 23:03:43.628179 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:43.628153    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4cc9ff56-bhmkb_b919f83e-d47b-4852-9af1-13a32ed03a4d/kube-auth-proxy/0.log"
Jul 16 23:03:44.239474 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:44.239450    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6474b87dd6-w47mb_723ca0d0-33df-4714-af5e-9513f7ec0c81/router/0.log"
Jul 16 23:03:44.605347 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:44.605253    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lvwm8_c793bc1a-af86-42e8-b0b9-49c1d940d443/main/0.log"
Jul 16 23:03:45.204343 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:45.204317    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-w7np2_89e021f5-461e-48f8-9a8e-244e0251504d/main/0.log"
Jul 16 23:03:50.638872 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.638841    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kj5sf/must-gather-2fxdr"]
Jul 16 23:03:50.639296 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.639217    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54744640-4343-42c5-974a-76252c616ef0" containerName="revoke-keys"
Jul 16 23:03:50.639296 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.639229    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="54744640-4343-42c5-974a-76252c616ef0" containerName="revoke-keys"
Jul 16 23:03:50.639296 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.639280    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="54744640-4343-42c5-974a-76252c616ef0" containerName="revoke-keys"
Jul 16 23:03:50.642612 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.642597    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.644680 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.644635    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj5sf\"/\"kube-root-ca.crt\""
Jul 16 23:03:50.644680 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.644636    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kj5sf\"/\"default-dockercfg-6lm9r\""
Jul 16 23:03:50.644897 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.644740    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj5sf\"/\"openshift-service-ca.crt\""
Jul 16 23:03:50.656312 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.656294    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj5sf/must-gather-2fxdr"]
Jul 16 23:03:50.661129 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.661085    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:50.667732 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.667711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/405d7fa9-363a-456f-adb0-e3815d859557-must-gather-output\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.667851 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.667772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5477\" (UniqueName: \"kubernetes.io/projected/405d7fa9-363a-456f-adb0-e3815d859557-kube-api-access-l5477\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.732365 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.732331    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:50.768947 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.768915    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5477\" (UniqueName: \"kubernetes.io/projected/405d7fa9-363a-456f-adb0-e3815d859557-kube-api-access-l5477\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.769122 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.769001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/405d7fa9-363a-456f-adb0-e3815d859557-must-gather-output\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.769323 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.769301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/405d7fa9-363a-456f-adb0-e3815d859557-must-gather-output\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.777211 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.777190    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj5sf\"/\"kube-root-ca.crt\""
Jul 16 23:03:50.788237 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.788219    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kj5sf\"/\"openshift-service-ca.crt\""
Jul 16 23:03:50.799950 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.799918    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5477\" (UniqueName: \"kubernetes.io/projected/405d7fa9-363a-456f-adb0-e3815d859557-kube-api-access-l5477\") pod \"must-gather-2fxdr\" (UID: \"405d7fa9-363a-456f-adb0-e3815d859557\") " pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:50.953250 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.953210    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kj5sf\"/\"default-dockercfg-6lm9r\""
Jul 16 23:03:50.961219 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:50.961197    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj5sf/must-gather-2fxdr"
Jul 16 23:03:51.091378 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:51.091355    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj5sf/must-gather-2fxdr"]
Jul 16 23:03:51.093258 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:03:51.093219    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod405d7fa9_363a_456f_adb0_e3815d859557.slice/crio-79c2fbdf23a98a23f7f57b4b49cccc2eabc2a2e6baaba1188f1bc6ff7bb8aea1 WatchSource:0}: Error finding container 79c2fbdf23a98a23f7f57b4b49cccc2eabc2a2e6baaba1188f1bc6ff7bb8aea1: Status 404 returned error can't find the container with id 79c2fbdf23a98a23f7f57b4b49cccc2eabc2a2e6baaba1188f1bc6ff7bb8aea1
Jul 16 23:03:51.352667 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:51.352559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj5sf/must-gather-2fxdr" event={"ID":"405d7fa9-363a-456f-adb0-e3815d859557","Type":"ContainerStarted","Data":"79c2fbdf23a98a23f7f57b4b49cccc2eabc2a2e6baaba1188f1bc6ff7bb8aea1"}
Jul 16 23:03:53.367607 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:53.367567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj5sf/must-gather-2fxdr" event={"ID":"405d7fa9-363a-456f-adb0-e3815d859557","Type":"ContainerStarted","Data":"e0dd30d767e7ecb803fc1a94a7e47148185f7b34dc619f23f9d9fc8fde4c44ab"}
Jul 16 23:03:53.368125 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:53.367618    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj5sf/must-gather-2fxdr" event={"ID":"405d7fa9-363a-456f-adb0-e3815d859557","Type":"ContainerStarted","Data":"e7ee91f22adcf3c7c8685764a99829361b1e5218ed64da7eb9a46f5475782816"}
Jul 16 23:03:53.385907 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:53.385846    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kj5sf/must-gather-2fxdr" podStartSLOduration=2.058355777 podStartE2EDuration="3.385824309s" totalImagesPullingTime="1.327468532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 23:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:03:53.384026174 +0000 UTC m=+3003.078608226" watchObservedRunningTime="2026-07-16 23:03:53.385824309 +0000 UTC m=+3003.080406170"
Jul 16 23:03:54.047376 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:54.047348    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p4d5g_f13b359c-19ed-410b-9eaf-58148f522ff7/global-pull-secret-syncer/0.log"
Jul 16 23:03:54.139532 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:54.139496    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log"
Jul 16 23:03:54.265848 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:54.265822    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-196.ec2.internal_94f9424feb20d8f7b53d0fbc4a5441ee/haproxy/0.log"
Jul 16 23:03:54.576120 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:54.576071    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fszkn_9be0f3d4-96b6-4874-a4f7-8d9cffbc254a/konnectivity-agent/0.log" limit=1024
Jul 16 23:03:58.547379 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:58.547256    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d2p6f_1ae77ea2-37a4-43f8-9360-c6f8b747dc29/kuadrant-console-plugin/0.log"
Jul 16 23:03:58.764701 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:03:58.764594    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ggpdz_3b45869b-1334-4e93-b0be-65955acd6202/limitador/0.log"
Jul 16 23:04:00.624715 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.624685    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sk5cz_10068efb-ca2a-45c4-a540-cf97919885c1/kube-state-metrics/0.log"
Jul 16 23:04:00.652138 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.652110    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sk5cz_10068efb-ca2a-45c4-a540-cf97919885c1/kube-rbac-proxy-main/0.log"
Jul 16 23:04:00.680119 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.679231    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sk5cz_10068efb-ca2a-45c4-a540-cf97919885c1/kube-rbac-proxy-self/0.log"
Jul 16 23:04:00.733781 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.733753    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-r9vdc_43fa0815-a91f-40a2-9e4d-f7ac88bc8324/monitoring-plugin/0.log"
Jul 16 23:04:00.770296 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.770271    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87xpq_4e880ff9-2a57-4efe-a266-9a62ef3b8b1c/node-exporter/0.log"
Jul 16 23:04:00.793722 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.793693    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87xpq_4e880ff9-2a57-4efe-a266-9a62ef3b8b1c/kube-rbac-proxy/0.log"
Jul 16 23:04:00.818004 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:00.817980    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-87xpq_4e880ff9-2a57-4efe-a266-9a62ef3b8b1c/init-textfile/0.log"
Jul 16 23:04:01.123615 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.123583    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/prometheus/0.log"
Jul 16 23:04:01.150882 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.150854    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/config-reloader/0.log"
Jul 16 23:04:01.183682 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.183646    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/thanos-sidecar/0.log"
Jul 16 23:04:01.233538 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.233453    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/kube-rbac-proxy-web/0.log"
Jul 16 23:04:01.278619 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.278558    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/kube-rbac-proxy/0.log"
Jul 16 23:04:01.311267 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.311210    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/kube-rbac-proxy-thanos/0.log"
Jul 16 23:04:01.337284 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:01.337245    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f7603c3-9316-4bfa-bb67-1312bade264b/init-config-reloader/0.log"
Jul 16 23:04:03.937361 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:03.937319    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"]
Jul 16 23:04:03.955812 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:03.955224    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:03.955986 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:03.955818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"]
Jul 16 23:04:04.007576 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.007541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-proc\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.007576 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.007576    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-lib-modules\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.007832 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.007713    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-sys\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.007832 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.007766    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-podres\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.007832 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.007794    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97nh6\" (UniqueName: \"kubernetes.io/projected/85c0220d-15f7-4a69-bd50-16e68fce5648-kube-api-access-97nh6\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108346 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108309    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-proc\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108346 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108352    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-lib-modules\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108633 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108438    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-sys\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108633 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108448    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-proc\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108633 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108480    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-podres\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108633 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108510    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97nh6\" (UniqueName: \"kubernetes.io/projected/85c0220d-15f7-4a69-bd50-16e68fce5648-kube-api-access-97nh6\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108931 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-lib-modules\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108931 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108686    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-sys\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.108931 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.108765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/85c0220d-15f7-4a69-bd50-16e68fce5648-podres\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.117308 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.117264    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97nh6\" (UniqueName: \"kubernetes.io/projected/85c0220d-15f7-4a69-bd50-16e68fce5648-kube-api-access-97nh6\") pod \"perf-node-gather-daemonset-sskn6\" (UID: \"85c0220d-15f7-4a69-bd50-16e68fce5648\") " pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.276299 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.276218    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:04.379447 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.379421    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bd646fcdd-wbgxb_0ae657ca-e87f-4ce0-abee-7fe6e1440ce6/console/0.log"
Jul 16 23:04:04.445644 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.445621    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"]
Jul 16 23:04:04.449277 ip-10-0-133-196 kubenswrapper[2556]: W0716 23:04:04.449252    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod85c0220d_15f7_4a69_bd50_16e68fce5648.slice/crio-54c9e92cd64e3d1516b1ec6c11e6d4679ae6926328699746e33c79791db8b7ae WatchSource:0}: Error finding container 54c9e92cd64e3d1516b1ec6c11e6d4679ae6926328699746e33c79791db8b7ae: Status 404 returned error can't find the container with id 54c9e92cd64e3d1516b1ec6c11e6d4679ae6926328699746e33c79791db8b7ae
Jul 16 23:04:04.474135 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:04.473123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6" event={"ID":"85c0220d-15f7-4a69-bd50-16e68fce5648","Type":"ContainerStarted","Data":"54c9e92cd64e3d1516b1ec6c11e6d4679ae6926328699746e33c79791db8b7ae"}
Jul 16 23:04:05.114613 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:05.114581    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-kkbnv_0b15d6e1-4de4-403e-a418-125c81ca4701/volume-data-source-validator/0.log"
Jul 16 23:04:05.482114 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:05.482044    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6" event={"ID":"85c0220d-15f7-4a69-bd50-16e68fce5648","Type":"ContainerStarted","Data":"d3b184af23cf6b7b84ca04d80ed53c8484c27779133d3e349639ee2bbc160c55"}
Jul 16 23:04:05.482114 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:05.482122    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:05.497338 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:05.497291    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6" podStartSLOduration=2.497275477 podStartE2EDuration="2.497275477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 23:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 23:04:05.49588218 +0000 UTC m=+3015.190464035" watchObservedRunningTime="2026-07-16 23:04:05.497275477 +0000 UTC m=+3015.191857331"
Jul 16 23:04:06.222452 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:06.222423    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wr797_cc26a0a7-6613-4580-a2f6-0cd5c3577633/dns/0.log"
Jul 16 23:04:06.249715 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:06.249691    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wr797_cc26a0a7-6613-4580-a2f6-0cd5c3577633/kube-rbac-proxy/0.log"
Jul 16 23:04:06.304072 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:06.304035    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-db84x_6ad72e05-81f9-449a-8f6f-a552cdbe9748/dns-node-resolver/0.log"
Jul 16 23:04:06.969181 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:06.969152    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7bngx_f3065d8c-65a8-4b78-b859-3fd67a18a667/node-ca/0.log"
Jul 16 23:04:08.388576 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:08.388551    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4cc9ff56-bhmkb_b919f83e-d47b-4852-9af1-13a32ed03a4d/kube-auth-proxy/0.log"
Jul 16 23:04:08.666318 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:08.666254    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6474b87dd6-w47mb_723ca0d0-33df-4714-af5e-9513f7ec0c81/router/0.log"
Jul 16 23:04:09.390856 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:09.390826    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bgqbq_e45c7d47-d770-41aa-bfba-28497d321d0a/serve-healthcheck-canary/0.log"
Jul 16 23:04:10.062693 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:10.062663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5f86_bee0c7fa-9c9e-4c2e-952a-099ac0774fbd/kube-rbac-proxy/0.log"
Jul 16 23:04:10.088393 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:10.088367    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5f86_bee0c7fa-9c9e-4c2e-952a-099ac0774fbd/exporter/0.log"
Jul 16 23:04:10.119936 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:10.119913    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5f86_bee0c7fa-9c9e-4c2e-952a-099ac0774fbd/extractor/0.log"
Jul 16 23:04:11.499824 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:11.499795    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kj5sf/perf-node-gather-daemonset-sskn6"
Jul 16 23:04:13.078136 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:13.078109    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7846b6849-dpdwb_c2766481-5285-4194-81e8-87a5aad77882/manager/0.log"
Jul 16 23:04:13.106925 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:13.106849    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7846b6849-dpdwb_c2766481-5285-4194-81e8-87a5aad77882/copy-manifests/0.log"
Jul 16 23:04:14.322992 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:14.322960    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4cc9ff56-bhmkb_b919f83e-d47b-4852-9af1-13a32ed03a4d/kube-auth-proxy/0.log"
Jul 16 23:04:14.474948 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:14.474916    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6474b87dd6-w47mb_723ca0d0-33df-4714-af5e-9513f7ec0c81/router/0.log"
Jul 16 23:04:14.507119 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:14.507038    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f787d6dbd-n4lnb_99f3c696-2099-4aeb-9665-f9800efe68e5/manager/0.log"
Jul 16 23:04:15.964467 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:15.964442    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4cc9ff56-bhmkb_b919f83e-d47b-4852-9af1-13a32ed03a4d/kube-auth-proxy/0.log"
Jul 16 23:04:16.119488 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:16.119459    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6474b87dd6-w47mb_723ca0d0-33df-4714-af5e-9513f7ec0c81/router/0.log"
Jul 16 23:04:19.595082 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:19.595046    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-gsq2z_07121a77-cc80-4445-8c0a-76d471019689/migrator/0.log"
Jul 16 23:04:19.617322 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:19.617294    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-gsq2z_07121a77-cc80-4445-8c0a-76d471019689/graceful-termination/0.log"
Jul 16 23:04:20.004130 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:20.004077    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-t5xcs_35ba01bc-9376-48d0-b96f-80ee68024cff/kube-storage-version-migrator-operator/1.log"
Jul 16 23:04:20.005131 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:20.005108    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-t5xcs_35ba01bc-9376-48d0-b96f-80ee68024cff/kube-storage-version-migrator-operator/0.log"
Jul 16 23:04:21.045891 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:21.045784    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8zhn4_a017371e-d257-4773-bf5d-4af39c11965f/kube-multus/0.log"
Jul 16 23:04:21.235868 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:21.235829    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4zxq5_89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b/kube-multus-additional-cni-plugins/0.log"
Jul 16 23:04:21.264406 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:21.264384    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4zxq5_89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b/egress-router-binary-copy/0.log"
Jul 16 23:04:21.291281 ip-10-0-133-196 kubenswrapper[2556]: I0716 23:04:21.291259    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4zxq5_89c39ba4-5c7f-4b98-a0bf-6d0c2fb5712b/cni-plugins/0.log"