Jul 29 00:54:24.805550 ip-10-0-129-224 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 00:54:25.211874 ip-10-0-129-224 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:25.211874 ip-10-0-129-224 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 00:54:25.211874 ip-10-0-129-224 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:25.211874 ip-10-0-129-224 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 00:54:25.211874 ip-10-0-129-224 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:25.213710 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.213600    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 00:54:25.217023 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217008    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.217023 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217022    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217026    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217030    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217035    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217040    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217043    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217048    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217052    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217055    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217058    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217061    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217064    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217067    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217070    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217073    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217075    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217078    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217080    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217083    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.217086 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217086    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217089    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217092    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217095    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217098    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217100    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217103    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217106    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217108    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217111    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217113    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217116    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217118    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217121    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217123    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217126    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217128    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217131    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217134    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217136    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.217553 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217139    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217142    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217146    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217149    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217151    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217154    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217158    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217160    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217163    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217166    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217169    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217171    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217174    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217178    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217181    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217184    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217187    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217189    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217192    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217194    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.218032 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217197    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217199    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217202    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217204    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217207    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217210    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217212    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217215    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217218    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217221    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217223    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217226    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217229    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217232    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217234    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217237    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217240    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217242    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217245    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217247    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.218562 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217250    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217252    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217267    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217270    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217273    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217275    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217279    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217282    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217286    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217289    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217291    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217294    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217297    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217299    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217302    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217305    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217307    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217310    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.217313    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218014    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.219043 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218021    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218024    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218026    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218029    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218033    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218036    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218039    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218042    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218044    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218047    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218049    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218052    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218055    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218057    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218060    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218062    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218065    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218068    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218070    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218073    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.219544 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218076    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218078    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218081    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218084    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218086    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218089    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218092    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218094    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218097    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218099    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218103    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218108    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218112    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218115    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218118    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218121    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218124    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218127    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218130    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.220037 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218133    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218136    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218139    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218141    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218143    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218146    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218148    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218151    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218154    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218156    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218159    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218163    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218165    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218168    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218171    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218173    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218176    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218179    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218181    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218184    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.220506 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218186    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218189    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218191    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218194    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218196    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218201    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218204    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218206    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218209    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218211    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218214    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218217    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218220    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218223    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218225    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218229    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218232    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218234    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218236    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218239    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.221004 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218242    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218244    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218247    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218250    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218252    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218269    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218272    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218275    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218277    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218280    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218283    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218286    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218289    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218293    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218295    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218298    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218300    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218305    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218308    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218667    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.221501 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218672    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218675    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218678    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218681    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218684    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218686    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218689    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218691    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218694    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218696    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218699    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218702    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218705    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218707    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218710    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218713    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218715    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218718    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218720    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218724    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.222003 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218726    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218729    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218731    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218734    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218736    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218739    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218742    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218744    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218747    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218750    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218753    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218755    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218758    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218761    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218763    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218766    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218768    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218771    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218773    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218776    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.222498 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218778    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218782    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218787    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218791    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218795    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218798    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218801    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218804    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218807    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218810    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218813    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218815    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218818    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218821    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218824    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218826    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218829    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218832    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218835    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.223052 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218837    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218840    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218842    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218845    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218848    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218850    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218853    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218855    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218858    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218862    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218865    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218868    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218870    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218873    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218875    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218878    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218880    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218883    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218887    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218889    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.223539 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218892    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218894    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218897    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218899    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218902    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218905    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218907    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218910    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218913    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218915    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218918    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218920    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218923    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218925    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218928    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218930    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218933    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218935    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.218938    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219004    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219011    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 00:54:25.224013 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219018    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219023    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219028    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219031    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219035    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219039    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219042    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219045    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219049    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219052    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219055    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219058    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219061    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219064    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219067    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219070    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219074    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219077    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219080    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219083    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219086    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219090    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219093    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 00:54:25.224516 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219096    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219099    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219102    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219105    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219108    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219111    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219114    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219118    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219121    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219125    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219127    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219130    2563 flags.go:64] FLAG: --enable-server="true"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219134    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219138    2563 flags.go:64] FLAG: --event-burst="100"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219141    2563 flags.go:64] FLAG: --event-qps="50"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219148    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219152    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219154    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219158    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219161    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219164    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219167    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219170    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219173    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219175    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 00:54:25.225073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219178    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219181    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219184    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219187    2563 flags.go:64] FLAG: --feature-gates=""
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219190    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219193    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219196    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219200    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219203    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219205    2563 flags.go:64] FLAG: --help="false"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219208    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219211    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219214    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219217    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219220    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219224    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219228    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219231    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219234    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219237    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219240    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219243    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219247    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219249    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 00:54:25.225708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219252    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219267    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219271    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219274    2563 flags.go:64] FLAG: --lock-file=""
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219276    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219279    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219282    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219287    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219290    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219293    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219296    2563 flags.go:64] FLAG: --logging-format="text"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219299    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219302    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219305    2563 flags.go:64] FLAG: --manifest-url=""
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219308    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219312    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219314    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219319    2563 flags.go:64] FLAG: --max-pods="110"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219322    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219325    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219328    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219330    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219333    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219336    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219340    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 00:54:25.226339 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219347    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219350    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219353    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219356    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219360    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219367    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219370    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219373    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219376    2563 flags.go:64] FLAG: --port="10250"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219379    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219382    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05659af204580a35e"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219385    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219388    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219391    2563 flags.go:64] FLAG: --register-node="true"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219394    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219397    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219400    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219403    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219405    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219409    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219412    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219415    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219418    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219420    2563 flags.go:64] FLAG: --runonce="false"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219423    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 00:54:25.227015 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219426    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219429    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219432    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219435    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219438    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219441    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219444    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219448    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219451    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219455    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219457    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219460    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219469    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219472    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219477    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219480    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219483    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219487    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219490    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219492    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219496    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219498    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219501    2563 flags.go:64] FLAG: --v="2"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219505    2563 flags.go:64] FLAG: --version="false"
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219509    2563 flags.go:64] FLAG: --vmodule=""
Jul 29 00:54:25.227638 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219513    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.219516    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219624    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219629    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219632    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219635    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219638    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219641    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219644    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219646    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219649    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219651    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219654    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219657    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219661    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219664    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219667    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219670    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219673    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219677    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.228246 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219680    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219683    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219686    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219688    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219691    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219693    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219696    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219699    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219701    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219703    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219706    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219708    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219711    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219713    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219716    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219718    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219721    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219723    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219726    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.228746 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219728    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219730    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219733    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219735    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219738    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219741    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219745    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219748    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219751    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219754    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219757    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219759    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219763    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219766    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219769    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219771    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219774    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219776    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219779    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219781    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.229210 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219784    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219787    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219789    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219792    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219794    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219797    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219800    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219802    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219804    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219807    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219810    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219812    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219815    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219817    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219820    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219822    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219825    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219827    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219830    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.229722 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219832    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219835    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219837    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219839    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219842    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219846    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219848    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219851    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219854    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219856    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219859    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219861    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219864    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219866    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219869    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219872    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219876    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219879    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219881    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.230191 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219884    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219886    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219889    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.219891    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.221226    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.227786    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.227801    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.228515    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.228522    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228579    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228584    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228588    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228591    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228594    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228597    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.230756 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228600    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228603    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228606    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228608    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228611    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228614    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228617    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228620    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228623    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228625    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228628    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228631    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228634    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228637    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228639    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228642    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228644    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228647    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228649    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228652    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.231128 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228655    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228657    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228659    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228664    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228668    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228671    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228674    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228677    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228679    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228682    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228684    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228687    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228690    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228692    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228695    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228698    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228700    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228703    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228705    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228709    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.231655 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228712    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228714    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228717    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228720    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228722    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228725    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228728    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228731    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228733    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228736    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228738    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228741    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228743    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228746    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228749    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228752    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228755    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228758    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228760    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228763    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.232140 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228767    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228771    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228773    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228776    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228779    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228781    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228784    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228787    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228789    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228792    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228794    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228797    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228800    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228803    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228805    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228808    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228810    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228815    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228818    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.232638 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228821    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228824    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228827    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228829    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228832    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228834    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228837    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228840    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228843    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228845    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228848    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228850    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228853    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228855    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.228860    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:25.233199 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228968    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228972    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228975    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228978    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228981    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228984    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228987    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228990    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228993    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228996    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.228998    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229002    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229004    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229007    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229010    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229014    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229017    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229020    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229023    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:25.233644 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229025    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229028    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229030    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229033    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229036    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229039    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229041    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229044    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229046    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229049    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229052    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229054    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229057    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229060    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229062    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229065    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229067    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229070    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229072    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229075    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:25.234203 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229078    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229080    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229083    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229085    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229088    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229091    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229094    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229096    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229099    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229101    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229104    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229107    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229109    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229111    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229114    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229117    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229119    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229122    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229125    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229127    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:25.234751 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229130    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229132    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229135    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229137    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229140    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229142    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229145    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229147    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229150    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229152    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229155    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229157    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229160    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229162    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229166    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229169    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229172    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229175    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229178    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229181    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:25.235552 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229184    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229186    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229189    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229191    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229194    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229196    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229199    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229201    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229204    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229206    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229209    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229211    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229216    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229219    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229222    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229225    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229228    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229231    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:25.236082 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229234    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:25.229237    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.229242    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.229349    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.231369    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.232628    2563 server.go:1019] "Starting client certificate rotation"
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.232745    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.232787    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:54:25.236654 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.234832    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:54:25.239964 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.239944    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:54:25.254644 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.254624    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 29 00:54:25.257416 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.257402    2563 log.go:25] "Validated CRI v1 image API"
Jul 29 00:54:25.258353 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.258332    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 00:54:25.262393 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.262362    2563 fs.go:135] Filesystem UUIDs: map[4c96f964-fb68-4bdc-a5f0-d27792adf056:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 875f0e8e-e0c2-4be5-8573-00354699d8be:/dev/nvme0n1p3]
Jul 29 00:54:25.262459 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.262391    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 00:54:25.267531 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.267421    2563 manager.go:216] Machine: {Timestamp:2026-07-29 00:54:25.266221704 +0000 UTC m=+0.344208695 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3086002 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24720ad59f2f8e337d42b16dc75d84 SystemUUID:ec24720a-d59f-2f8e-337d-42b16dc75d84 BootID:db511b44-0195-4804-b0f0-960410ba9cf3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:60:2f:47:8e:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:60:2f:47:8e:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:b3:44:94:a0:41 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 00:54:25.267531 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.267526    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 00:54:25.267637 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.267597    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 00:54:25.270171 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.270154    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:54:25.270447 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.270419    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 00:54:25.270586 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.270450    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 00:54:25.271095 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.271085    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 00:54:25.271128 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.271098    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 00:54:25.271128 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.271105    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:54:25.271739 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.271729    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:54:25.272522 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.272511    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 00:54:25.275118 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275105    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:54:25.275232 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275222    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 00:54:25.275283 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275254    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 00:54:25.275283 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275280    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 00:54:25.275343 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275294    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 00:54:25.275343 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275303    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 29 00:54:25.275343 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275311    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 00:54:25.275641 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275629    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:54:25.275677 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.275648    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:54:25.278300 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.278281    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 00:54:25.280047 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.280031    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 00:54:25.280126 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.280070    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 00:54:25.281636 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281621    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281640    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281650    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281657    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281669    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281677    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281689    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281697    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281707    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281715    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 00:54:25.281726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281728    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 00:54:25.282034 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.281737    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 00:54:25.283559 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.283547    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 00:54:25.283609 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.283561    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 00:54:25.286724 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.286702    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 00:54:25.286830 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.286806    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 00:54:25.286884 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.286854    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 00:54:25.287176 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.287161    2563 server.go:1268] "Started kubelet"
Jul 29 00:54:25.287686 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.287649    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 00:54:25.288135 ip-10-0-129-224 systemd[1]: Started Kubernetes Kubelet.
Jul 29 00:54:25.288402 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.288131    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 00:54:25.288402 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.288198    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 00:54:25.289469 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.289454    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 00:54:25.289617 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.289600    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 00:54:25.290950 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.290933    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxgbf"
Jul 29 00:54:25.293250 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.293231    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 00:54:25.296282 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.295419    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-224.ec2.internal.18c69c94d1b94698  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-224.ec2.internal,UID:ip-10-0-129-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-224.ec2.internal,},FirstTimestamp:2026-07-29 00:54:25.286653592 +0000 UTC m=+0.364640586,LastTimestamp:2026-07-29 00:54:25.286653592 +0000 UTC m=+0.364640586,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-224.ec2.internal,}"
Jul 29 00:54:25.296564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.296546    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 00:54:25.296564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.296554    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 00:54:25.296705 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.296622    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.297082 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.297067    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 00:54:25.297167 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.297070    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 00:54:25.297167 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.297163    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 00:54:25.297290 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.297251    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 00:54:25.297290 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.297277    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 00:54:25.298512 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.298459    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nxgbf"
Jul 29 00:54:25.298787 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.298652    2563 factory.go:55] Registering systemd factory
Jul 29 00:54:25.298787 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.298670    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 29 00:54:25.299072 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299055    2563 factory.go:153] Registering CRI-O factory
Jul 29 00:54:25.299072 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299074    2563 factory.go:223] Registration of the crio container factory successfully
Jul 29 00:54:25.299270 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299124    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 00:54:25.299270 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299145    2563 factory.go:103] Registering Raw factory
Jul 29 00:54:25.299270 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299158    2563 manager.go:1196] Started watching for new ooms in manager
Jul 29 00:54:25.299568 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.299544    2563 manager.go:349] Starting recovery of all containers
Jul 29 00:54:25.305106 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.305075    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 00:54:25.307101 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.307076    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:25.309141 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.309121    2563 manager.go:354] Recovery completed
Jul 29 00:54:25.309517 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.309497    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-224.ec2.internal\" not found" node="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.313512 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.313498    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.316138 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316124    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.316195 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316151    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.316195 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316162    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.316695 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316682    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 00:54:25.316744 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316695    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 00:54:25.316744 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316710    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:54:25.316744 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.316683    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.317856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.317842    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.317917 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.317869    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.317917 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.317882    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.319056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.319044    2563 policy_none.go:49] "None policy: Start"
Jul 29 00:54:25.319092 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.319059    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 00:54:25.319092 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.319069    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 00:54:25.320970 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.320958    2563 policy_none.go:47] "Start"
Jul 29 00:54:25.361867 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.361737    2563 manager.go:337] "Starting Device Plugin manager"
Jul 29 00:54:25.361951 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.361886    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 00:54:25.361951 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.361899    2563 server.go:87] "Starting device plugin registration server"
Jul 29 00:54:25.362151 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.362139    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 00:54:25.362183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.362153    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 00:54:25.362287 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.362271    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 00:54:25.362453 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.362436    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 00:54:25.362453 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.362450    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 00:54:25.362800 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.362782    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 00:54:25.362873 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.362852    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.462546 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.462455    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.464485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.463392    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.464485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.463420    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.464485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.463431    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.464485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.463460    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.468633 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.468610    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 00:54:25.468721 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.468647    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 00:54:25.468721 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.468669    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 00:54:25.468721 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.468698    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 00:54:25.471145 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.471128    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:25.471476 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.471461    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.471531 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.471491    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-224.ec2.internal\": node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.511869 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.511852    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.569631 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.569009    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"]
Jul 29 00:54:25.569716 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.569675    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.570519 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.570501    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.570626 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.570535    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.570626 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.570551    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.572974 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.572958    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.573102 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.573154 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573118    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.573714 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573695    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.573808 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573725    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.573808 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573736    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.573808 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573695    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.573921 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573808    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.573921 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.573822    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.576163 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.576149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.576212 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.576174    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:25.576874 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.576860    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:25.576955 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.576883    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:25.576955 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.576893    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:25.597190 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.597169    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-224.ec2.internal\" not found" node="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.599318 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.599284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.599422 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.599345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.599422 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.599375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac1805d9fb4faade8c3be0e92a054aab-config\") pod \"kube-apiserver-proxy-ip-10-0-129-224.ec2.internal\" (UID: \"ac1805d9fb4faade8c3be0e92a054aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.601888 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.601871    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-224.ec2.internal\" not found" node="ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.612782 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.612761    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.700554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.700554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.700768 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.700768 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac1805d9fb4faade8c3be0e92a054aab-config\") pod \"kube-apiserver-proxy-ip-10-0-129-224.ec2.internal\" (UID: \"ac1805d9fb4faade8c3be0e92a054aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.700768 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac1805d9fb4faade8c3be0e92a054aab-config\") pod \"kube-apiserver-proxy-ip-10-0-129-224.ec2.internal\" (UID: \"ac1805d9fb4faade8c3be0e92a054aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.700768 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.700668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f124187e52f9f4dd8797d0bf6271156-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal\" (UID: \"9f124187e52f9f4dd8797d0bf6271156\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.713575 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.713522    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.814322 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.814280    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:25.902812 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.902773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.906061 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:25.906043    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:25.914604 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:25.914586    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.015335 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:26.015246    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.115868 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:26.115838    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.141450 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.141423    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:26.216022 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:26.215989    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.232350 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.232328    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 00:54:26.232460 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.232443    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:54:26.232524 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.232502    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:54:26.232559 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.232499    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:54:26.303569 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.303486    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 00:49:25 +0000 UTC" deadline="2028-01-04 16:51:50.053872224 +0000 UTC"
Jul 29 00:54:26.303569 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.303526    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12591h57m23.750349796s"
Jul 29 00:54:26.316941 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:26.316921    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.417065 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:26.417036    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-224.ec2.internal\" not found"
Jul 29 00:54:26.417545 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.417528    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:26.498719 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.498694    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:26.510850 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.510832    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:54:26.511835 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.511822    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal"
Jul 29 00:54:26.520985 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.520967    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:54:26.525654 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:26.525626    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1805d9fb4faade8c3be0e92a054aab.slice/crio-59f915cd6431be2e11b98a04f869b7cbe407dc979937fb80e732d76580c95c1f WatchSource:0}: Error finding container 59f915cd6431be2e11b98a04f869b7cbe407dc979937fb80e732d76580c95c1f: Status 404 returned error can't find the container with id 59f915cd6431be2e11b98a04f869b7cbe407dc979937fb80e732d76580c95c1f
Jul 29 00:54:26.526789 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:26.526761    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f124187e52f9f4dd8797d0bf6271156.slice/crio-a18e99706142c1dac3d1e6af7752e372fdc397e7732cbca3e0171c948ea701cd WatchSource:0}: Error finding container a18e99706142c1dac3d1e6af7752e372fdc397e7732cbca3e0171c948ea701cd: Status 404 returned error can't find the container with id a18e99706142c1dac3d1e6af7752e372fdc397e7732cbca3e0171c948ea701cd
Jul 29 00:54:26.533773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:26.532168    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:54:27.141144 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.141111    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:27.278453 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.278419    2563 apiserver.go:52] "Watching apiserver"
Jul 29 00:54:27.290247 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.290218    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 00:54:27.290786 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.290762    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7fr2p","kube-system/konnectivity-agent-cfhvf","kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q","openshift-cluster-node-tuning-operator/tuned-nhhsf","openshift-dns/node-resolver-g2jdj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal","openshift-multus/multus-n5c74","openshift-image-registry/node-ca-kfvrh","openshift-multus/multus-additional-cni-plugins-4q7dd","openshift-multus/network-metrics-daemon-ckvff","openshift-network-diagnostics/network-check-target-564xp","openshift-network-operator/iptables-alerter-db2dj","openshift-ovn-kubernetes/ovnkube-node-f4sgp"]
Jul 29 00:54:27.293692 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.293665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.296464 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.296445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.297145 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.297119    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.297598    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.297895    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.298080    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.298247    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.298318    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pthxm\""
Jul 29 00:54:27.298471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.298250    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 00:54:27.300298 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.300261    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.300467 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.300449    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.300528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.300486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k9gmb\""
Jul 29 00:54:27.300990 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.300967    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 00:54:27.301832 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.301436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.301986 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.301967    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.303742 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.303701    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 00:54:27.304187 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.304162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.305750 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.305728    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.307874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309068    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-ovn\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-env-overrides\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-netd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-script-lib\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-os-release\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-log-socket\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cnibin\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovn-node-metrics-cert\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6xck\" (UniqueName: \"kubernetes.io/projected/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-kube-api-access-z6xck\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-slash\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.312324 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzgbp\" (UniqueName: \"kubernetes.io/projected/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-kube-api-access-vzgbp\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-system-cni-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-binary-copy\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309877    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-iptables-alerter-script\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-systemd-units\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-config\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.309974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ssmh\" (UniqueName: \"kubernetes.io/projected/1d9f36bc-1cf4-4e46-8333-315bf80cb050-kube-api-access-6ssmh\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-host-slash\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-var-lib-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-etc-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-netns\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-node-log\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-bin\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-kubelet\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-systemd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.313277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310680    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9sk8x\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.310882    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312026    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312264    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312490    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hrdgv\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312546    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312561    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312699    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312732    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.314096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.312844    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vc797\""
Jul 29 00:54:27.314635 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.314407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.315065 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.315048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.316370 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.316124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.316370 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.316136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sgs2g\""
Jul 29 00:54:27.318057 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.318029    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 00:54:27.318161 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.318079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.320055 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.320027    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6d2n4\""
Jul 29 00:54:27.321107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.321075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 00:54:27.321437 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.321418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 00:54:27.322803 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.322784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.324372 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.324350    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 00:54:27.324456 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.324417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7t24q\""
Jul 29 00:54:27.325352 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.325325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.325447 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.325416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:27.326511 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.326487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-98hxg\""
Jul 29 00:54:27.326511 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.326501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 00:54:27.326714 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.326695    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 00:54:27.327918 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.327898    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.328014 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.327988    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:27.330175 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.330157    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjtwg"
Jul 29 00:54:27.330500 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.330470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:27.330604 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.330534    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:27.343398 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.343378    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjtwg"
Jul 29 00:54:27.364702 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.364671    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:27.398666 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.398647    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 00:54:27.411006 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.410803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df7478cd-6084-4776-b323-38fa6eeaeddd-serviceca\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.411006 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.410848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcxxf\" (UniqueName: \"kubernetes.io/projected/df7478cd-6084-4776-b323-38fa6eeaeddd-kube-api-access-hcxxf\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.411006 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.410883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.411006 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.410912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-multus\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.411006 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.410974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-etc-kubernetes\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-kubelet-config\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-modprobe-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-os-release\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-hostroot\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdb6z\" (UniqueName: \"kubernetes.io/projected/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-kube-api-access-wdb6z\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.411390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.411233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-device-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.415447 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-os-release\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.415557 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgrs\" (UniqueName: \"kubernetes.io/projected/403c6a83-604e-4ebc-a605-ea919378a31b-kube-api-access-8sgrs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.415624 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.415682 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.415742 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-sys\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.415799 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.415856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovn-node-metrics-cert\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.415856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-conf-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.415959 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-daemon-config\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.415959 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df7478cd-6084-4776-b323-38fa6eeaeddd-host\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.416073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-socket-dir-parent\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.415995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-k8s-cni-cncf-io\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-lib-modules\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.416073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-var-lib-kubelet\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.416279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:27.416279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-system-cni-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.416279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-systemd-units\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.416279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-config\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.416279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-system-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-os-release\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cni-binary-copy\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-bin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-conf\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ssmh\" (UniqueName: \"kubernetes.io/projected/1d9f36bc-1cf4-4e46-8333-315bf80cb050-kube-api-access-6ssmh\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-systemd-units\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cnibin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-run\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.416668 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-host\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-node-log\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-bin\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-systemd\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-tmp\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4645\" (UniqueName: \"kubernetes.io/projected/a78dbafb-b423-4af6-8a40-0aebedf0bddb-kube-api-access-l4645\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-systemd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-ovn\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-env-overrides\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.416978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2e3b46e-2cdd-48e3-8704-dfa897beccda-hosts-file\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.417107 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417193    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-ovn\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417398    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-node-log\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-system-cni-dir\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-systemd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-bin\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.417663 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-netd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-env-overrides\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-script-lib\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-tuned\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-log-socket\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-kubelet\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eefea536-19ad-4de7-86c6-9e6a5d58bb17-konnectivity-ca\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-kubernetes\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cnibin\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.417980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-dbus\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.418035 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6xck\" (UniqueName: \"kubernetes.io/projected/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-kube-api-access-z6xck\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-slash\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzgbp\" (UniqueName: \"kubernetes.io/projected/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-kube-api-access-vzgbp\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-multus-certs\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-binary-copy\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-iptables-alerter-script\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-netns\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4f8p\" (UniqueName: \"kubernetes.io/projected/bcb66aad-89ed-482e-8d49-abefa4390ab9-kube-api-access-z4f8p\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-host-slash\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.418792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-var-lib-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-etc-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-netns\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e3b46e-2cdd-48e3-8704-dfa897beccda-tmp-dir\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418789    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cni-binary-copy\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gd5p\" (UniqueName: \"kubernetes.io/projected/e2e3b46e-2cdd-48e3-8704-dfa897beccda-kube-api-access-2gd5p\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysconfig\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-kubelet\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.418926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eefea536-19ad-4de7-86c6-9e6a5d58bb17-agent-certs\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-cni-netd\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-slash\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-run-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.419615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-log-socket\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d9f36bc-1cf4-4e46-8333-315bf80cb050-cnibin\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-iptables-alerter-script\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-config\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-etc-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-ovn-kubernetes\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-var-lib-openvswitch\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-kubelet\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-host-slash\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.420260 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.419918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-host-run-netns\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.420806 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.420781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovnkube-script-lib\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.423558 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.423532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-ovn-node-metrics-cert\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.426648 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.426622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ssmh\" (UniqueName: \"kubernetes.io/projected/1d9f36bc-1cf4-4e46-8333-315bf80cb050-kube-api-access-6ssmh\") pod \"multus-additional-cni-plugins-4q7dd\" (UID: \"1d9f36bc-1cf4-4e46-8333-315bf80cb050\") " pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.428920 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.428896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzgbp\" (UniqueName: \"kubernetes.io/projected/7742ce84-5d8c-45f2-bbed-e2e0fd23b428-kube-api-access-vzgbp\") pod \"ovnkube-node-f4sgp\" (UID: \"7742ce84-5d8c-45f2-bbed-e2e0fd23b428\") " pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.429313 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.429270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6xck\" (UniqueName: \"kubernetes.io/projected/3a89c0c7-585a-4d4f-906c-cf82e4baeba0-kube-api-access-z6xck\") pod \"iptables-alerter-db2dj\" (UID: \"3a89c0c7-585a-4d4f-906c-cf82e4baeba0\") " pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.474796 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.474746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal" event={"ID":"9f124187e52f9f4dd8797d0bf6271156","Type":"ContainerStarted","Data":"a18e99706142c1dac3d1e6af7752e372fdc397e7732cbca3e0171c948ea701cd"}
Jul 29 00:54:27.476130 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.476104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal" event={"ID":"ac1805d9fb4faade8c3be0e92a054aab","Type":"ContainerStarted","Data":"59f915cd6431be2e11b98a04f869b7cbe407dc979937fb80e732d76580c95c1f"}
Jul 29 00:54:27.519381 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-multus-certs\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-netns\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-multus-certs\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.519528 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-netns\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4f8p\" (UniqueName: \"kubernetes.io/projected/bcb66aad-89ed-482e-8d49-abefa4390ab9-kube-api-access-z4f8p\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e3b46e-2cdd-48e3-8704-dfa897beccda-tmp-dir\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gd5p\" (UniqueName: \"kubernetes.io/projected/e2e3b46e-2cdd-48e3-8704-dfa897beccda-kube-api-access-2gd5p\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.519674    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysconfig\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eefea536-19ad-4de7-86c6-9e6a5d58bb17-agent-certs\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.519730    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df7478cd-6084-4776-b323-38fa6eeaeddd-serviceca\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.519782    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:28.019738335 +0000 UTC m=+3.097725321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.519861    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:28.019841971 +0000 UTC m=+3.097828980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:27.519895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcxxf\" (UniqueName: \"kubernetes.io/projected/df7478cd-6084-4776-b323-38fa6eeaeddd-kube-api-access-hcxxf\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-multus\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysconfig\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.519990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-etc-kubernetes\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-kubelet-config\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-modprobe-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e3b46e-2cdd-48e3-8704-dfa897beccda-tmp-dir\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-hostroot\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdb6z\" (UniqueName: \"kubernetes.io/projected/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-kube-api-access-wdb6z\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-device-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgrs\" (UniqueName: \"kubernetes.io/projected/403c6a83-604e-4ebc-a605-ea919378a31b-kube-api-access-8sgrs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df7478cd-6084-4776-b323-38fa6eeaeddd-serviceca\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-sys\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-etc-kubernetes\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.520697 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-modprobe-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-multus\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-conf-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-kubelet-config\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-hostroot\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-daemon-config\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-conf-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-sys\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df7478cd-6084-4776-b323-38fa6eeaeddd-host\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-socket-dir-parent\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df7478cd-6084-4776-b323-38fa6eeaeddd-host\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-device-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-k8s-cni-cncf-io\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-d\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-lib-modules\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-socket-dir-parent\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-var-lib-kubelet\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-run-k8s-cni-cncf-io\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.521564 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-var-lib-kubelet\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-system-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-lib-modules\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-os-release\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-multus-daemon-config\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.520961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-system-cni-dir\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cni-binary-copy\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-bin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-conf\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cnibin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-os-release\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-cni-bin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-run\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cnibin\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-run\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-sysctl-conf\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.522554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-host\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-host\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-systemd\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-tmp\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4645\" (UniqueName: \"kubernetes.io/projected/a78dbafb-b423-4af6-8a40-0aebedf0bddb-kube-api-access-l4645\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2e3b46e-2cdd-48e3-8704-dfa897beccda-hosts-file\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-tuned\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-systemd\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-kubelet\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2e3b46e-2cdd-48e3-8704-dfa897beccda-hosts-file\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-cni-binary-copy\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eefea536-19ad-4de7-86c6-9e6a5d58bb17-konnectivity-ca\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-kubernetes\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-dbus\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/403c6a83-604e-4ebc-a605-ea919378a31b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87481eaf-e0ac-43d7-87be-b329439e1fe5-dbus\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-kubernetes\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.523526 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.521767    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-host-var-lib-kubelet\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.524313 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.522172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/eefea536-19ad-4de7-86c6-9e6a5d58bb17-konnectivity-ca\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.525004 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.524982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-tmp\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.525090 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.525071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/eefea536-19ad-4de7-86c6-9e6a5d58bb17-agent-certs\") pod \"konnectivity-agent-cfhvf\" (UID: \"eefea536-19ad-4de7-86c6-9e6a5d58bb17\") " pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:27.525615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.525590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a78dbafb-b423-4af6-8a40-0aebedf0bddb-etc-tuned\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.534610 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.534561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gd5p\" (UniqueName: \"kubernetes.io/projected/e2e3b46e-2cdd-48e3-8704-dfa897beccda-kube-api-access-2gd5p\") pod \"node-resolver-g2jdj\" (UID: \"e2e3b46e-2cdd-48e3-8704-dfa897beccda\") " pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.534736 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.534717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4f8p\" (UniqueName: \"kubernetes.io/projected/bcb66aad-89ed-482e-8d49-abefa4390ab9-kube-api-access-z4f8p\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:27.535721 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.535688    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:27.535721 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.535708    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:27.535721 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.535718    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:27.535932 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:27.535772    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:28.035755384 +0000 UTC m=+3.113742383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:27.536926 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.536901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgrs\" (UniqueName: \"kubernetes.io/projected/403c6a83-604e-4ebc-a605-ea919378a31b-kube-api-access-8sgrs\") pod \"aws-ebs-csi-driver-node-kb25q\" (UID: \"403c6a83-604e-4ebc-a605-ea919378a31b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.538791 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.538768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdb6z\" (UniqueName: \"kubernetes.io/projected/5f5148aa-8824-489b-b0c6-7fbd1c950c8b-kube-api-access-wdb6z\") pod \"multus-n5c74\" (UID: \"5f5148aa-8824-489b-b0c6-7fbd1c950c8b\") " pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.539264 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.539243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcxxf\" (UniqueName: \"kubernetes.io/projected/df7478cd-6084-4776-b323-38fa6eeaeddd-kube-api-access-hcxxf\") pod \"node-ca-kfvrh\" (UID: \"df7478cd-6084-4776-b323-38fa6eeaeddd\") " pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.539349 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.539255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4645\" (UniqueName: \"kubernetes.io/projected/a78dbafb-b423-4af6-8a40-0aebedf0bddb-kube-api-access-l4645\") pod \"tuned-nhhsf\" (UID: \"a78dbafb-b423-4af6-8a40-0aebedf0bddb\") " pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.618487 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.618456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4q7dd"
Jul 29 00:54:27.632168 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.632144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-db2dj"
Jul 29 00:54:27.647723 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.647660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:27.649777 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.649752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q"
Jul 29 00:54:27.657042 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.657022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nhhsf"
Jul 29 00:54:27.672080 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.672055    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g2jdj"
Jul 29 00:54:27.680927 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.680908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kfvrh"
Jul 29 00:54:27.689061 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.689035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n5c74"
Jul 29 00:54:27.695899 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:27.695878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:28.026997 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.026907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:28.026997 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.026964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:28.027240 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.027093    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:28.027240 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.027129    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:28.027240 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.027175    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:29.02715216 +0000 UTC m=+4.105139160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:28.027240 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.027199    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:29.027185876 +0000 UTC m=+4.105172867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:28.127894 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.127857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:28.128089 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.128058    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:28.128089 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.128085    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:28.128193 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.128101    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:28.128193 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.128169    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:29.128149547 +0000 UTC m=+4.206136525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:28.347909 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.347811    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:49:27 +0000 UTC" deadline="2028-05-04 18:12:46.427208742 +0000 UTC"
Jul 29 00:54:28.347909 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.347847    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15497h18m18.079365824s"
Jul 29 00:54:28.469471 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.469439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:28.469636 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:28.469445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:28.472121 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.470024    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:28.472121 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:28.470140    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:28.611163 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.611083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e3b46e_2cdd_48e3_8704_dfa897beccda.slice/crio-40dc8358576dd4508a992355ab701d247242b7a3d60262a8f9276468c51ef3f5 WatchSource:0}: Error finding container 40dc8358576dd4508a992355ab701d247242b7a3d60262a8f9276468c51ef3f5: Status 404 returned error can't find the container with id 40dc8358576dd4508a992355ab701d247242b7a3d60262a8f9276468c51ef3f5
Jul 29 00:54:28.613368 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.613277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9f36bc_1cf4_4e46_8333_315bf80cb050.slice/crio-fdd3c00ef2d0668aee4b9ff94f12aa72b1880aca39d4a0a7378be8eb91b2008a WatchSource:0}: Error finding container fdd3c00ef2d0668aee4b9ff94f12aa72b1880aca39d4a0a7378be8eb91b2008a: Status 404 returned error can't find the container with id fdd3c00ef2d0668aee4b9ff94f12aa72b1880aca39d4a0a7378be8eb91b2008a
Jul 29 00:54:28.617340 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.616960    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5148aa_8824_489b_b0c6_7fbd1c950c8b.slice/crio-a8bda1668373221aebe5b1aced733f4441f270b095939265ff56401872c6a3ba WatchSource:0}: Error finding container a8bda1668373221aebe5b1aced733f4441f270b095939265ff56401872c6a3ba: Status 404 returned error can't find the container with id a8bda1668373221aebe5b1aced733f4441f270b095939265ff56401872c6a3ba
Jul 29 00:54:28.618447 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.618359    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeefea536_19ad_4de7_86c6_9e6a5d58bb17.slice/crio-2d4f51fb4ed702952fb54b41776e432f2e91e2d7708e852c4b1db8a45f57c83f WatchSource:0}: Error finding container 2d4f51fb4ed702952fb54b41776e432f2e91e2d7708e852c4b1db8a45f57c83f: Status 404 returned error can't find the container with id 2d4f51fb4ed702952fb54b41776e432f2e91e2d7708e852c4b1db8a45f57c83f
Jul 29 00:54:28.619957 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.619933    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403c6a83_604e_4ebc_a605_ea919378a31b.slice/crio-a72862c9e8d6158ecfe3b5c365496130873cfa8534a405b44699d67029167478 WatchSource:0}: Error finding container a72862c9e8d6158ecfe3b5c365496130873cfa8534a405b44699d67029167478: Status 404 returned error can't find the container with id a72862c9e8d6158ecfe3b5c365496130873cfa8534a405b44699d67029167478
Jul 29 00:54:28.620930 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.620864    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda78dbafb_b423_4af6_8a40_0aebedf0bddb.slice/crio-33afa738391511e4f90ce1552b0ecd6cfc97fd4d34d4eeef90d910c08b8ccb8c WatchSource:0}: Error finding container 33afa738391511e4f90ce1552b0ecd6cfc97fd4d34d4eeef90d910c08b8ccb8c: Status 404 returned error can't find the container with id 33afa738391511e4f90ce1552b0ecd6cfc97fd4d34d4eeef90d910c08b8ccb8c
Jul 29 00:54:28.622051 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.621952    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7742ce84_5d8c_45f2_bbed_e2e0fd23b428.slice/crio-b8e198ca51a51543fa21e025f71b27b9a017bbbef1f61e3b479a6517fa8efcc3 WatchSource:0}: Error finding container b8e198ca51a51543fa21e025f71b27b9a017bbbef1f61e3b479a6517fa8efcc3: Status 404 returned error can't find the container with id b8e198ca51a51543fa21e025f71b27b9a017bbbef1f61e3b479a6517fa8efcc3
Jul 29 00:54:28.623057 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.623034    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf7478cd_6084_4776_b323_38fa6eeaeddd.slice/crio-0f3e6181edfdcf8cc437038c6c27b1ca47ea378e5040994bb69b49e5bf61e860 WatchSource:0}: Error finding container 0f3e6181edfdcf8cc437038c6c27b1ca47ea378e5040994bb69b49e5bf61e860: Status 404 returned error can't find the container with id 0f3e6181edfdcf8cc437038c6c27b1ca47ea378e5040994bb69b49e5bf61e860
Jul 29 00:54:28.624656 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:54:28.624530    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a89c0c7_585a_4d4f_906c_cf82e4baeba0.slice/crio-91cc69d9f7543f19fc59946398aea4e8412e9f0be42d924fbbe2bddd9551d539 WatchSource:0}: Error finding container 91cc69d9f7543f19fc59946398aea4e8412e9f0be42d924fbbe2bddd9551d539: Status 404 returned error can't find the container with id 91cc69d9f7543f19fc59946398aea4e8412e9f0be42d924fbbe2bddd9551d539
Jul 29 00:54:29.012023 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.011917    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:29.034864 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.034828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:29.034978 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.034869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:29.035027 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.034994    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:29.035064 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.035051    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:31.035036501 +0000 UTC m=+6.113023480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:29.035173 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.034997    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:29.035173 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.035143    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:31.035130455 +0000 UTC m=+6.113117435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:29.136600 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.135902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:29.136600 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.136133    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:29.136600 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.136154    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:29.136600 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.136168    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:29.136600 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.136232    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:31.136209317 +0000 UTC m=+6.214196311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:29.348139 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.348005    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:49:27 +0000 UTC" deadline="2028-03-10 18:40:36.109541393 +0000 UTC"
Jul 29 00:54:29.348139 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.348068    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14177h46m6.761477917s"
Jul 29 00:54:29.473578 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.473544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:29.473755 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:29.473693    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:29.511591 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.511520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g2jdj" event={"ID":"e2e3b46e-2cdd-48e3-8704-dfa897beccda","Type":"ContainerStarted","Data":"40dc8358576dd4508a992355ab701d247242b7a3d60262a8f9276468c51ef3f5"}
Jul 29 00:54:29.528175 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.528142    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f124187e52f9f4dd8797d0bf6271156" containerID="880cc2ee6972e67c10d9b589a0115e0809ae2e71ab07df7801d3c412c00cac5a" exitCode=0
Jul 29 00:54:29.528312 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.528288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal" event={"ID":"9f124187e52f9f4dd8797d0bf6271156","Type":"ContainerDied","Data":"880cc2ee6972e67c10d9b589a0115e0809ae2e71ab07df7801d3c412c00cac5a"}
Jul 29 00:54:29.546180 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.546148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q" event={"ID":"403c6a83-604e-4ebc-a605-ea919378a31b","Type":"ContainerStarted","Data":"a72862c9e8d6158ecfe3b5c365496130873cfa8534a405b44699d67029167478"}
Jul 29 00:54:29.567621 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.565938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfhvf" event={"ID":"eefea536-19ad-4de7-86c6-9e6a5d58bb17","Type":"ContainerStarted","Data":"2d4f51fb4ed702952fb54b41776e432f2e91e2d7708e852c4b1db8a45f57c83f"}
Jul 29 00:54:29.572856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.572824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal" event={"ID":"ac1805d9fb4faade8c3be0e92a054aab","Type":"ContainerStarted","Data":"499a098c6ba9230234f9c4e407cc78c4c5ee03699c5999cd115643e1da7f292c"}
Jul 29 00:54:29.582933 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.582863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-db2dj" event={"ID":"3a89c0c7-585a-4d4f-906c-cf82e4baeba0","Type":"ContainerStarted","Data":"91cc69d9f7543f19fc59946398aea4e8412e9f0be42d924fbbe2bddd9551d539"}
Jul 29 00:54:29.592512 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.592220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kfvrh" event={"ID":"df7478cd-6084-4776-b323-38fa6eeaeddd","Type":"ContainerStarted","Data":"0f3e6181edfdcf8cc437038c6c27b1ca47ea378e5040994bb69b49e5bf61e860"}
Jul 29 00:54:29.605761 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.605692    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"b8e198ca51a51543fa21e025f71b27b9a017bbbef1f61e3b479a6517fa8efcc3"}
Jul 29 00:54:29.620916 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.620829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nhhsf" event={"ID":"a78dbafb-b423-4af6-8a40-0aebedf0bddb","Type":"ContainerStarted","Data":"33afa738391511e4f90ce1552b0ecd6cfc97fd4d34d4eeef90d910c08b8ccb8c"}
Jul 29 00:54:29.632720 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.632586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerStarted","Data":"fdd3c00ef2d0668aee4b9ff94f12aa72b1880aca39d4a0a7378be8eb91b2008a"}
Jul 29 00:54:29.643965 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:29.643860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5c74" event={"ID":"5f5148aa-8824-489b-b0c6-7fbd1c950c8b","Type":"ContainerStarted","Data":"a8bda1668373221aebe5b1aced733f4441f270b095939265ff56401872c6a3ba"}
Jul 29 00:54:30.469930 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:30.469893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:30.470418 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:30.470045    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:30.470418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:30.470344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:30.470532 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:30.470438    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:30.658811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:30.658770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal" event={"ID":"9f124187e52f9f4dd8797d0bf6271156","Type":"ContainerStarted","Data":"a6f6992a32bdcc491cd93925e6a245c4d2afde83bea1ab603d268d0f48047150"}
Jul 29 00:54:30.674774 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:30.674047    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-224.ec2.internal" podStartSLOduration=4.674026043 podStartE2EDuration="4.674026043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:29.59037955 +0000 UTC m=+4.668366555" watchObservedRunningTime="2026-07-29 00:54:30.674026043 +0000 UTC m=+5.752013043"
Jul 29 00:54:31.051354 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:31.051310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:31.051544 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:31.051361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:31.051544 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.051500    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:31.051663 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.051560    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:35.051541286 +0000 UTC m=+10.129528281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:31.052042 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.051927    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:31.052042 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.051987    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:35.051962875 +0000 UTC m=+10.129949857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:31.152185 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:31.152148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:31.152687 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.152378    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:31.152687 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.152402    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:31.152687 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.152415    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:31.152687 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.152473    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:35.152456276 +0000 UTC m=+10.230443269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:31.469171 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:31.469070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:31.471498 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:31.471465    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:32.469189 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:32.469156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:32.469395 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:32.469316    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:32.469395 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:32.469380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:32.469501 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:32.469454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:33.470064 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:33.470034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:33.470439 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:33.470164    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:34.469590 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:34.469553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:34.469757 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:34.469704    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:34.469757 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:34.469721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:34.469876 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:34.469836    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:35.081440 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:35.081403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:35.081889 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:35.081447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:35.081889 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.081546    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:35.081889 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.081590    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:35.081889 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.081619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:43.081597995 +0000 UTC m=+18.159584995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:35.081889 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.081645    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:43.081629091 +0000 UTC m=+18.159616089 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:35.182312 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:35.182221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:35.182475 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.182393    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:35.182475 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.182412    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:35.182475 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.182424    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:35.182587 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.182479    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:43.182460975 +0000 UTC m=+18.260447958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:35.470348 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:35.470249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:35.470496 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:35.470454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:36.469643 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:36.469610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:36.470085 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:36.469756    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:36.470325 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:36.470154    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:36.470325 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:36.470278    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:37.469208 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:37.468963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:37.469208 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:37.469107    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:38.469016 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:38.468982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:38.469462 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:38.469122    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:38.469462 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:38.469167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:38.469462 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:38.469251    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:39.469103 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:39.469071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:39.469581 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:39.469185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:40.469436 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:40.469401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:40.469911 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:40.469450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:40.469911 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:40.469556    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:40.469911 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:40.469681    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:41.468955 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:41.468917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:41.469147 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:41.469054    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:42.469539 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:42.469499    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:42.470051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:42.469502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:42.470051 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:42.469622    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:42.470051 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:42.469717    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:43.148114 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:43.148086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:43.148292 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:43.148122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:43.148292 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.148233    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:43.148370 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.148294    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:59.148279568 +0000 UTC m=+34.226266545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:43.148370 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.148233    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:43.148370 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.148352    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:59.148337546 +0000 UTC m=+34.226324523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:43.248834 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:43.248799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:43.248977 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.248953    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:43.248977 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.248970    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:43.248977 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.248979    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:43.249138 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.249028    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:59.249011684 +0000 UTC m=+34.326998678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:43.469799 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:43.469716    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:43.470248 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:43.469856    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:44.469568 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:44.469539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:44.469813 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:44.469655    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:44.469813 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:44.469715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:44.470178 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:44.469829    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:45.470189 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:45.470157    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:45.470641 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:45.470272    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:46.469238 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:46.469201    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:46.469435 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:46.469213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:46.469435 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:46.469337    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:46.469435 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:46.469379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:47.470190 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.469884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:47.471125 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:47.470297    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:47.692729 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.692700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cfhvf" event={"ID":"eefea536-19ad-4de7-86c6-9e6a5d58bb17","Type":"ContainerStarted","Data":"d07565451cf6179bd65e0a566ab555f9400720441f99476c32dee5706c2b7e4b"}
Jul 29 00:54:47.694317 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.694251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kfvrh" event={"ID":"df7478cd-6084-4776-b323-38fa6eeaeddd","Type":"ContainerStarted","Data":"149d1f5332804e7ace3223312bee49d5d603e42f72f05137a76e6c172487c8d2"}
Jul 29 00:54:47.697254 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"c9d3924ca6575d075b43239c81993c534ff1d54dcf19ace551afa741bfe51758"}
Jul 29 00:54:47.697373 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"ef3808abc86782090e9a3ff2777efe169908de0a3356341ac39dca0d630b4745"}
Jul 29 00:54:47.697425 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"7ce7c88c232d14c1466943a28d77a52ab2b233b748309db6fc6060bf6050e5b2"}
Jul 29 00:54:47.697479 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"ffd8da4b18cecd66dd4187209ef71aa5e95779b1cab92e4469edfd8502cb7258"}
Jul 29 00:54:47.697535 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"2631cdb69245d1838035ec36a27244d784bb83ef739efbfb2d93fe1d1e87a121"}
Jul 29 00:54:47.697585 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.697535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"6555a9c1d00a40a24294b4a855e5a9969ab8ba59af5de6031e9e8a9595fcbf76"}
Jul 29 00:54:47.698764 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.698737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nhhsf" event={"ID":"a78dbafb-b423-4af6-8a40-0aebedf0bddb","Type":"ContainerStarted","Data":"b25a5d0248d90806585aaabe80b3577cbbe34ce09a8d0fe6efdfff7e9a897652"}
Jul 29 00:54:47.700331 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.700305    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="89342098fafe7bd732c9822c5ddb03dd7603f8443982b96fd198e6458a2bbd65" exitCode=0
Jul 29 00:54:47.700421 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.700373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"89342098fafe7bd732c9822c5ddb03dd7603f8443982b96fd198e6458a2bbd65"}
Jul 29 00:54:47.701884 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.701845    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n5c74" event={"ID":"5f5148aa-8824-489b-b0c6-7fbd1c950c8b","Type":"ContainerStarted","Data":"1aabbf4821d793794a5c2051ffed3d65bb682942c29f02ad86cea4d731f4fa14"}
Jul 29 00:54:47.703317 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.703294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g2jdj" event={"ID":"e2e3b46e-2cdd-48e3-8704-dfa897beccda","Type":"ContainerStarted","Data":"e24060f4af6421a79ec8856159ad649ec5e273c65ca2dd0e479f2a2c96f07dda"}
Jul 29 00:54:47.704629 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.704610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q" event={"ID":"403c6a83-604e-4ebc-a605-ea919378a31b","Type":"ContainerStarted","Data":"ba467debc235e5c774e17da825c81fb86c4d455b4633c06df2f3056f84d3aea7"}
Jul 29 00:54:47.707790 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.707749    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cfhvf" podStartSLOduration=4.237681966 podStartE2EDuration="22.707737228s" totalImagesPullingTime="18.470055262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:47.707396834 +0000 UTC m=+22.785383835" watchObservedRunningTime="2026-07-29 00:54:47.707737228 +0000 UTC m=+22.785724232"
Jul 29 00:54:47.707975 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.707956    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-224.ec2.internal" podStartSLOduration=21.707952089 podStartE2EDuration="21.707952089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:30.674757733 +0000 UTC m=+5.752744734" watchObservedRunningTime="2026-07-29 00:54:47.707952089 +0000 UTC m=+22.785939087"
Jul 29 00:54:47.737467 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.737429    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g2jdj" podStartSLOduration=4.256777817 podStartE2EDuration="22.737419253s" totalImagesPullingTime="18.480641436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:47.737012057 +0000 UTC m=+22.814999055" watchObservedRunningTime="2026-07-29 00:54:47.737419253 +0000 UTC m=+22.815406251"
Jul 29 00:54:47.753877 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.753840    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n5c74" podStartSLOduration=4.222116842 podStartE2EDuration="22.753827756s" totalImagesPullingTime="18.531710914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:47.753306891 +0000 UTC m=+22.831293889" watchObservedRunningTime="2026-07-29 00:54:47.753827756 +0000 UTC m=+22.831814754"
Jul 29 00:54:47.772807 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.772762    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nhhsf" podStartSLOduration=4.304944316 podStartE2EDuration="22.772753967s" totalImagesPullingTime="18.467809651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:47.772361656 +0000 UTC m=+22.850348656" watchObservedRunningTime="2026-07-29 00:54:47.772753967 +0000 UTC m=+22.850740965"
Jul 29 00:54:47.786067 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.786019    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kfvrh" podStartSLOduration=4.319011447 podStartE2EDuration="22.786006927s" totalImagesPullingTime="18.46699548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:47.785939105 +0000 UTC m=+22.863926109" watchObservedRunningTime="2026-07-29 00:54:47.786006927 +0000 UTC m=+22.863993925"
Jul 29 00:54:47.895005 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.894934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:47.943886 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:47.943838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:48.469657 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.469618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:48.469987 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.469624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:48.469987 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:48.469749    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:48.469987 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:48.469790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:48.708452 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.708419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-db2dj" event={"ID":"3a89c0c7-585a-4d4f-906c-cf82e4baeba0","Type":"ContainerStarted","Data":"ffe9124bf105e5aad8e4a71362bdd43c6e66b960f400246512f2ed96196db2f4"}
Jul 29 00:54:48.739503 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.739454    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-db2dj" podStartSLOduration=5.2749246450000005 podStartE2EDuration="23.739432388s" totalImagesPullingTime="18.464507743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:48.738884191 +0000 UTC m=+23.816871193" watchObservedRunningTime="2026-07-29 00:54:48.739432388 +0000 UTC m=+23.817419388"
Jul 29 00:54:48.855619 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.855543    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 00:54:48.856206 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.856176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:48.956341 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:48.956308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:49.373239 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.373208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:49.382619 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.382512    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T00:54:48.855614215Z","UUID":"e297652b-6134-4094-a077-e3b495f5b11f","Handler":null,"Name":"","Endpoint":""}
Jul 29 00:54:49.384289 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.384270    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 00:54:49.384410 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.384302    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 00:54:49.470014 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.469982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:49.470190 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:49.470106    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:49.473605 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.473556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:49.713031 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.712917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q" event={"ID":"403c6a83-604e-4ebc-a605-ea919378a31b","Type":"ContainerStarted","Data":"0d75237174325e3c5db1fc96e728e02410737d69411c47eaeb4c8d5d066fb5ca"}
Jul 29 00:54:49.793815 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.793781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:49.894546 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:49.894518    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:50.469434 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:50.469401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:50.469705 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:50.469418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:50.469705 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:50.469546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:50.469705 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:50.469624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:50.719722 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:50.719640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"421a75789f4474872e5c6a0bda030c8902f5c6b26281dc17561c9d33e9f060c7"}
Jul 29 00:54:51.469077 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.469048    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:51.469241 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:51.469164    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:51.562087 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.562057    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:51.562768 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.562745    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:51.728761 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.728609    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:51.729067 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.729051    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cfhvf"
Jul 29 00:54:51.790791 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:51.790766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:52.469390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.469171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:52.469518 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.469171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:52.469518 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:52.469419    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:52.469518 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:52.469492    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:52.727031 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.726950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" event={"ID":"7742ce84-5d8c-45f2-bbed-e2e0fd23b428","Type":"ContainerStarted","Data":"2d4c6514d399585a95bee042e6694d4dc09cc35a73b971421c7cf2df715037dd"}
Jul 29 00:54:52.727234 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.727209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:52.727331 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.727301    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:52.728862 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.728838    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="bfbea53e836647f0a0ba8cc37ea4b98f133ddb6fc41327934ed6472a8820a22a" exitCode=0
Jul 29 00:54:52.729588 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.728864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"bfbea53e836647f0a0ba8cc37ea4b98f133ddb6fc41327934ed6472a8820a22a"}
Jul 29 00:54:52.730976 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.730956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q" event={"ID":"403c6a83-604e-4ebc-a605-ea919378a31b","Type":"ContainerStarted","Data":"8a78dc34333a4b77431b278d181d24e6716e7a24488e1b932a2f1be1b3e567b0"}
Jul 29 00:54:52.743726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.743707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:52.753023 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.752991    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp" podStartSLOduration=9.215482396 podStartE2EDuration="27.752978945s" totalImagesPullingTime="18.537496549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:52.752849346 +0000 UTC m=+27.830836344" watchObservedRunningTime="2026-07-29 00:54:52.752978945 +0000 UTC m=+27.830965935"
Jul 29 00:54:52.817267 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:52.817227    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb25q" podStartSLOduration=5.0997251200000004 podStartE2EDuration="27.817215234s" totalImagesPullingTime="22.717490114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:52.816956248 +0000 UTC m=+27.894943240" watchObservedRunningTime="2026-07-29 00:54:52.817215234 +0000 UTC m=+27.895202229"
Jul 29 00:54:53.469287 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:53.469073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:53.469477 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:53.469383    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:53.736938 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:53.736823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:53.751730 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:53.751705    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:54:54.045502 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.045436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7fr2p"]
Jul 29 00:54:54.045615 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.045563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:54.045693 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:54.045673    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:54.048660 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.048634    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-564xp"]
Jul 29 00:54:54.048759 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.048741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:54.048855 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:54.048833    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:54.049376 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.049356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckvff"]
Jul 29 00:54:54.049477 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.049438    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:54.049538 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:54.049520    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:54.737058 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.737025    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="d0e8a2e19c5f56d60d5bcfeca8ef0a0608257fc1bddb7fbbd408ef768316d482" exitCode=0
Jul 29 00:54:54.737495 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:54.737068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"d0e8a2e19c5f56d60d5bcfeca8ef0a0608257fc1bddb7fbbd408ef768316d482"}
Jul 29 00:54:55.470392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:55.470363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:55.470561 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:55.470485    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:55.470561 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:55.470551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:55.470651 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:55.470622    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:55.470771 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:55.470751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:55.470891 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:55.470857    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:56.743756 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:56.743581    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="0b8b219f807da55d195c3917c71834ee14f9d6873d3c349c087e43e91329fb55" exitCode=0
Jul 29 00:54:56.744075 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:56.743655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"0b8b219f807da55d195c3917c71834ee14f9d6873d3c349c087e43e91329fb55"}
Jul 29 00:54:57.468966 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:57.468938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:57.469125 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:57.468939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:57.469125 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:57.469053    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:57.469125 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:57.469079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:57.469219 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:57.469144    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:57.469253 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:57.469224    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:54:59.176770 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.176692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:59.176770 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.176726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:59.177224 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.176813    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:59.177224 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.176830    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:59.177224 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.176873    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret podName:87481eaf-e0ac-43d7-87be-b329439e1fe5 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:31.176852512 +0000 UTC m=+66.254839494 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret") pod "global-pull-secret-syncer-7fr2p" (UID: "87481eaf-e0ac-43d7-87be-b329439e1fe5") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:59.177224 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.176892    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:31.176882017 +0000 UTC m=+66.254868994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:59.278037 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.278008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:59.278192 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.278113    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:59.278192 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.278127    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:59.278192 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.278136    2563 projected.go:196] Error preparing data for projected volume kube-api-access-mc9dc for pod openshift-network-diagnostics/network-check-target-564xp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:59.278192 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.278181    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc podName:6fb25797-27d8-46d5-ba7e-b5e91c81b0f1 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:31.278168101 +0000 UTC m=+66.356155078 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mc9dc" (UniqueName: "kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc") pod "network-check-target-564xp" (UID: "6fb25797-27d8-46d5-ba7e-b5e91c81b0f1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:59.469437 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.469352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:54:59.469595 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.469464    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7fr2p" podUID="87481eaf-e0ac-43d7-87be-b329439e1fe5"
Jul 29 00:54:59.469595 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.469352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:54:59.469595 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:54:59.469513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:54:59.469742 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.469640    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:54:59.469742 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:54:59.469722    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-564xp" podUID="6fb25797-27d8-46d5-ba7e-b5e91c81b0f1"
Jul 29 00:55:00.768979 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.768952    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-224.ec2.internal" event="NodeReady"
Jul 29 00:55:00.769601 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.769103    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 00:55:00.811078 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.811040    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-89gsp"]
Jul 29 00:55:00.813601 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.813580    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s269z"]
Jul 29 00:55:00.813732 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.813708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:00.815326 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.815309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:00.816492 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.816474    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 00:55:00.816609 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.816516    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 00:55:00.816718 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.816629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9mcp8\""
Jul 29 00:55:00.817820 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.817766    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-58969\""
Jul 29 00:55:00.817820 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.817780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 00:55:00.818056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.818040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 00:55:00.818116 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.818061    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 00:55:00.824472 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.824444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89gsp"]
Jul 29 00:55:00.827219 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.827172    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s269z"]
Jul 29 00:55:00.988765 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:00.988765 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gpkk\" (UniqueName: \"kubernetes.io/projected/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-kube-api-access-6gpkk\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:00.988765 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd86z\" (UniqueName: \"kubernetes.io/projected/057a7481-05af-465c-acad-ff70ddbef93f-kube-api-access-wd86z\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:00.989011 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/057a7481-05af-465c-acad-ff70ddbef93f-config-volume\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:00.989011 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057a7481-05af-465c-acad-ff70ddbef93f-tmp-dir\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:00.989011 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:00.988863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-cert\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.089162 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.089270 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gpkk\" (UniqueName: \"kubernetes.io/projected/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-kube-api-access-6gpkk\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.089270 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd86z\" (UniqueName: \"kubernetes.io/projected/057a7481-05af-465c-acad-ff70ddbef93f-kube-api-access-wd86z\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.089392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/057a7481-05af-465c-acad-ff70ddbef93f-config-volume\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.089392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057a7481-05af-465c-acad-ff70ddbef93f-tmp-dir\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.089392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-cert\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.089531 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:01.089308    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:01.089531 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:01.089509    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:01.58948539 +0000 UTC m=+36.667472373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:01.089845 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.089822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057a7481-05af-465c-acad-ff70ddbef93f-tmp-dir\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.090235 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.090191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/057a7481-05af-465c-acad-ff70ddbef93f-config-volume\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.094033 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.094015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-cert\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.098488 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.098462    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd86z\" (UniqueName: \"kubernetes.io/projected/057a7481-05af-465c-acad-ff70ddbef93f-kube-api-access-wd86z\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.098594 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.098573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gpkk\" (UniqueName: \"kubernetes.io/projected/8d76dbfe-b355-43ae-a6ff-bf58ffae18c4-kube-api-access-6gpkk\") pod \"ingress-canary-s269z\" (UID: \"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4\") " pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.132947 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.132919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s269z"
Jul 29 00:55:01.264739 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.264514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s269z"]
Jul 29 00:55:01.286030 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:55:01.286006    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d76dbfe_b355_43ae_a6ff_bf58ffae18c4.slice/crio-b3a9833b8aec53626e82ba25ec1c3684d0e1f8835889f7ee02fdf2391df76a6b WatchSource:0}: Error finding container b3a9833b8aec53626e82ba25ec1c3684d0e1f8835889f7ee02fdf2391df76a6b: Status 404 returned error can't find the container with id b3a9833b8aec53626e82ba25ec1c3684d0e1f8835889f7ee02fdf2391df76a6b
Jul 29 00:55:01.468920 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.468875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:55:01.468920 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.468910    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:55:01.469133 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.468915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:55:01.472128 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.472107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:55:01.472289 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.472138    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:55:01.472289 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.472138    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:55:01.472289 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.472250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxdbf\""
Jul 29 00:55:01.473815 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.473575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:55:01.473946 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.473817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ndg5h\""
Jul 29 00:55:01.594100 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.594041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:01.594189 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:01.594130    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:01.594189 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:01.594177    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:02.594164288 +0000 UTC m=+37.672151267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:01.758435 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.758399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s269z" event={"ID":"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4","Type":"ContainerStarted","Data":"b3a9833b8aec53626e82ba25ec1c3684d0e1f8835889f7ee02fdf2391df76a6b"}
Jul 29 00:55:01.760881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.760859    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="c2ac149b77b2093da0bbe1077c438e06662c309b5662fc25ecace184b12a558d" exitCode=0
Jul 29 00:55:01.760969 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:01.760887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"c2ac149b77b2093da0bbe1077c438e06662c309b5662fc25ecace184b12a558d"}
Jul 29 00:55:02.601923 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:02.601873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:02.602476 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:02.602032    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:02.602476 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:02.602105    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:04.60208336 +0000 UTC m=+39.680070341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:02.765800 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:02.765763    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d9f36bc-1cf4-4e46-8333-315bf80cb050" containerID="95a622e2943100be8609541c8473ee481e1da88722d61e03f8d467598568cf4c" exitCode=0
Jul 29 00:55:02.765961 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:02.765818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerDied","Data":"95a622e2943100be8609541c8473ee481e1da88722d61e03f8d467598568cf4c"}
Jul 29 00:55:03.771802 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:03.771772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" event={"ID":"1d9f36bc-1cf4-4e46-8333-315bf80cb050","Type":"ContainerStarted","Data":"11624746c29961175c5a334b3c509ecdb622cbdaacddbecb4cf62e83411091b3"}
Jul 29 00:55:03.795234 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:03.795186    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4q7dd" podStartSLOduration=9.539037002 podStartE2EDuration="38.795166617s" totalImagesPullingTime="29.256129615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:03.793512625 +0000 UTC m=+38.871499830" watchObservedRunningTime="2026-07-29 00:55:03.795166617 +0000 UTC m=+38.873153619"
Jul 29 00:55:04.619106 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:04.618928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:04.619245 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:04.619078    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:04.619245 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:04.619216    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:08.61919994 +0000 UTC m=+43.697186917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:04.775191 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:04.775086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s269z" event={"ID":"8d76dbfe-b355-43ae-a6ff-bf58ffae18c4","Type":"ContainerStarted","Data":"2813a376e7ad145d635d2d588835ff727f54e12378a2ebc97fab1fe505dbadea"}
Jul 29 00:55:04.789919 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:04.789869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s269z" podStartSLOduration=1.698497642 podStartE2EDuration="4.789855488s" totalImagesPullingTime="3.091357846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:04.789054962 +0000 UTC m=+39.867041958" watchObservedRunningTime="2026-07-29 00:55:04.789855488 +0000 UTC m=+39.867842501"
Jul 29 00:55:08.645577 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:08.645540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:08.645961 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:08.645647    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:08.645961 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:08.645697    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:16.645683475 +0000 UTC m=+51.723670451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:16.701043 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:16.700896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:16.701043 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:16.700968    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:16.701043 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:16.701042    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:55:32.701020086 +0000 UTC m=+67.779007079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:25.756320 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:25.756286    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f4sgp"
Jul 29 00:55:31.196876 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.196818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:55:31.196876 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.196873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:55:31.199634 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.199610    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:55:31.199779 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.199758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:55:31.207904 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:31.207888    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:55:31.207954 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:31.207948    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:56:35.20793137 +0000 UTC m=+130.285918347 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : secret "metrics-daemon-secret" not found
Jul 29 00:55:31.211492 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.211463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87481eaf-e0ac-43d7-87be-b329439e1fe5-original-pull-secret\") pod \"global-pull-secret-syncer-7fr2p\" (UID: \"87481eaf-e0ac-43d7-87be-b329439e1fe5\") " pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:55:31.298050 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.298015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:55:31.301011 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.300995    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:55:31.311800 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.311780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:55:31.322040 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.322017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc9dc\" (UniqueName: \"kubernetes.io/projected/6fb25797-27d8-46d5-ba7e-b5e91c81b0f1-kube-api-access-mc9dc\") pod \"network-check-target-564xp\" (UID: \"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1\") " pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:55:31.485224 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.485155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ndg5h\""
Jul 29 00:55:31.487215 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.487198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7fr2p"
Jul 29 00:55:31.493220 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.493202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:55:31.613279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.613235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7fr2p"]
Jul 29 00:55:31.616129 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:55:31.616105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87481eaf_e0ac_43d7_87be_b329439e1fe5.slice/crio-2043cda885b7a04db1c51f6c970afcacbce66c38bce1bf0447d37994273cf306 WatchSource:0}: Error finding container 2043cda885b7a04db1c51f6c970afcacbce66c38bce1bf0447d37994273cf306: Status 404 returned error can't find the container with id 2043cda885b7a04db1c51f6c970afcacbce66c38bce1bf0447d37994273cf306
Jul 29 00:55:31.624168 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.624143    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-564xp"]
Jul 29 00:55:31.626850 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:55:31.626830    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb25797_27d8_46d5_ba7e_b5e91c81b0f1.slice/crio-88fd288d80443a5b5649fe71b72c037901df619eb4fc0efb738b97eab7edd031 WatchSource:0}: Error finding container 88fd288d80443a5b5649fe71b72c037901df619eb4fc0efb738b97eab7edd031: Status 404 returned error can't find the container with id 88fd288d80443a5b5649fe71b72c037901df619eb4fc0efb738b97eab7edd031
Jul 29 00:55:31.836731 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.836703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-564xp" event={"ID":"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1","Type":"ContainerStarted","Data":"88fd288d80443a5b5649fe71b72c037901df619eb4fc0efb738b97eab7edd031"}
Jul 29 00:55:31.837650 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:31.837621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7fr2p" event={"ID":"87481eaf-e0ac-43d7-87be-b329439e1fe5","Type":"ContainerStarted","Data":"2043cda885b7a04db1c51f6c970afcacbce66c38bce1bf0447d37994273cf306"}
Jul 29 00:55:32.709309 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:32.709271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:55:32.709760 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:32.709462    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:32.709760 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:55:32.709545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:56:04.709521846 +0000 UTC m=+99.787508839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:36.852372 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:36.852324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-564xp" event={"ID":"6fb25797-27d8-46d5-ba7e-b5e91c81b0f1","Type":"ContainerStarted","Data":"bd0a0eb58a76e23fbab8226d5345f486c042994ea8c03d8efb4a1638ec3c2b00"}
Jul 29 00:55:36.852881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:36.852444    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:55:36.853672 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:36.853648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7fr2p" event={"ID":"87481eaf-e0ac-43d7-87be-b329439e1fe5","Type":"ContainerStarted","Data":"653a60f2ee30620359d055f69590e81e9bcb0d0b1285940629eb95da4ed3c55e"}
Jul 29 00:55:36.883604 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:36.883565    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-564xp" podStartSLOduration=67.263978704 podStartE2EDuration="1m11.883552796s" totalImagesPullingTime="4.619574092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:36.868401035 +0000 UTC m=+71.946388034" watchObservedRunningTime="2026-07-29 00:55:36.883552796 +0000 UTC m=+71.961539791"
Jul 29 00:55:36.883851 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:55:36.883826    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7fr2p" podStartSLOduration=67.240857841 podStartE2EDuration="1m11.883817229s" totalImagesPullingTime="4.642959388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:36.883502945 +0000 UTC m=+71.961489945" watchObservedRunningTime="2026-07-29 00:55:36.883817229 +0000 UTC m=+71.961804228"
Jul 29 00:56:02.016527 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.016411    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"]
Jul 29 00:56:02.019062 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.019045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.022910 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.022886    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 00:56:02.023019 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.022908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ghldv\""
Jul 29 00:56:02.023019 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.022908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 00:56:02.023019 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.022949    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 00:56:02.023019 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.022908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 00:56:02.028972 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.028954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"]
Jul 29 00:56:02.051191 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.051172    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"]
Jul 29 00:56:02.053111 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.053098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.055469 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.055451    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 29 00:56:02.063602 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.063584    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"]
Jul 29 00:56:02.100441 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.100416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbbq\" (UniqueName: \"kubernetes.io/projected/0b0ea4fe-c106-4aba-b601-c3165398758e-kube-api-access-xhbbq\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.100534 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.100460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b0ea4fe-c106-4aba-b601-c3165398758e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.201186 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.201162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29e39bed-17c8-4870-a47c-bf74c191dfff-tmp\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.201281 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.201205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/29e39bed-17c8-4870-a47c-bf74c191dfff-klusterlet-config\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.201281 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.201255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b0ea4fe-c106-4aba-b601-c3165398758e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.201348 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.201325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwxrq\" (UniqueName: \"kubernetes.io/projected/29e39bed-17c8-4870-a47c-bf74c191dfff-kube-api-access-wwxrq\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.201384 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.201353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbbq\" (UniqueName: \"kubernetes.io/projected/0b0ea4fe-c106-4aba-b601-c3165398758e-kube-api-access-xhbbq\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.203746 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.203719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0b0ea4fe-c106-4aba-b601-c3165398758e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.209689 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.209660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbbq\" (UniqueName: \"kubernetes.io/projected/0b0ea4fe-c106-4aba-b601-c3165398758e-kube-api-access-xhbbq\") pod \"managed-serviceaccount-addon-agent-595588b8b6-hbwj2\" (UID: \"0b0ea4fe-c106-4aba-b601-c3165398758e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.301915 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.301860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwxrq\" (UniqueName: \"kubernetes.io/projected/29e39bed-17c8-4870-a47c-bf74c191dfff-kube-api-access-wwxrq\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.301915 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.301912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29e39bed-17c8-4870-a47c-bf74c191dfff-tmp\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.302057 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.301953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/29e39bed-17c8-4870-a47c-bf74c191dfff-klusterlet-config\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.302308 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.302248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29e39bed-17c8-4870-a47c-bf74c191dfff-tmp\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.304064 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.304047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/29e39bed-17c8-4870-a47c-bf74c191dfff-klusterlet-config\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.309958 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.309929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwxrq\" (UniqueName: \"kubernetes.io/projected/29e39bed-17c8-4870-a47c-bf74c191dfff-kube-api-access-wwxrq\") pod \"klusterlet-addon-workmgr-7449f774c4-hsczs\" (UID: \"29e39bed-17c8-4870-a47c-bf74c191dfff\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.340432 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.340409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"
Jul 29 00:56:02.361708 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.361655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:02.466456 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.466427    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2"]
Jul 29 00:56:02.469968 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:56:02.469927    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b0ea4fe_c106_4aba_b601_c3165398758e.slice/crio-30c768f1312b3429d3d92a541b5aaa0dc240cc9d6293cf62ad74cb21187baaad WatchSource:0}: Error finding container 30c768f1312b3429d3d92a541b5aaa0dc240cc9d6293cf62ad74cb21187baaad: Status 404 returned error can't find the container with id 30c768f1312b3429d3d92a541b5aaa0dc240cc9d6293cf62ad74cb21187baaad
Jul 29 00:56:02.488106 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.488086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"]
Jul 29 00:56:02.490915 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:56:02.490891    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e39bed_17c8_4870_a47c_bf74c191dfff.slice/crio-1a288389f37ffdd2bdd292883bf9e307cc83e40fc99a3e67c0ccbd572dc819b6 WatchSource:0}: Error finding container 1a288389f37ffdd2bdd292883bf9e307cc83e40fc99a3e67c0ccbd572dc819b6: Status 404 returned error can't find the container with id 1a288389f37ffdd2bdd292883bf9e307cc83e40fc99a3e67c0ccbd572dc819b6
Jul 29 00:56:02.917387 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.917354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" event={"ID":"29e39bed-17c8-4870-a47c-bf74c191dfff","Type":"ContainerStarted","Data":"1a288389f37ffdd2bdd292883bf9e307cc83e40fc99a3e67c0ccbd572dc819b6"}
Jul 29 00:56:02.918276 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:02.918239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2" event={"ID":"0b0ea4fe-c106-4aba-b601-c3165398758e","Type":"ContainerStarted","Data":"30c768f1312b3429d3d92a541b5aaa0dc240cc9d6293cf62ad74cb21187baaad"}
Jul 29 00:56:04.719901 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:04.719870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:56:04.720306 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:56:04.720027    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:56:04.720306 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:56:04.720085    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:57:08.720069344 +0000 UTC m=+163.798056321 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:56:07.858115 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:07.858036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-564xp"
Jul 29 00:56:07.932038 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:07.932007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2" event={"ID":"0b0ea4fe-c106-4aba-b601-c3165398758e","Type":"ContainerStarted","Data":"e0dc32771e27b8f731562361bd3a94a92115f91848d74295167c7bb64872ca06"}
Jul 29 00:56:07.947915 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:07.947869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2" podStartSLOduration=0.827577422 podStartE2EDuration="5.947850423s" totalImagesPullingTime="5.120273001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:56:07.946743595 +0000 UTC m=+103.024730594" watchObservedRunningTime="2026-07-29 00:56:07.947850423 +0000 UTC m=+103.025837422"
Jul 29 00:56:15.950822 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:15.950788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" event={"ID":"29e39bed-17c8-4870-a47c-bf74c191dfff","Type":"ContainerStarted","Data":"d35cf91f402ed37163a748374dd1cd52fa868f3911371e314b014f4dbd99d23e"}
Jul 29 00:56:15.951224 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:15.951004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:15.952450 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:15.952432    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:56:15.968880 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:15.968843    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" podStartSLOduration=0.951489719 podStartE2EDuration="13.968831318s" totalImagesPullingTime="13.017341599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:56:15.967095496 +0000 UTC m=+111.045082515" watchObservedRunningTime="2026-07-29 00:56:15.968831318 +0000 UTC m=+111.046818318"
Jul 29 00:56:35.217690 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:56:35.217642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:56:35.218190 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:56:35.217771    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:56:35.218190 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:56:35.217841    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.217823653 +0000 UTC m=+252.295810650 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : secret "metrics-daemon-secret" not found
Jul 29 00:57:03.825836 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:57:03.825788    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-89gsp" podUID="057a7481-05af-465c-acad-ff70ddbef93f"
Jul 29 00:57:04.062671 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:57:04.062594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:57:04.492557 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:57:04.492510    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:57:08.733633 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:57:08.733597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:57:08.734127 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:57:08.733713    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:57:08.734127 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:57:08.733770    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls podName:057a7481-05af-465c-acad-ff70ddbef93f nodeName:}" failed. No retries permitted until 2026-07-29 00:59:10.733752063 +0000 UTC m=+285.811739048 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls") pod "dns-default-89gsp" (UID: "057a7481-05af-465c-acad-ff70ddbef93f") : secret "dns-default-metrics-tls" not found
Jul 29 00:57:17.472795 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:57:17.472722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:58:08.213450 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:08.213415    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b0ea4fe-c106-4aba-b601-c3165398758e" containerID="e0dc32771e27b8f731562361bd3a94a92115f91848d74295167c7bb64872ca06" exitCode=255
Jul 29 00:58:08.213841 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:08.213486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2" event={"ID":"0b0ea4fe-c106-4aba-b601-c3165398758e","Type":"ContainerDied","Data":"e0dc32771e27b8f731562361bd3a94a92115f91848d74295167c7bb64872ca06"}
Jul 29 00:58:08.219082 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:08.219066    2563 scope.go:117] "RemoveContainer" containerID="e0dc32771e27b8f731562361bd3a94a92115f91848d74295167c7bb64872ca06"
Jul 29 00:58:09.217856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:09.217821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595588b8b6-hbwj2" event={"ID":"0b0ea4fe-c106-4aba-b601-c3165398758e","Type":"ContainerStarted","Data":"5026c8bed70bf47cdb1386d3692221ad6ad8ea912903985b04b42bb818083bba"}
Jul 29 00:58:15.952358 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:15.952243    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" podUID="29e39bed-17c8-4870-a47c-bf74c191dfff" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused"
Jul 29 00:58:16.239485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:16.239412    2563 generic.go:356] "Generic (PLEG): container finished" podID="29e39bed-17c8-4870-a47c-bf74c191dfff" containerID="d35cf91f402ed37163a748374dd1cd52fa868f3911371e314b014f4dbd99d23e" exitCode=1
Jul 29 00:58:16.239485 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:16.239465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" event={"ID":"29e39bed-17c8-4870-a47c-bf74c191dfff","Type":"ContainerDied","Data":"d35cf91f402ed37163a748374dd1cd52fa868f3911371e314b014f4dbd99d23e"}
Jul 29 00:58:16.239723 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:16.239711    2563 scope.go:117] "RemoveContainer" containerID="d35cf91f402ed37163a748374dd1cd52fa868f3911371e314b014f4dbd99d23e"
Jul 29 00:58:17.243912 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:17.243879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs" event={"ID":"29e39bed-17c8-4870-a47c-bf74c191dfff","Type":"ContainerStarted","Data":"786862aa38c63a332bb5e3d333143eb8d14b8a61b93456ec8fccf2cd972b7447"}
Jul 29 00:58:17.244335 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:17.244175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:58:17.244822 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:17.244799    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7449f774c4-hsczs"
Jul 29 00:58:29.421735 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.421704    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:58:29.424517 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.424501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.427106 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.427085    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 00:58:29.427475 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.427459    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 00:58:29.427579 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.427483    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6dlx2\""
Jul 29 00:58:29.428509 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.428492    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 00:58:29.433451 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.433424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 00:58:29.440420 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.440403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:58:29.521051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.521027    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ql5tv"]
Jul 29 00:58:29.523770 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.523756    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.526551 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.526533    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 00:58:29.526637 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.526612    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 00:58:29.526810 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.526794    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 00:58:29.527481 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.527468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-d5l2t\""
Jul 29 00:58:29.527711 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.527696    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 00:58:29.534428 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.534409    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 00:58:29.536444 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.536421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ql5tv"]
Jul 29 00:58:29.575448 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575540 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575540 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575540 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbrmw\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575540 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575690 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575690 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.575690 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.575654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676305 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676305 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5lvs\" (UniqueName: \"kubernetes.io/projected/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-kube-api-access-x5lvs\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.676431 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676431 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-snapshots\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.676431 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676431 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-tmp\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.676584 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676584 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.676584 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676584 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:29.676624    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:29.676638    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbc88dcf7-fjgnl: secret "image-registry-tls" not found
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrmw\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:29.676687    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls podName:7d5e68cb-8c2d-4ca7-a142-3a14111e9e32 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:30.176667596 +0000 UTC m=+245.254654581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls") pod "image-registry-7cbc88dcf7-fjgnl" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32") : secret "image-registry-tls" not found
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676689    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-serving-cert\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.676848 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-service-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.677094 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.676861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.677274 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.677240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.677506 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.677484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.678856 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.678833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.678942 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.678833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.684623 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.684603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.684726 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.684682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrmw\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:29.777723 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-snapshots\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.777838 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-tmp\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.777838 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.777952 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-serving-cert\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.777952 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-service-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.777952 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.777931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5lvs\" (UniqueName: \"kubernetes.io/projected/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-kube-api-access-x5lvs\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.778274 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.778242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-tmp\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.778365 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.778325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-snapshots\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.778464 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.778445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-service-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.778621 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.778600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.779914 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.779892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-serving-cert\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.785918 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.785897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5lvs\" (UniqueName: \"kubernetes.io/projected/6add1afe-4f0f-4c2e-b2e5-3ce07696f232-kube-api-access-x5lvs\") pod \"insights-operator-5885fdb759-ql5tv\" (UID: \"6add1afe-4f0f-4c2e-b2e5-3ce07696f232\") " pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.832398 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.832378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-ql5tv"
Jul 29 00:58:29.946869 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:29.946809    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-ql5tv"]
Jul 29 00:58:29.949673 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:58:29.949647    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6add1afe_4f0f_4c2e_b2e5_3ce07696f232.slice/crio-1d46891503cd6f3dd4b516b0b5dbac8b2c82ffa4103a06811af1d254d0bebd02 WatchSource:0}: Error finding container 1d46891503cd6f3dd4b516b0b5dbac8b2c82ffa4103a06811af1d254d0bebd02: Status 404 returned error can't find the container with id 1d46891503cd6f3dd4b516b0b5dbac8b2c82ffa4103a06811af1d254d0bebd02
Jul 29 00:58:30.181693 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:30.181649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:30.181868 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:30.181772    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:58:30.181868 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:30.181783    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbc88dcf7-fjgnl: secret "image-registry-tls" not found
Jul 29 00:58:30.181868 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:30.181841    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls podName:7d5e68cb-8c2d-4ca7-a142-3a14111e9e32 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:31.181825659 +0000 UTC m=+246.259812636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls") pod "image-registry-7cbc88dcf7-fjgnl" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32") : secret "image-registry-tls" not found
Jul 29 00:58:30.281446 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:30.281364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ql5tv" event={"ID":"6add1afe-4f0f-4c2e-b2e5-3ce07696f232","Type":"ContainerStarted","Data":"1d46891503cd6f3dd4b516b0b5dbac8b2c82ffa4103a06811af1d254d0bebd02"}
Jul 29 00:58:31.189068 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:31.189032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:31.189446 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:31.189175    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:58:31.189446 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:31.189195    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbc88dcf7-fjgnl: secret "image-registry-tls" not found
Jul 29 00:58:31.189446 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:31.189245    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls podName:7d5e68cb-8c2d-4ca7-a142-3a14111e9e32 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:33.189230482 +0000 UTC m=+248.267217460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls") pod "image-registry-7cbc88dcf7-fjgnl" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32") : secret "image-registry-tls" not found
Jul 29 00:58:33.205567 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.205525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:33.205924 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.205670    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:58:33.205924 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.205689    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbc88dcf7-fjgnl: secret "image-registry-tls" not found
Jul 29 00:58:33.205924 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.205742    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls podName:7d5e68cb-8c2d-4ca7-a142-3a14111e9e32 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.205727409 +0000 UTC m=+252.283714389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls") pod "image-registry-7cbc88dcf7-fjgnl" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32") : secret "image-registry-tls" not found
Jul 29 00:58:33.292025 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.291982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ql5tv" event={"ID":"6add1afe-4f0f-4c2e-b2e5-3ce07696f232","Type":"ContainerStarted","Data":"7baac8096e1143910946974b1c3c8531c999ba8bf5e643122f705647a8625ada"}
Jul 29 00:58:33.309556 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.309513    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-ql5tv" podStartSLOduration=1.8456911489999999 podStartE2EDuration="4.309499382s" totalImagesPullingTime="2.463808233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:58:33.307560365 +0000 UTC m=+248.385547363" watchObservedRunningTime="2026-07-29 00:58:33.309499382 +0000 UTC m=+248.387486700"
Jul 29 00:58:33.571974 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.571898    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bgqlp"]
Jul 29 00:58:33.575207 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.575189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.577789 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.577766    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 00:58:33.577927 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.577851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 00:58:33.577927 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.577896    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vj9dt\""
Jul 29 00:58:33.583514 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.583492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bgqlp"]
Jul 29 00:58:33.708841 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.708809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a585056b-05ae-4451-93a3-cccbbdd3107b-data-volume\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.708841 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.708840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.708999 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.708900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a585056b-05ae-4451-93a3-cccbbdd3107b-crio-socket\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.708999 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.708954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.708999 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.708973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxjdn\" (UniqueName: \"kubernetes.io/projected/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-api-access-nxjdn\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810105 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a585056b-05ae-4451-93a3-cccbbdd3107b-crio-socket\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810236 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810236 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxjdn\" (UniqueName: \"kubernetes.io/projected/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-api-access-nxjdn\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810236 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a585056b-05ae-4451-93a3-cccbbdd3107b-data-volume\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810236 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a585056b-05ae-4451-93a3-cccbbdd3107b-crio-socket\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810236 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.810417 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.810207    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:33.810417 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.810317    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:33.810417 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.810359    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:34.310336366 +0000 UTC m=+249.388323379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:33.810417 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:33.810378    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:34.310369123 +0000 UTC m=+249.388356111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:33.810549 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.810511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a585056b-05ae-4451-93a3-cccbbdd3107b-data-volume\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:33.818777 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:33.818752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxjdn\" (UniqueName: \"kubernetes.io/projected/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-api-access-nxjdn\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:34.313325 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:34.313295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:34.313679 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:34.313359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:34.313679 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:34.313423    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:34.313679 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:34.313431    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:34.313679 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:34.313490    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:35.313473427 +0000 UTC m=+250.391460417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:34.313679 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:34.313504    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:35.313497267 +0000 UTC m=+250.391484244 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:35.320842 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:35.320813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:35.321214 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:35.320863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:35.321214 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:35.320929    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:35.321214 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:35.320961    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:35.321214 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:35.320988    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.320972997 +0000 UTC m=+252.398959975 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:35.321214 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:35.321002    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.320996052 +0000 UTC m=+252.398983029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:35.346151 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:35.346129    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g2jdj_e2e3b46e-2cdd-48e3-8704-dfa897beccda/dns-node-resolver/0.log"
Jul 29 00:58:36.345699 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.345673    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kfvrh_df7478cd-6084-4776-b323-38fa6eeaeddd/node-ca/0.log"
Jul 29 00:58:36.541108 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.541073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-n9jtm"]
Jul 29 00:58:36.544186 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.544171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.546995 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.546968    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 00:58:36.548278 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.548245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 00:58:36.548369 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.548298    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 00:58:36.548369 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.548331    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 00:58:36.548369 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.548298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m5ch2\""
Jul 29 00:58:36.551725 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.551707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-n9jtm"]
Jul 29 00:58:36.631843 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.631790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/de59426a-5e09-480b-ba47-21851054ed07-signing-cabundle\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.631843 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.631820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhhx8\" (UniqueName: \"kubernetes.io/projected/de59426a-5e09-480b-ba47-21851054ed07-kube-api-access-qhhx8\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.631967 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.631885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/de59426a-5e09-480b-ba47-21851054ed07-signing-key\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.732307 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.732284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/de59426a-5e09-480b-ba47-21851054ed07-signing-key\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.732371 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.732328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/de59426a-5e09-480b-ba47-21851054ed07-signing-cabundle\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.732371 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.732348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhhx8\" (UniqueName: \"kubernetes.io/projected/de59426a-5e09-480b-ba47-21851054ed07-kube-api-access-qhhx8\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.732895 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.732873    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/de59426a-5e09-480b-ba47-21851054ed07-signing-cabundle\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.734695 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.734666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/de59426a-5e09-480b-ba47-21851054ed07-signing-key\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.741467 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.741446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhhx8\" (UniqueName: \"kubernetes.io/projected/de59426a-5e09-480b-ba47-21851054ed07-kube-api-access-qhhx8\") pod \"service-ca-658d7b7c78-n9jtm\" (UID: \"de59426a-5e09-480b-ba47-21851054ed07\") " pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.853878 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.853853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm"
Jul 29 00:58:36.972279 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:36.972228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-n9jtm"]
Jul 29 00:58:36.975001 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:58:36.974976    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde59426a_5e09_480b_ba47_21851054ed07.slice/crio-2dd993cbddf58226f96be480c6cad48302e2aa68a138a03ab26895df113fe21d WatchSource:0}: Error finding container 2dd993cbddf58226f96be480c6cad48302e2aa68a138a03ab26895df113fe21d: Status 404 returned error can't find the container with id 2dd993cbddf58226f96be480c6cad48302e2aa68a138a03ab26895df113fe21d
Jul 29 00:58:37.235792 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.235718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:58:37.235927 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.235795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:37.235927 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.235873    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:58:37.235927 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.235891    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 00:58:37.235927 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.235902    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbc88dcf7-fjgnl: secret "image-registry-tls" not found
Jul 29 00:58:37.236048 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.235951    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs podName:bcb66aad-89ed-482e-8d49-abefa4390ab9 nodeName:}" failed. No retries permitted until 2026-07-29 01:00:39.235932572 +0000 UTC m=+374.313919557 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs") pod "network-metrics-daemon-ckvff" (UID: "bcb66aad-89ed-482e-8d49-abefa4390ab9") : secret "metrics-daemon-secret" not found
Jul 29 00:58:37.236048 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.235969    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls podName:7d5e68cb-8c2d-4ca7-a142-3a14111e9e32 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:45.235961006 +0000 UTC m=+260.313947988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls") pod "image-registry-7cbc88dcf7-fjgnl" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32") : secret "image-registry-tls" not found
Jul 29 00:58:37.304390 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.304363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm" event={"ID":"de59426a-5e09-480b-ba47-21851054ed07","Type":"ContainerStarted","Data":"2dd993cbddf58226f96be480c6cad48302e2aa68a138a03ab26895df113fe21d"}
Jul 29 00:58:37.336742 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.336719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:37.336851 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.336770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:37.336899 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.336854    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:37.336899 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.336894    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:41.336881922 +0000 UTC m=+256.414868903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:37.337010 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.336950    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:37.337061 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:37.337032    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:41.337011952 +0000 UTC m=+256.414998928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:37.546575 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:37.546500    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s269z_8d76dbfe-b355-43ae-a6ff-bf58ffae18c4/serve-healthcheck-canary/0.log"
Jul 29 00:58:39.312583 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:39.312549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm" event={"ID":"de59426a-5e09-480b-ba47-21851054ed07","Type":"ContainerStarted","Data":"e077757fc56e3fb2852eb0a45f8291a172ed60a8ffca486610a347caab15c459"}
Jul 29 00:58:39.327892 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:39.327847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-n9jtm" podStartSLOduration=1.67321132 podStartE2EDuration="3.327833309s" totalImagesPullingTime="1.654621989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:58:39.327004108 +0000 UTC m=+254.404991127" watchObservedRunningTime="2026-07-29 00:58:39.327833309 +0000 UTC m=+254.405820305"
Jul 29 00:58:41.369836 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:41.369795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:41.370321 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:41.369878    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:41.370321 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:41.369907    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:41.370321 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:41.369972    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:49.369955408 +0000 UTC m=+264.447942384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:41.370321 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:41.369999    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:41.370321 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:41.370052    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:58:49.370036699 +0000 UTC m=+264.448023691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:45.300727 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:45.300692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:45.303176 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:45.303141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"image-registry-7cbc88dcf7-fjgnl\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") " pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:45.332694 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:45.332668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:45.454280 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:45.454237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:58:45.457628 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:58:45.457596    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5e68cb_8c2d_4ca7_a142_3a14111e9e32.slice/crio-c099adeb6ff2f7779e9d6dae2287f8c48ef6d65fa18599fe35986029baea6e61 WatchSource:0}: Error finding container c099adeb6ff2f7779e9d6dae2287f8c48ef6d65fa18599fe35986029baea6e61: Status 404 returned error can't find the container with id c099adeb6ff2f7779e9d6dae2287f8c48ef6d65fa18599fe35986029baea6e61
Jul 29 00:58:46.339084 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:46.339052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" event={"ID":"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32","Type":"ContainerStarted","Data":"0990f09b5ee5d6d24e9178f71dce5338e856f223273bae61fb398078e5a5190a"}
Jul 29 00:58:46.339565 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:46.339107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" event={"ID":"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32","Type":"ContainerStarted","Data":"c099adeb6ff2f7779e9d6dae2287f8c48ef6d65fa18599fe35986029baea6e61"}
Jul 29 00:58:46.339565 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:46.339180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:58:46.358042 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:46.357997    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" podStartSLOduration=17.357982683 podStartE2EDuration="17.357982683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:58:46.357153314 +0000 UTC m=+261.435140313" watchObservedRunningTime="2026-07-29 00:58:46.357982683 +0000 UTC m=+261.435969682"
Jul 29 00:58:49.431890 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:49.431819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:49.431890 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:49.431884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:49.432360 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:49.431980    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:49.432360 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:58:49.432045    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls podName:a585056b-05ae-4451-93a3-cccbbdd3107b nodeName:}" failed. No retries permitted until 2026-07-29 00:59:05.432030692 +0000 UTC m=+280.510017670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bgqlp" (UID: "a585056b-05ae-4451-93a3-cccbbdd3107b") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:49.432440 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:49.432388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a585056b-05ae-4451-93a3-cccbbdd3107b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:58:58.313969 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:58:58.313935    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:59:05.442456 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:05.442412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:59:05.444623 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:05.444603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a585056b-05ae-4451-93a3-cccbbdd3107b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bgqlp\" (UID: \"a585056b-05ae-4451-93a3-cccbbdd3107b\") " pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:59:05.684366 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:05.684337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bgqlp"
Jul 29 00:59:05.809379 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:05.809347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bgqlp"]
Jul 29 00:59:05.812330 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:05.812304    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda585056b_05ae_4451_93a3_cccbbdd3107b.slice/crio-cdabfb2ae681c0345ffd813aebb5f830330b738fceef625e15f300f5b566077e WatchSource:0}: Error finding container cdabfb2ae681c0345ffd813aebb5f830330b738fceef625e15f300f5b566077e: Status 404 returned error can't find the container with id cdabfb2ae681c0345ffd813aebb5f830330b738fceef625e15f300f5b566077e
Jul 29 00:59:06.398073 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:06.398028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bgqlp" event={"ID":"a585056b-05ae-4451-93a3-cccbbdd3107b","Type":"ContainerStarted","Data":"cdabfb2ae681c0345ffd813aebb5f830330b738fceef625e15f300f5b566077e"}
Jul 29 00:59:07.063207 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:07.063167    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-89gsp" podUID="057a7481-05af-465c-acad-ff70ddbef93f"
Jul 29 00:59:07.401517 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:07.401484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:08.321734 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:08.321687    2563 patch_prober.go:28] interesting pod/image-registry-7cbc88dcf7-fjgnl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 00:59:08.322175 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:08.321749    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 00:59:10.785784 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:10.785750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:10.787884 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:10.787853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/057a7481-05af-465c-acad-ff70ddbef93f-metrics-tls\") pod \"dns-default-89gsp\" (UID: \"057a7481-05af-465c-acad-ff70ddbef93f\") " pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:11.004706 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:11.004643    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9mcp8\""
Jul 29 00:59:11.012780 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:11.012765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:11.170303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:11.170228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89gsp"]
Jul 29 00:59:11.174245 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:11.174213    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057a7481_05af_465c_acad_ff70ddbef93f.slice/crio-4ab58dbe9c69e60d8eb28a0321b50724889b54112c23983a6d58e9601ca16cee WatchSource:0}: Error finding container 4ab58dbe9c69e60d8eb28a0321b50724889b54112c23983a6d58e9601ca16cee: Status 404 returned error can't find the container with id 4ab58dbe9c69e60d8eb28a0321b50724889b54112c23983a6d58e9601ca16cee
Jul 29 00:59:11.417395 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:11.417303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bgqlp" event={"ID":"a585056b-05ae-4451-93a3-cccbbdd3107b","Type":"ContainerStarted","Data":"ed6cd4b5bd6ca588fb07246dcb2a2b97ad38c16316bc389dfa5589b9afbba1a1"}
Jul 29 00:59:11.418684 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:11.418645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89gsp" event={"ID":"057a7481-05af-465c-acad-ff70ddbef93f","Type":"ContainerStarted","Data":"4ab58dbe9c69e60d8eb28a0321b50724889b54112c23983a6d58e9601ca16cee"}
Jul 29 00:59:12.931659 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.931614    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"]
Jul 29 00:59:12.945075 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.945049    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"]
Jul 29 00:59:12.945245 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.945179    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:12.950531 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.949325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 00:59:12.950531 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.949537    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hhsd9\""
Jul 29 00:59:12.950899 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.950871    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 00:59:12.955274 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.951089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 00:59:12.955274 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.951336    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 00:59:12.956285 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.955529    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 00:59:12.957154 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.957045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dpcrf"]
Jul 29 00:59:12.962482 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.962462    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:12.965463 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.965393    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 00:59:12.966425 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.965809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 00:59:12.966425 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.966030    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f2tnk\""
Jul 29 00:59:12.966425 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:12.966247    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 00:59:13.002688 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-wtmp\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.002811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.002811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-textfile\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.002811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-sys\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.002811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.002811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brxlr\" (UniqueName: \"kubernetes.io/projected/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-kube-api-access-brxlr\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wjb\" (UniqueName: \"kubernetes.io/projected/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-kube-api-access-52wjb\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-root\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.002978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-metrics-client-ca\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.003063 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.003002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.104245 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-wtmp\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104245 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-textfile\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-sys\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brxlr\" (UniqueName: \"kubernetes.io/projected/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-kube-api-access-brxlr\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wjb\" (UniqueName: \"kubernetes.io/projected/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-kube-api-access-52wjb\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.104508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-root\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-metrics-client-ca\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.104598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:13.104754    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 29 00:59:13.104881 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:13.104817    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls podName:a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:13.604797708 +0000 UTC m=+288.682784691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-59h4n" (UID: "a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4") : secret "openshift-state-metrics-tls" not found
Jul 29 00:59:13.105200 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.105137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-wtmp\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:13.105397    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:13.105456    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls podName:bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:13.605437159 +0000 UTC m=+288.683424148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls") pod "node-exporter-dpcrf" (UID: "bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8") : secret "node-exporter-tls" not found
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.105671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.105793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-textfile\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.106129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-metrics-client-ca\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.106171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-root\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.106213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-sys\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.106572 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.106529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.108842 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.108759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.110852 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.110811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.117611 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.117584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wjb\" (UniqueName: \"kubernetes.io/projected/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-kube-api-access-52wjb\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.120142 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.120091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brxlr\" (UniqueName: \"kubernetes.io/projected/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-kube-api-access-brxlr\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.608627 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.608593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.608817 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.608638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.611201 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.611174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8-node-exporter-tls\") pod \"node-exporter-dpcrf\" (UID: \"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8\") " pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:13.611341 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.611273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-59h4n\" (UID: \"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.862939 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.862858    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"
Jul 29 00:59:13.874302 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:13.874278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dpcrf"
Jul 29 00:59:14.011906 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.011879    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 00:59:14.017306 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.017230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.020511 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020433    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 00:59:14.020511 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020453    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 00:59:14.020511 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020493    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 00:59:14.020733 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 00:59:14.020733 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 00:59:14.020733 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.020650    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x5gsh\""
Jul 29 00:59:14.021780 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.021759    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 00:59:14.021881 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.021768    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 00:59:14.022051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.021926    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 00:59:14.030337 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.030304    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 00:59:14.034481 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.034458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 00:59:14.112387 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112570 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112570 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112570 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112740 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112740 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112740 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112740 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.112740 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc48j\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.113014 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.113014 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.113014 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.113014 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.112848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.213782 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.213750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.213782 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.213785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214047 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.213811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214047 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.213987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214047 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc48j\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214536 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214536 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.214536 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.215165 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.214954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.215307 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.215236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.215449 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.215425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.216979 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.216952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.217359 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.217335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.218200 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.218177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.218308 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.218279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.219620 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.219568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.219717 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.219687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.219783 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.219714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.220143 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.220123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.220308 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.220286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.222646 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.222628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc48j\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j\") pod \"alertmanager-main-0\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.330627 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.330591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 00:59:14.613312 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:14.613273    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd6b80a4_f48b_43fd_bcd9_fb8bd36f22d8.slice/crio-0c29c79f9c444ba4647d477c53a6567c9ae99e6de9c81c76256b7159074883b3 WatchSource:0}: Error finding container 0c29c79f9c444ba4647d477c53a6567c9ae99e6de9c81c76256b7159074883b3: Status 404 returned error can't find the container with id 0c29c79f9c444ba4647d477c53a6567c9ae99e6de9c81c76256b7159074883b3
Jul 29 00:59:14.776489 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.776416    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n"]
Jul 29 00:59:14.781676 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:14.781648    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e416d9_c8aa_4cf8_a7f4_d6a6b79c06d4.slice/crio-4e17d761e3ce86ab09034acfb86887e0bee02702b4ecb10501876cf1a98ce12e WatchSource:0}: Error finding container 4e17d761e3ce86ab09034acfb86887e0bee02702b4ecb10501876cf1a98ce12e: Status 404 returned error can't find the container with id 4e17d761e3ce86ab09034acfb86887e0bee02702b4ecb10501876cf1a98ce12e
Jul 29 00:59:14.801547 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.801525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 00:59:14.811988 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:14.811945    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4969ef26_db89_401c_9d81_6868fa45ca5a.slice/crio-9490719b771a4f2f872934818936e5eaf0933ae04aaa90e2f9106f9bf240c23c WatchSource:0}: Error finding container 9490719b771a4f2f872934818936e5eaf0933ae04aaa90e2f9106f9bf240c23c: Status 404 returned error can't find the container with id 9490719b771a4f2f872934818936e5eaf0933ae04aaa90e2f9106f9bf240c23c
Jul 29 00:59:14.926508 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.926473    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"]
Jul 29 00:59:14.930545 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.930523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:14.933773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.933749    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 00:59:14.933916 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.933783    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 00:59:14.933916 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.933912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 00:59:14.934051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.933959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q6b7s\""
Jul 29 00:59:14.934051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.933975    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bt4ebo8ltqr58\""
Jul 29 00:59:14.934051 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.934012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 00:59:14.934192 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.934097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 00:59:14.940973 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:14.940954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"]
Jul 29 00:59:15.020867 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.020826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb5fa64b-6822-4560-b89c-2186b8d7cdec-metrics-client-ca\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.020875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.020929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rfzj\" (UniqueName: \"kubernetes.io/projected/fb5fa64b-6822-4560-b89c-2186b8d7cdec-kube-api-access-7rfzj\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.020956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-grpc-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.021011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.021056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.021075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.021328 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.021206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb5fa64b-6822-4560-b89c-2186b8d7cdec-metrics-client-ca\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rfzj\" (UniqueName: \"kubernetes.io/projected/fb5fa64b-6822-4560-b89c-2186b8d7cdec-kube-api-access-7rfzj\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.122741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-grpc-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.123303 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.123198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb5fa64b-6822-4560-b89c-2186b8d7cdec-metrics-client-ca\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.125469 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.125443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.125608 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.125443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.125874 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.125838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.126347 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.126322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.126757 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.126714    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-grpc-tls\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.127845 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.127814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fb5fa64b-6822-4560-b89c-2186b8d7cdec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.131349 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.131323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rfzj\" (UniqueName: \"kubernetes.io/projected/fb5fa64b-6822-4560-b89c-2186b8d7cdec-kube-api-access-7rfzj\") pod \"thanos-querier-5c5b5df75f-jlthr\" (UID: \"fb5fa64b-6822-4560-b89c-2186b8d7cdec\") " pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.240112 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.239977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:15.434975 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.434895    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bgqlp" event={"ID":"a585056b-05ae-4451-93a3-cccbbdd3107b","Type":"ContainerStarted","Data":"af650b0bf797bcc6e3163a3312b4bb2c487368da9be2194a9f3ff422c03b9698"}
Jul 29 00:59:15.436811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.436720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89gsp" event={"ID":"057a7481-05af-465c-acad-ff70ddbef93f","Type":"ContainerStarted","Data":"9bffb3ab009864f8816fc80895f63ec4a76cb31ef408d641a653b3ccc6e0959e"}
Jul 29 00:59:15.436811 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.436779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89gsp" event={"ID":"057a7481-05af-465c-acad-ff70ddbef93f","Type":"ContainerStarted","Data":"82ef686e5a5cbb00d5254b888bdc1bc9a98784c03fb6edcdac03ba5464cac646"}
Jul 29 00:59:15.437018 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.436849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:15.438434 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.438404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"9490719b771a4f2f872934818936e5eaf0933ae04aaa90e2f9106f9bf240c23c"}
Jul 29 00:59:15.440563 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.440539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n" event={"ID":"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4","Type":"ContainerStarted","Data":"dfa31b140c0d4b4031582ba7e2227f987115a7063fa233a4accd1f869f18e879"}
Jul 29 00:59:15.440674 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.440601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n" event={"ID":"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4","Type":"ContainerStarted","Data":"6d58f13e157036895568f93bea009c80c1284701689cc391b19515e78036adab"}
Jul 29 00:59:15.440674 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.440631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n" event={"ID":"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4","Type":"ContainerStarted","Data":"4e17d761e3ce86ab09034acfb86887e0bee02702b4ecb10501876cf1a98ce12e"}
Jul 29 00:59:15.442044 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.442022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dpcrf" event={"ID":"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8","Type":"ContainerStarted","Data":"0c29c79f9c444ba4647d477c53a6567c9ae99e6de9c81c76256b7159074883b3"}
Jul 29 00:59:15.454148 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.454095    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-89gsp" podStartSLOduration=252.01848047 podStartE2EDuration="4m15.454078436s" totalImagesPullingTime="3.435597966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:15.452665849 +0000 UTC m=+290.530652870" watchObservedRunningTime="2026-07-29 00:59:15.454078436 +0000 UTC m=+290.532065436"
Jul 29 00:59:15.580837 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:15.580718    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"]
Jul 29 00:59:16.449386 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:16.449142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"}
Jul 29 00:59:16.451705 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:16.451571    2563 generic.go:356] "Generic (PLEG): container finished" podID="bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8" containerID="cfbcf043500ce9e30e98cc4ade76cc396064d8d8b8b08e3a7abdd22d223a9917" exitCode=0
Jul 29 00:59:16.451850 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:16.451694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dpcrf" event={"ID":"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8","Type":"ContainerDied","Data":"cfbcf043500ce9e30e98cc4ade76cc396064d8d8b8b08e3a7abdd22d223a9917"}
Jul 29 00:59:16.454798 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:16.454713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"1bf30ff13b81d30434fbde630d206c44ab0e66bb1779d9a77b84f0e34a4cecad"}
Jul 29 00:59:17.460047 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:17.460013    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7" exitCode=0
Jul 29 00:59:17.460496 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:17.460097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"}
Jul 29 00:59:17.462706 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:17.462681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n" event={"ID":"a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4","Type":"ContainerStarted","Data":"ea80d9e62663d62b02861294e99d0c5ee9d56e2c19955ad039c67287f230c95a"}
Jul 29 00:59:17.464962 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:17.464935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dpcrf" event={"ID":"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8","Type":"ContainerStarted","Data":"5868ecf36cb8f7eaed91779837090bf9dc7519184adc96bea55e9a53cda34942"}
Jul 29 00:59:17.507011 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:17.506965    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-59h4n" podStartSLOduration=3.6414672980000002 podStartE2EDuration="5.506949037s" totalImagesPullingTime="1.865481739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:17.505394343 +0000 UTC m=+292.583381367" watchObservedRunningTime="2026-07-29 00:59:17.506949037 +0000 UTC m=+292.584936038"
Jul 29 00:59:18.130100 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.130071    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58b56ff58c-95x9j"]
Jul 29 00:59:18.144837 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.144801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.147629 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.147603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58b56ff58c-95x9j"]
Jul 29 00:59:18.148408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-crhfh\""
Jul 29 00:59:18.148408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148162    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 00:59:18.148408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148174    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 00:59:18.148408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 00:59:18.148408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 00:59:18.148742 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.148432    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 00:59:18.154932 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.154495    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 00:59:18.253911 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.253874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254069 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.253919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-federate-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254069 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.253955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-serving-certs-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254069 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.254010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254069 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.254035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254316 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.254109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254316 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.254135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-metrics-client-ca\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.254316 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.254206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgj62\" (UniqueName: \"kubernetes.io/projected/f4c37f3d-8622-4627-97a5-adde485f486a-kube-api-access-jgj62\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.319397 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.319372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:59:18.356286 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356451 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-federate-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356451 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-serving-certs-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356451 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356451 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356661 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356661 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-metrics-client-ca\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.356661 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.356616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgj62\" (UniqueName: \"kubernetes.io/projected/f4c37f3d-8622-4627-97a5-adde485f486a-kube-api-access-jgj62\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.359392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.358997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.359392 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.359298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-serving-certs-ca-bundle\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.361111 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.361077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c37f3d-8622-4627-97a5-adde485f486a-metrics-client-ca\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.365593 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.365540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-telemeter-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.366486 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.366440    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-federate-client-tls\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.367345 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.367296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.367609 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.367534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgj62\" (UniqueName: \"kubernetes.io/projected/f4c37f3d-8622-4627-97a5-adde485f486a-kube-api-access-jgj62\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.367755 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.367730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4c37f3d-8622-4627-97a5-adde485f486a-secret-telemeter-client\") pod \"telemeter-client-58b56ff58c-95x9j\" (UID: \"f4c37f3d-8622-4627-97a5-adde485f486a\") " pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:18.457602 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:18.457513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j"
Jul 29 00:59:19.287554 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.287526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58b56ff58c-95x9j"]
Jul 29 00:59:19.290961 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:19.290931    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c37f3d_8622_4627_97a5_adde485f486a.slice/crio-1aa7fbc0237e9daca203caed6f574e7ed62b80f64de1260966ebd60b3898308d WatchSource:0}: Error finding container 1aa7fbc0237e9daca203caed6f574e7ed62b80f64de1260966ebd60b3898308d: Status 404 returned error can't find the container with id 1aa7fbc0237e9daca203caed6f574e7ed62b80f64de1260966ebd60b3898308d
Jul 29 00:59:19.478310 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.478273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dpcrf" event={"ID":"bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8","Type":"ContainerStarted","Data":"0bc7809dac7a4afd3c7027718132e427726dee4e9cba7ddf774dfff2e1c9871b"}
Jul 29 00:59:19.480699 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.480665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"0ed56dd3842da72e33f9ae0cccdd7f5e13f0a88184ba32e7bdff2270cc56b350"}
Jul 29 00:59:19.480825 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.480741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"a4591d7920e50684e6fbc6bd4df694d570e517bdde4b8765c759e3a34d43669b"}
Jul 29 00:59:19.480825 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.480784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"c080e94f8f7f68d0e99ecdac31068386fad2f7a63751dc8115ab4f931b2a8e59"}
Jul 29 00:59:19.482092 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.482040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j" event={"ID":"f4c37f3d-8622-4627-97a5-adde485f486a","Type":"ContainerStarted","Data":"1aa7fbc0237e9daca203caed6f574e7ed62b80f64de1260966ebd60b3898308d"}
Jul 29 00:59:19.484570 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.484548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bgqlp" event={"ID":"a585056b-05ae-4451-93a3-cccbbdd3107b","Type":"ContainerStarted","Data":"3e8c15e8927d1b391b102177b6f91661997a02d7a60a6bccc7c0459152cf60ff"}
Jul 29 00:59:19.496044 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.496010    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dpcrf" podStartSLOduration=6.547815938 podStartE2EDuration="7.495998198s" totalImagesPullingTime="868.431563ms" totalInitContainerRuntime="79.750697ms" isStatefulPod=true podCreationTimestamp="2026-07-29 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:19.49524364 +0000 UTC m=+294.573230704" watchObservedRunningTime="2026-07-29 00:59:19.495998198 +0000 UTC m=+294.573985195"
Jul 29 00:59:19.513205 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:19.513159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bgqlp" podStartSLOduration=33.35639853 podStartE2EDuration="46.513142579s" totalImagesPullingTime="13.156744049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:58:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:19.511816824 +0000 UTC m=+294.589803842" watchObservedRunningTime="2026-07-29 00:59:19.513142579 +0000 UTC m=+294.591129579"
Jul 29 00:59:20.473806 ip-10-0-129-224 kubenswrapper[2563]: E0729 00:59:20.473780    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ckvff" podUID="bcb66aad-89ed-482e-8d49-abefa4390ab9"
Jul 29 00:59:20.493237 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"}
Jul 29 00:59:20.493332 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"}
Jul 29 00:59:20.493396 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"}
Jul 29 00:59:20.493396 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"}
Jul 29 00:59:20.493487 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"}
Jul 29 00:59:20.493487 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.493467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerStarted","Data":"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"}
Jul 29 00:59:20.496305 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.496281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"55d7b89fefbff0c699036b9442eefc8f54ce00a026e7cbcb3931fbd036625538"}
Jul 29 00:59:20.496408 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.496337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"17166d8ba974c9a0db5205cf6484c2ac3f1e49f9c7f49a1f04f2cb4a9370ef84"}
Jul 29 00:59:20.521812 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:20.521762    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.459225093 podStartE2EDuration="7.521747881s" totalImagesPullingTime="4.046668613s" totalInitContainerRuntime="15.854175ms" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:20.519385257 +0000 UTC m=+295.597372267" watchObservedRunningTime="2026-07-29 00:59:20.521747881 +0000 UTC m=+295.599734879"
Jul 29 00:59:21.502097 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.502066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" event={"ID":"fb5fa64b-6822-4560-b89c-2186b8d7cdec","Type":"ContainerStarted","Data":"245ba8dc3bb006c4d91c62e67a6aa6e840ef683aed42c3f42066efae86b7f5b7"}
Jul 29 00:59:21.502470 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.502163    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:21.504002 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.503980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j" event={"ID":"f4c37f3d-8622-4627-97a5-adde485f486a","Type":"ContainerStarted","Data":"3027a217b345c188db3e0ac1d49736df39f96f8b99b742d41fcd1fd12a637f48"}
Jul 29 00:59:21.504114 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.504023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j" event={"ID":"f4c37f3d-8622-4627-97a5-adde485f486a","Type":"ContainerStarted","Data":"68cdabf8dd5c87dfaf0e9f4307b5e17e0e1d56dd5b8ec4f09f59376b44914347"}
Jul 29 00:59:21.504114 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.504043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j" event={"ID":"f4c37f3d-8622-4627-97a5-adde485f486a","Type":"ContainerStarted","Data":"15166f0aa0b1ae7c0b499f9d069e3a28994e319155e561e233579ae5029119b6"}
Jul 29 00:59:21.525466 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.525427    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr" podStartSLOduration=3.04374733 podStartE2EDuration="7.525414912s" totalImagesPullingTime="4.481667582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:21.523385364 +0000 UTC m=+296.601372373" watchObservedRunningTime="2026-07-29 00:59:21.525414912 +0000 UTC m=+296.603401911"
Jul 29 00:59:21.543833 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:21.543757    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58b56ff58c-95x9j" podStartSLOduration=1.550555344 podStartE2EDuration="3.543745632s" totalImagesPullingTime="1.993190288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:21.54187546 +0000 UTC m=+296.619862438" watchObservedRunningTime="2026-07-29 00:59:21.543745632 +0000 UTC m=+296.621732631"
Jul 29 00:59:23.332321 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.332253    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerName="registry" containerID="cri-o://0990f09b5ee5d6d24e9178f71dce5338e856f223273bae61fb398078e5a5190a" gracePeriod=30
Jul 29 00:59:23.513093 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.513066    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerID="0990f09b5ee5d6d24e9178f71dce5338e856f223273bae61fb398078e5a5190a" exitCode=0
Jul 29 00:59:23.513221 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.513147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" event={"ID":"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32","Type":"ContainerDied","Data":"0990f09b5ee5d6d24e9178f71dce5338e856f223273bae61fb398078e5a5190a"}
Jul 29 00:59:23.575369 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.575348    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:59:23.702966 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.702908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.702966 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.702936    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbrmw\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.702966 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.702966    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.702995    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703080    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703183 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted\") pod \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\" (UID: \"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32\") "
Jul 29 00:59:23.703644 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703617    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:59:23.703784 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.703759    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 00:59:23.705444 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.705419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:59:23.705542 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.705470    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 00:59:23.705614 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.705578    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:59:23.705614 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.705597    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw" (OuterVolumeSpecName: "kube-api-access-sbrmw") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "kube-api-access-sbrmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:59:23.705773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.705750    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 00:59:23.711798 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.711779    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" (UID: "7d5e68cb-8c2d-4ca7-a142-3a14111e9e32"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 00:59:23.804277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804235    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-bound-sa-token\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804277 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804276    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-ca-trust-extracted\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804290    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-tls\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804299    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbrmw\" (UniqueName: \"kubernetes.io/projected/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-kube-api-access-sbrmw\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804309    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-installation-pull-secrets\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804317    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-image-registry-private-configuration\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804327    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-trusted-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:23.804418 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:23.804336    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32-registry-certificates\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 00:59:24.518272 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:24.518227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl" event={"ID":"7d5e68cb-8c2d-4ca7-a142-3a14111e9e32","Type":"ContainerDied","Data":"c099adeb6ff2f7779e9d6dae2287f8c48ef6d65fa18599fe35986029baea6e61"}
Jul 29 00:59:24.518272 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:24.518275    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"
Jul 29 00:59:24.518711 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:24.518318    2563 scope.go:117] "RemoveContainer" containerID="0990f09b5ee5d6d24e9178f71dce5338e856f223273bae61fb398078e5a5190a"
Jul 29 00:59:24.538817 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:24.538790    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:59:24.541917 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:24.541897    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cbc88dcf7-fjgnl"]
Jul 29 00:59:25.332701 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:25.332684    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 00:59:25.457129 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:25.457105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-89gsp"
Jul 29 00:59:25.476499 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:25.476241    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" path="/var/lib/kubelet/pods/7d5e68cb-8c2d-4ca7-a142-3a14111e9e32/volumes"
Jul 29 00:59:27.514443 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:27.514417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c5b5df75f-jlthr"
Jul 29 00:59:35.472241 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.472213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 00:59:35.896662 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.896635    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 00:59:35.896953 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.896940    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerName="registry"
Jul 29 00:59:35.896999 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.896956    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerName="registry"
Jul 29 00:59:35.897033 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.897008    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d5e68cb-8c2d-4ca7-a142-3a14111e9e32" containerName="registry"
Jul 29 00:59:35.902003 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.901984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.905041 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.905017    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hqvls\""
Jul 29 00:59:35.905145 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.905018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 00:59:35.905145 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.905017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 00:59:35.906350 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.906333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 00:59:35.906702 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.906686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 00:59:35.906823 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.906803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 00:59:35.906891 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.906871    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 00:59:35.906891 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.906879    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 00:59:35.910547 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.910525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 00:59:35.993189 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.993311 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qkzc\" (UniqueName: \"kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.993311 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.993387 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.993387 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:35.993450 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:35.993411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094583 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qkzc\" (UniqueName: \"kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094583 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.094773 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.094696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.095380 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.095352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.095509 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.095492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.095551 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.095512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.097091 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.097066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.097196 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.097181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.102897 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.102867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qkzc\" (UniqueName: \"kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc\") pod \"console-766c6896b5-bsznj\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") " pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.211890 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.211823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:36.339050 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.339018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 00:59:36.341838 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:36.341806    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a10bb30_a2fa_4a32_8ff4_176263582bb9.slice/crio-a4e06615e457063e302b2a129a0688b6a06f81b2c512b6c5ec4c0c791ad95cc3 WatchSource:0}: Error finding container a4e06615e457063e302b2a129a0688b6a06f81b2c512b6c5ec4c0c791ad95cc3: Status 404 returned error can't find the container with id a4e06615e457063e302b2a129a0688b6a06f81b2c512b6c5ec4c0c791ad95cc3
Jul 29 00:59:36.344161 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.344144    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:59:36.564732 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:36.564652    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766c6896b5-bsznj" event={"ID":"9a10bb30-a2fa-4a32-8ff4-176263582bb9","Type":"ContainerStarted","Data":"a4e06615e457063e302b2a129a0688b6a06f81b2c512b6c5ec4c0c791ad95cc3"}
Jul 29 00:59:39.577025 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:39.576994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766c6896b5-bsznj" event={"ID":"9a10bb30-a2fa-4a32-8ff4-176263582bb9","Type":"ContainerStarted","Data":"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"}
Jul 29 00:59:39.595197 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:39.595151    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-766c6896b5-bsznj" podStartSLOduration=1.7874444870000001 podStartE2EDuration="4.595137571s" totalImagesPullingTime="2.807693084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:39.593959689 +0000 UTC m=+314.671946687" watchObservedRunningTime="2026-07-29 00:59:39.595137571 +0000 UTC m=+314.673124569"
Jul 29 00:59:44.016327 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.016285    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 00:59:44.089069 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.089040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 00:59:44.089209 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.089128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.097453 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.097424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 00:59:44.263914 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.263879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.263921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmk2\" (UniqueName: \"kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.263944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.263995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.264019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264056 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.264037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.264235 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.264093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.364746 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.364718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.364891 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.364751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmk2\" (UniqueName: \"kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.364891 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.364773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.364991 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.364895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.364991 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.364935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365096 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365483 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365614 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365696 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.365759 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.365697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.367389 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.367371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.367461 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.367444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.372302 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.372278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmk2\" (UniqueName: \"kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2\") pod \"console-748c96d796-df9qd\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") " pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.398120 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.398101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:44.562878 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.562855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 00:59:44.565420 ip-10-0-129-224 kubenswrapper[2563]: W0729 00:59:44.565392    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bb4c79_9338_4990_b2ee_4977add8f813.slice/crio-810da998c08b1967e206d07c8aab6774b8cdb4f5606856e2bad5c3ef5514c4ab WatchSource:0}: Error finding container 810da998c08b1967e206d07c8aab6774b8cdb4f5606856e2bad5c3ef5514c4ab: Status 404 returned error can't find the container with id 810da998c08b1967e206d07c8aab6774b8cdb4f5606856e2bad5c3ef5514c4ab
Jul 29 00:59:44.594636 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:44.594613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748c96d796-df9qd" event={"ID":"39bb4c79-9338-4990-b2ee-4977add8f813","Type":"ContainerStarted","Data":"810da998c08b1967e206d07c8aab6774b8cdb4f5606856e2bad5c3ef5514c4ab"}
Jul 29 00:59:45.599712 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:45.599677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748c96d796-df9qd" event={"ID":"39bb4c79-9338-4990-b2ee-4977add8f813","Type":"ContainerStarted","Data":"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"}
Jul 29 00:59:45.617442 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:45.617400    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-748c96d796-df9qd" podStartSLOduration=1.617386577 podStartE2EDuration="1.617386577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:45.615569429 +0000 UTC m=+320.693556429" watchObservedRunningTime="2026-07-29 00:59:45.617386577 +0000 UTC m=+320.695373576"
Jul 29 00:59:46.212880 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:46.212850    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:46.213089 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:46.212907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:46.217816 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:46.217796    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:46.607689 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:46.607658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 00:59:54.398948 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:54.398914    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:54.399412 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:54.399002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:54.403551 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:54.403532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:54.636250 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:54.636226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 00:59:54.676892 ip-10-0-129-224 kubenswrapper[2563]: I0729 00:59:54.676823    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 01:00:03.666194 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:03.666158    2563 generic.go:356] "Generic (PLEG): container finished" podID="6add1afe-4f0f-4c2e-b2e5-3ce07696f232" containerID="7baac8096e1143910946974b1c3c8531c999ba8bf5e643122f705647a8625ada" exitCode=0
Jul 29 01:00:03.666602 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:03.666232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ql5tv" event={"ID":"6add1afe-4f0f-4c2e-b2e5-3ce07696f232","Type":"ContainerDied","Data":"7baac8096e1143910946974b1c3c8531c999ba8bf5e643122f705647a8625ada"}
Jul 29 01:00:03.666602 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:03.666552    2563 scope.go:117] "RemoveContainer" containerID="7baac8096e1143910946974b1c3c8531c999ba8bf5e643122f705647a8625ada"
Jul 29 01:00:04.671643 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:04.671607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-ql5tv" event={"ID":"6add1afe-4f0f-4c2e-b2e5-3ce07696f232","Type":"ContainerStarted","Data":"8fb668621074fc6df7fc909189fff35b8760e973ddc0e654b88cddadc8b3f9a7"}
Jul 29 01:00:19.696106 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:19.696000    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-766c6896b5-bsznj" podUID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" containerName="console" containerID="cri-o://7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750" gracePeriod=15
Jul 29 01:00:19.940213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:19.940188    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 01:00:20.059402 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059402 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059383    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qkzc\" (UniqueName: \"kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059609 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059422    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059609 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059448    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059609 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059609 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059529    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert\") pod \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\" (UID: \"9a10bb30-a2fa-4a32-8ff4-176263582bb9\") "
Jul 29 01:00:20.059806 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059735    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config" (OuterVolumeSpecName: "console-config") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:20.059905 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:20.059953 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.059881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:20.061678 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.061646    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:20.061788 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.061763    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc" (OuterVolumeSpecName: "kube-api-access-6qkzc") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "kube-api-access-6qkzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:00:20.061832 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.061793    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a10bb30-a2fa-4a32-8ff4-176263582bb9" (UID: "9a10bb30-a2fa-4a32-8ff4-176263582bb9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:20.160431 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160394    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-oauth-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.160431 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160425    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.160431 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160435    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-console-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.160644 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160444    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qkzc\" (UniqueName: \"kubernetes.io/projected/9a10bb30-a2fa-4a32-8ff4-176263582bb9-kube-api-access-6qkzc\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.160644 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160454    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-oauth-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.160644 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.160462    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a10bb30-a2fa-4a32-8ff4-176263582bb9-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:20.735813 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.735780    2563 generic.go:356] "Generic (PLEG): container finished" podID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" containerID="7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750" exitCode=0
Jul 29 01:00:20.736208 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.735822    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766c6896b5-bsznj"
Jul 29 01:00:20.736208 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.735870    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766c6896b5-bsznj" event={"ID":"9a10bb30-a2fa-4a32-8ff4-176263582bb9","Type":"ContainerDied","Data":"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"}
Jul 29 01:00:20.736208 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.735917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766c6896b5-bsznj" event={"ID":"9a10bb30-a2fa-4a32-8ff4-176263582bb9","Type":"ContainerDied","Data":"a4e06615e457063e302b2a129a0688b6a06f81b2c512b6c5ec4c0c791ad95cc3"}
Jul 29 01:00:20.736208 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.735938    2563 scope.go:117] "RemoveContainer" containerID="7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"
Jul 29 01:00:20.744636 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.744620    2563 scope.go:117] "RemoveContainer" containerID="7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"
Jul 29 01:00:20.744885 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:20.744863    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750\": container with ID starting with 7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750 not found: ID does not exist" containerID="7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"
Jul 29 01:00:20.744931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.744893    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750"} err="failed to get container status \"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750\": rpc error: code = NotFound desc = could not find container \"7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750\": container with ID starting with 7ff410b35e247747b1de151f053a2970bae3e63782db418fbb6c794772302750 not found: ID does not exist"
Jul 29 01:00:20.756314 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.756293    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 01:00:20.759484 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:20.759466    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-766c6896b5-bsznj"]
Jul 29 01:00:21.474253 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:21.474220    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" path="/var/lib/kubelet/pods/9a10bb30-a2fa-4a32-8ff4-176263582bb9/volumes"
Jul 29 01:00:33.272576 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272538    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272825    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="alertmanager" containerID="cri-o://eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303" gracePeriod=120
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272883    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-metric" containerID="cri-o://4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4" gracePeriod=120
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272920    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="config-reloader" containerID="cri-o://86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72" gracePeriod=120
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272908    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-web" containerID="cri-o://f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90" gracePeriod=120
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.272944    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="prom-label-proxy" containerID="cri-o://ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62" gracePeriod=120
Jul 29 01:00:33.273170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.273022    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy" containerID="cri-o://c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b" gracePeriod=120
Jul 29 01:00:33.789839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789806    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62" exitCode=0
Jul 29 01:00:33.789839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789830    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b" exitCode=0
Jul 29 01:00:33.789839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789837    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72" exitCode=0
Jul 29 01:00:33.789839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789843    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303" exitCode=0
Jul 29 01:00:33.790090 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"}
Jul 29 01:00:33.790090 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"}
Jul 29 01:00:33.790090 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"}
Jul 29 01:00:33.790090 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:33.789969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"}
Jul 29 01:00:34.519432 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.519408    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.677896 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.677896 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677872    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677908    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc48j\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.677996    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678028    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678059    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678163    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web\") pod \"4969ef26-db89-401c-9d81-6868fa45ca5a\" (UID: \"4969ef26-db89-401c-9d81-6868fa45ca5a\") "
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678360    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:34.678500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678393    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:00:34.678788 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678554    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-metrics-client-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.678788 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.678575    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-main-db\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.679244 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.679205    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:34.680862 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.680715    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.680862 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.680820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j" (OuterVolumeSpecName: "kube-api-access-pc48j") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "kube-api-access-pc48j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:00:34.681155 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.681111    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume" (OuterVolumeSpecName: "config-volume") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.681295 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.681244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.681397 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.681319    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out" (OuterVolumeSpecName: "config-out") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:00:34.681477 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.681457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:00:34.682216 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.682187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.682436 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.682403    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.685587 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.685562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.692308 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.692286    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config" (OuterVolumeSpecName: "web-config") pod "4969ef26-db89-401c-9d81-6868fa45ca5a" (UID: "4969ef26-db89-401c-9d81-6868fa45ca5a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:34.779507 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779475    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4969ef26-db89-401c-9d81-6868fa45ca5a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779507 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779505    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4969ef26-db89-401c-9d81-6868fa45ca5a-config-out\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779507 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779515    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779526    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-tls-assets\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779535    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779544    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc48j\" (UniqueName: \"kubernetes.io/projected/4969ef26-db89-401c-9d81-6868fa45ca5a-kube-api-access-pc48j\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779553    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-config-volume\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779561    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-main-tls\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779569    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-cluster-tls-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779577    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-web-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.779666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.779585    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4969ef26-db89-401c-9d81-6868fa45ca5a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:00:34.796574 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796548    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4" exitCode=0
Jul 29 01:00:34.796645 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796585    2563 generic.go:356] "Generic (PLEG): container finished" podID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerID="f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90" exitCode=0
Jul 29 01:00:34.796645 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"}
Jul 29 01:00:34.796722 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796646    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.796722 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"}
Jul 29 01:00:34.796790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796765    2563 scope.go:117] "RemoveContainer" containerID="ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"
Jul 29 01:00:34.796790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.796777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4969ef26-db89-401c-9d81-6868fa45ca5a","Type":"ContainerDied","Data":"9490719b771a4f2f872934818936e5eaf0933ae04aaa90e2f9106f9bf240c23c"}
Jul 29 01:00:34.807410 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.807392    2563 scope.go:117] "RemoveContainer" containerID="4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"
Jul 29 01:00:34.814026 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.814011    2563 scope.go:117] "RemoveContainer" containerID="c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"
Jul 29 01:00:34.820721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.820697    2563 scope.go:117] "RemoveContainer" containerID="f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"
Jul 29 01:00:34.821596 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.821573    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:34.826930 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.826907    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:34.828215 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.828201    2563 scope.go:117] "RemoveContainer" containerID="86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"
Jul 29 01:00:34.838890 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.838862    2563 scope.go:117] "RemoveContainer" containerID="eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"
Jul 29 01:00:34.845659 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.845643    2563 scope.go:117] "RemoveContainer" containerID="800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"
Jul 29 01:00:34.848547 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848531    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:34.848807 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848795    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-web"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848808    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-web"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848816    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="prom-label-proxy"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848822    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="prom-label-proxy"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848833    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" containerName="console"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848838    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" containerName="console"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848846    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848852    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy"
Jul 29 01:00:34.848856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848858    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="alertmanager"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848863    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="alertmanager"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848870    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-metric"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848875    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-metric"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848883    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="config-reloader"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848888    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="config-reloader"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848896    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="init-config-reloader"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848900    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="init-config-reloader"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848954    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-web"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848961    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy-metric"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848967    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="config-reloader"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848975    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="prom-label-proxy"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848981    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a10bb30-a2fa-4a32-8ff4-176263582bb9" containerName="console"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848987    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="kube-rbac-proxy"
Jul 29 01:00:34.849080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.848992    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" containerName="alertmanager"
Jul 29 01:00:34.852480 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.852459    2563 scope.go:117] "RemoveContainer" containerID="ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"
Jul 29 01:00:34.852702 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.852682    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62\": container with ID starting with ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62 not found: ID does not exist" containerID="ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"
Jul 29 01:00:34.852747 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.852713    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"} err="failed to get container status \"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62\": rpc error: code = NotFound desc = could not find container \"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62\": container with ID starting with ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62 not found: ID does not exist"
Jul 29 01:00:34.852747 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.852734    2563 scope.go:117] "RemoveContainer" containerID="4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"
Jul 29 01:00:34.852928 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.852913    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4\": container with ID starting with 4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4 not found: ID does not exist" containerID="4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"
Jul 29 01:00:34.852966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.852932    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"} err="failed to get container status \"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4\": rpc error: code = NotFound desc = could not find container \"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4\": container with ID starting with 4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4 not found: ID does not exist"
Jul 29 01:00:34.852966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.852944    2563 scope.go:117] "RemoveContainer" containerID="c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"
Jul 29 01:00:34.853165 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.853148    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b\": container with ID starting with c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b not found: ID does not exist" containerID="c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"
Jul 29 01:00:34.853213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853171    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"} err="failed to get container status \"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b\": rpc error: code = NotFound desc = could not find container \"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b\": container with ID starting with c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b not found: ID does not exist"
Jul 29 01:00:34.853213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853187    2563 scope.go:117] "RemoveContainer" containerID="f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"
Jul 29 01:00:34.853436 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.853417    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90\": container with ID starting with f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90 not found: ID does not exist" containerID="f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"
Jul 29 01:00:34.853477 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853440    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"} err="failed to get container status \"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90\": rpc error: code = NotFound desc = could not find container \"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90\": container with ID starting with f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90 not found: ID does not exist"
Jul 29 01:00:34.853477 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853456    2563 scope.go:117] "RemoveContainer" containerID="86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"
Jul 29 01:00:34.853663 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.853643    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72\": container with ID starting with 86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72 not found: ID does not exist" containerID="86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"
Jul 29 01:00:34.853719 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853672    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"} err="failed to get container status \"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72\": rpc error: code = NotFound desc = could not find container \"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72\": container with ID starting with 86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72 not found: ID does not exist"
Jul 29 01:00:34.853719 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853693    2563 scope.go:117] "RemoveContainer" containerID="eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"
Jul 29 01:00:34.853834 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853819    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.853943 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.853923    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303\": container with ID starting with eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303 not found: ID does not exist" containerID="eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"
Jul 29 01:00:34.854000 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853958    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"} err="failed to get container status \"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303\": rpc error: code = NotFound desc = could not find container \"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303\": container with ID starting with eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303 not found: ID does not exist"
Jul 29 01:00:34.854000 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.853975    2563 scope.go:117] "RemoveContainer" containerID="800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"
Jul 29 01:00:34.854414 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:00:34.854393    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7\": container with ID starting with 800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7 not found: ID does not exist" containerID="800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"
Jul 29 01:00:34.854518 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854422    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"} err="failed to get container status \"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7\": rpc error: code = NotFound desc = could not find container \"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7\": container with ID starting with 800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7 not found: ID does not exist"
Jul 29 01:00:34.854518 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854442    2563 scope.go:117] "RemoveContainer" containerID="ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"
Jul 29 01:00:34.854695 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854678    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62"} err="failed to get container status \"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62\": rpc error: code = NotFound desc = could not find container \"ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62\": container with ID starting with ccded64811517e7702eccd1b95240e7ba195d7a43c07a65dd12a74f1c8a14e62 not found: ID does not exist"
Jul 29 01:00:34.854746 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854696    2563 scope.go:117] "RemoveContainer" containerID="4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"
Jul 29 01:00:34.854914 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854894    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4"} err="failed to get container status \"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4\": rpc error: code = NotFound desc = could not find container \"4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4\": container with ID starting with 4b1cce09ed0eb790b731db80eb27c941968b55eb54a2a2489ece86b1c00332c4 not found: ID does not exist"
Jul 29 01:00:34.854973 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.854915    2563 scope.go:117] "RemoveContainer" containerID="c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"
Jul 29 01:00:34.855162 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855144    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b"} err="failed to get container status \"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b\": rpc error: code = NotFound desc = could not find container \"c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b\": container with ID starting with c412952a10b8b1a4c9ea3af9fa52ab69ef9a36044b0c59d5a1e1ad43d41fc60b not found: ID does not exist"
Jul 29 01:00:34.855215 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855163    2563 scope.go:117] "RemoveContainer" containerID="f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"
Jul 29 01:00:34.855406 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855388    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90"} err="failed to get container status \"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90\": rpc error: code = NotFound desc = could not find container \"f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90\": container with ID starting with f90062ebddd4c27ef5cc46b4abf034463284cf79a593e810d0de812f6772bf90 not found: ID does not exist"
Jul 29 01:00:34.855480 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855407    2563 scope.go:117] "RemoveContainer" containerID="86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"
Jul 29 01:00:34.855650 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855621    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72"} err="failed to get container status \"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72\": rpc error: code = NotFound desc = could not find container \"86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72\": container with ID starting with 86b812ce79dda069b448cfae71b4344d30ceff138f218033b19f5e113c07ff72 not found: ID does not exist"
Jul 29 01:00:34.855650 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855649    2563 scope.go:117] "RemoveContainer" containerID="eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"
Jul 29 01:00:34.855868 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855850    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303"} err="failed to get container status \"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303\": rpc error: code = NotFound desc = could not find container \"eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303\": container with ID starting with eb2cb7668ea91bc099eda93478fda382e733220ccb089458d32350a3eef72303 not found: ID does not exist"
Jul 29 01:00:34.855868 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.855868    2563 scope.go:117] "RemoveContainer" containerID="800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"
Jul 29 01:00:34.856082 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856067    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7"} err="failed to get container status \"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7\": rpc error: code = NotFound desc = could not find container \"800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7\": container with ID starting with 800ba81c8c19f0c3b3a4244bd8b4dd4bdb2a878869f80dc8cb3477ccc03493f7 not found: ID does not exist"
Jul 29 01:00:34.856758 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856738    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 01:00:34.856858 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856784    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 01:00:34.856858 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856741    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 01:00:34.856985 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856875    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 01:00:34.856985 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x5gsh\""
Jul 29 01:00:34.856985 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 01:00:34.856985 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.856920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 01:00:34.857168 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.857037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 01:00:34.857242 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.857224    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 01:00:34.861992 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.861971    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 01:00:34.867072 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.867051    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:34.981615 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981615 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-config-out\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981615 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-web-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.981828 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-config-volume\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.982016 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4blkf\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-kube-api-access-4blkf\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.982016 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.982016 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:34.982016 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:34.981927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082332 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-config-out\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082332 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082538 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082538 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082538 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-web-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-config-volume\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4blkf\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-kube-api-access-4blkf\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.082904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.082830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.083246 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.083218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.083487 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.083462    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/855b1250-9f85-4266-900f-bc0755aad237-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.085426 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.085368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-web-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.085426 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.085368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-tls-assets\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.085426 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.085414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/855b1250-9f85-4266-900f-bc0755aad237-config-out\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.085617 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.085563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.085961 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.085936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.086076 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.086060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-config-volume\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.086329 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.086312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.086477 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.086456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.087029 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.087014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/855b1250-9f85-4266-900f-bc0755aad237-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.091225 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.091204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4blkf\" (UniqueName: \"kubernetes.io/projected/855b1250-9f85-4266-900f-bc0755aad237-kube-api-access-4blkf\") pod \"alertmanager-main-0\" (UID: \"855b1250-9f85-4266-900f-bc0755aad237\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.164094 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.164066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 01:00:35.293797 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.293768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 01:00:35.296838 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:00:35.296815    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855b1250_9f85_4266_900f_bc0755aad237.slice/crio-2d7b9615d2d8ac51e231bc83a2232d5ad5cd536568bc1bedda942147d16feeec WatchSource:0}: Error finding container 2d7b9615d2d8ac51e231bc83a2232d5ad5cd536568bc1bedda942147d16feeec: Status 404 returned error can't find the container with id 2d7b9615d2d8ac51e231bc83a2232d5ad5cd536568bc1bedda942147d16feeec
Jul 29 01:00:35.473810 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.473782    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4969ef26-db89-401c-9d81-6868fa45ca5a" path="/var/lib/kubelet/pods/4969ef26-db89-401c-9d81-6868fa45ca5a/volumes"
Jul 29 01:00:35.802845 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.802813    2563 generic.go:356] "Generic (PLEG): container finished" podID="855b1250-9f85-4266-900f-bc0755aad237" containerID="8cb53dd51867531ccdf9bc9270d6bae97baf31dcfcc6004620cc0cfb3af91f07" exitCode=0
Jul 29 01:00:35.803320 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.802896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerDied","Data":"8cb53dd51867531ccdf9bc9270d6bae97baf31dcfcc6004620cc0cfb3af91f07"}
Jul 29 01:00:35.803320 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:35.802962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"2d7b9615d2d8ac51e231bc83a2232d5ad5cd536568bc1bedda942147d16feeec"}
Jul 29 01:00:36.809581 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"b2c91abef316753c853032aff6ef1cc07f90c4c061f68a94666e412bc901959b"}
Jul 29 01:00:36.810012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"70a5410422646abdb405849262c8d88efd1aefb9c61647846b38e5dcd0aefdd3"}
Jul 29 01:00:36.810012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"24c7c659cd1948d4cb073e4e14271ff18160356a2985838aa38571125caddb74"}
Jul 29 01:00:36.810012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"61cdc765fc66977a82e31ff641daa0a467f1a5b70e621fb0801014f1811a61ea"}
Jul 29 01:00:36.810012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"e728ea3b3a5612341bb9493fbf62dda60febd053f01ac8cac268a9de226e90a9"}
Jul 29 01:00:36.810012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.809739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"855b1250-9f85-4266-900f-bc0755aad237","Type":"ContainerStarted","Data":"596cde03c2abdff0bb58c1c21023473bdf1a7f4a4fa517781a03fa8ebac175e4"}
Jul 29 01:00:36.838798 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:36.838751    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.817817033 podStartE2EDuration="2.838733274s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.916241ms" isStatefulPod=false podCreationTimestamp="2026-07-29 01:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:36.83822981 +0000 UTC m=+371.916216850" watchObservedRunningTime="2026-07-29 01:00:36.838733274 +0000 UTC m=+371.916720273"
Jul 29 01:00:37.483647 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.483616    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:00:37.486801 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.486780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.498176 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.498153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:00:37.601809 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr2fh\" (UniqueName: \"kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.601809 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.601979 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.601979 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.601979 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.601979 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.601942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.602115 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.602040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.702981 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.702950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.702988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703358 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703358 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr2fh\" (UniqueName: \"kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703794 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703880 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.703996 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.703974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.704063 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.704048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.705532 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.705510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.705674 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.705652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.711025 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.711000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr2fh\" (UniqueName: \"kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh\") pod \"console-6b5897785d-hvrng\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") " pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.796231 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.796163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:37.920966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:37.920934    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:00:37.923788 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:00:37.923757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod695df1b3_ed81_44b3_a333_c1d84fdd4547.slice/crio-f5a131621e802b59be15e9ba52200983ee62c9fbce5a437ba1c59c4f4ff5c006 WatchSource:0}: Error finding container f5a131621e802b59be15e9ba52200983ee62c9fbce5a437ba1c59c4f4ff5c006: Status 404 returned error can't find the container with id f5a131621e802b59be15e9ba52200983ee62c9fbce5a437ba1c59c4f4ff5c006
Jul 29 01:00:38.819784 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:38.819747    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5897785d-hvrng" event={"ID":"695df1b3-ed81-44b3-a333-c1d84fdd4547","Type":"ContainerStarted","Data":"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"}
Jul 29 01:00:38.819995 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:38.819797    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5897785d-hvrng" event={"ID":"695df1b3-ed81-44b3-a333-c1d84fdd4547","Type":"ContainerStarted","Data":"f5a131621e802b59be15e9ba52200983ee62c9fbce5a437ba1c59c4f4ff5c006"}
Jul 29 01:00:38.838044 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:38.837985    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b5897785d-hvrng" podStartSLOduration=1.8379704540000001 podStartE2EDuration="1.837970454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:38.83734491 +0000 UTC m=+373.915331909" watchObservedRunningTime="2026-07-29 01:00:38.837970454 +0000 UTC m=+373.915957453"
Jul 29 01:00:39.314266 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.314235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 01:00:39.316493 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.316473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb66aad-89ed-482e-8d49-abefa4390ab9-metrics-certs\") pod \"network-metrics-daemon-ckvff\" (UID: \"bcb66aad-89ed-482e-8d49-abefa4390ab9\") " pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 01:00:39.375802 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.375778    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxdbf\""
Jul 29 01:00:39.384005 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.383990    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ckvff"
Jul 29 01:00:39.506233 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.506205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ckvff"]
Jul 29 01:00:39.509663 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:00:39.509637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb66aad_89ed_482e_8d49_abefa4390ab9.slice/crio-2c93f375baf099ead65797aa87fb951074e8278dcb874741e3eaca316edc87ec WatchSource:0}: Error finding container 2c93f375baf099ead65797aa87fb951074e8278dcb874741e3eaca316edc87ec: Status 404 returned error can't find the container with id 2c93f375baf099ead65797aa87fb951074e8278dcb874741e3eaca316edc87ec
Jul 29 01:00:39.824945 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:39.824898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckvff" event={"ID":"bcb66aad-89ed-482e-8d49-abefa4390ab9","Type":"ContainerStarted","Data":"2c93f375baf099ead65797aa87fb951074e8278dcb874741e3eaca316edc87ec"}
Jul 29 01:00:40.829997 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:40.829908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckvff" event={"ID":"bcb66aad-89ed-482e-8d49-abefa4390ab9","Type":"ContainerStarted","Data":"c79d7097de0b6cd4057e14fa7af15a6cea84e54facf33dd6f21139daf65f6bc8"}
Jul 29 01:00:40.829997 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:40.829967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ckvff" event={"ID":"bcb66aad-89ed-482e-8d49-abefa4390ab9","Type":"ContainerStarted","Data":"9212b96a247aa957ab72ec208672719bfdc04497a733544b3ffe17c693f5da2e"}
Jul 29 01:00:40.845820 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:40.845776    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ckvff" podStartSLOduration=374.962507571 podStartE2EDuration="6m15.845760601s" totalImagesPullingTime="883.25303ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:40.844232813 +0000 UTC m=+375.922219814" watchObservedRunningTime="2026-07-29 01:00:40.845760601 +0000 UTC m=+375.923747600"
Jul 29 01:00:47.796433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:47.796393    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:47.796868 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:47.796459    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:47.801470 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:47.801445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:47.859997 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:47.859968    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:00:47.904509 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:00:47.904479    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 01:01:12.929030 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:12.928992    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-748c96d796-df9qd" podUID="39bb4c79-9338-4990-b2ee-4977add8f813" containerName="console" containerID="cri-o://4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67" gracePeriod=15
Jul 29 01:01:13.182464 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.182410    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 01:01:13.257267 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257289    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257311    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257497 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgmk2\" (UniqueName: \"kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257497 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257676 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257676 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config\") pod \"39bb4c79-9338-4990-b2ee-4977add8f813\" (UID: \"39bb4c79-9338-4990-b2ee-4977add8f813\") "
Jul 29 01:01:13.257676 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257625    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config" (OuterVolumeSpecName: "console-config") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:13.257676 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca" (OuterVolumeSpecName: "service-ca") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:13.257676 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:13.257940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257859    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-console-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.257940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257871    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.257940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257879    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-oauth-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.257940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.257906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:01:13.259619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.259583    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:01:13.259727 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.259614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:01:13.259727 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.259633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2" (OuterVolumeSpecName: "kube-api-access-lgmk2") pod "39bb4c79-9338-4990-b2ee-4977add8f813" (UID: "39bb4c79-9338-4990-b2ee-4977add8f813"). InnerVolumeSpecName "kube-api-access-lgmk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:01:13.358247 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.358215    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.358247 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.358241    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39bb4c79-9338-4990-b2ee-4977add8f813-console-oauth-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.358247 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.358254    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgmk2\" (UniqueName: \"kubernetes.io/projected/39bb4c79-9338-4990-b2ee-4977add8f813-kube-api-access-lgmk2\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.358458 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.358289    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bb4c79-9338-4990-b2ee-4977add8f813-trusted-ca-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:01:13.958544 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.958507    2563 generic.go:356] "Generic (PLEG): container finished" podID="39bb4c79-9338-4990-b2ee-4977add8f813" containerID="4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67" exitCode=0
Jul 29 01:01:13.959036 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.958556    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-748c96d796-df9qd"
Jul 29 01:01:13.959036 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.958599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748c96d796-df9qd" event={"ID":"39bb4c79-9338-4990-b2ee-4977add8f813","Type":"ContainerDied","Data":"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"}
Jul 29 01:01:13.959036 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.958655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-748c96d796-df9qd" event={"ID":"39bb4c79-9338-4990-b2ee-4977add8f813","Type":"ContainerDied","Data":"810da998c08b1967e206d07c8aab6774b8cdb4f5606856e2bad5c3ef5514c4ab"}
Jul 29 01:01:13.959036 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.958678    2563 scope.go:117] "RemoveContainer" containerID="4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"
Jul 29 01:01:13.966668 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.966654    2563 scope.go:117] "RemoveContainer" containerID="4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"
Jul 29 01:01:13.966920 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:01:13.966901    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67\": container with ID starting with 4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67 not found: ID does not exist" containerID="4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"
Jul 29 01:01:13.966994 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.966928    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67"} err="failed to get container status \"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67\": rpc error: code = NotFound desc = could not find container \"4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67\": container with ID starting with 4d95d1f21a06803edbd2c25ea786d6691d2979532146a9272a543cf87d29fa67 not found: ID does not exist"
Jul 29 01:01:13.974823 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.974798    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 01:01:13.978054 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:13.978036    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-748c96d796-df9qd"]
Jul 29 01:01:15.474104 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:01:15.474071    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39bb4c79-9338-4990-b2ee-4977add8f813" path="/var/lib/kubelet/pods/39bb4c79-9338-4990-b2ee-4977add8f813/volumes"
Jul 29 01:02:08.552172 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.552140    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"]
Jul 29 01:02:08.553204 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.553182    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39bb4c79-9338-4990-b2ee-4977add8f813" containerName="console"
Jul 29 01:02:08.553365 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.553351    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="39bb4c79-9338-4990-b2ee-4977add8f813" containerName="console"
Jul 29 01:02:08.553520 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.553509    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="39bb4c79-9338-4990-b2ee-4977add8f813" containerName="console"
Jul 29 01:02:08.573126 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.573102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"]
Jul 29 01:02:08.573279 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.573192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.575941 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.575915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:02:08.576065 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.576029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vph5x\""
Jul 29 01:02:08.576124 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.576091    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 01:02:08.669366 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.669342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ctg6\" (UniqueName: \"kubernetes.io/projected/e8b75578-4db3-4994-ac63-f511d1270b99-kube-api-access-2ctg6\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.669480 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.669374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8b75578-4db3-4994-ac63-f511d1270b99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.770174 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.770149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ctg6\" (UniqueName: \"kubernetes.io/projected/e8b75578-4db3-4994-ac63-f511d1270b99-kube-api-access-2ctg6\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.770286 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.770179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8b75578-4db3-4994-ac63-f511d1270b99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.770522 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.770502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8b75578-4db3-4994-ac63-f511d1270b99-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.778146 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.778127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ctg6\" (UniqueName: \"kubernetes.io/projected/e8b75578-4db3-4994-ac63-f511d1270b99-kube-api-access-2ctg6\") pod \"cert-manager-operator-controller-manager-55c444bf86-w62vm\" (UID: \"e8b75578-4db3-4994-ac63-f511d1270b99\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:08.882839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:08.882818    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"
Jul 29 01:02:09.018480 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:09.018448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm"]
Jul 29 01:02:09.021764 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:02:09.021741    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b75578_4db3_4994_ac63_f511d1270b99.slice/crio-27cd0bfc1d3b4a0f6b91eef436517f5ccd94eb1f04bf4624518d305221be0b0f WatchSource:0}: Error finding container 27cd0bfc1d3b4a0f6b91eef436517f5ccd94eb1f04bf4624518d305221be0b0f: Status 404 returned error can't find the container with id 27cd0bfc1d3b4a0f6b91eef436517f5ccd94eb1f04bf4624518d305221be0b0f
Jul 29 01:02:09.170058 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:09.169972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm" event={"ID":"e8b75578-4db3-4994-ac63-f511d1270b99","Type":"ContainerStarted","Data":"27cd0bfc1d3b4a0f6b91eef436517f5ccd94eb1f04bf4624518d305221be0b0f"}
Jul 29 01:02:14.193577 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:14.193538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm" event={"ID":"e8b75578-4db3-4994-ac63-f511d1270b99","Type":"ContainerStarted","Data":"2b1544a7abe64cd234f547084cca9a6686d89c3a61e56109ec8431336c0a9e01"}
Jul 29 01:02:14.218499 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:14.218448    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-w62vm" podStartSLOduration=1.962129736 podStartE2EDuration="6.218434748s" totalImagesPullingTime="4.256305012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:02:14.21634878 +0000 UTC m=+469.294335770" watchObservedRunningTime="2026-07-29 01:02:14.218434748 +0000 UTC m=+469.296421746"
Jul 29 01:02:15.866165 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.866127    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"]
Jul 29 01:02:15.869890 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.869873    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:15.872543 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.872521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6bqq\""
Jul 29 01:02:15.872652 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.872543    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 01:02:15.872652 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.872564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 01:02:15.877530 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.877507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"]
Jul 29 01:02:15.927948 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.927915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:15.928092 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.927980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw7sl\" (UniqueName: \"kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:15.928092 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:15.928015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.028912 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.028876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw7sl\" (UniqueName: \"kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.028912 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.028913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.029125 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.028965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.029352 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.029331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.029419 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.029348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.037013 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.036995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw7sl\" (UniqueName: \"kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.179200 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.179138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:16.308824 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:16.308800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"]
Jul 29 01:02:16.310543 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:02:16.310516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fe7e9f_da03_4a29_9adf_3015cbedeb96.slice/crio-2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df WatchSource:0}: Error finding container 2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df: Status 404 returned error can't find the container with id 2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df
Jul 29 01:02:17.209128 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:17.209093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerStarted","Data":"2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df"}
Jul 29 01:02:22.235835 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:22.235800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerStarted","Data":"2033401272b43b376ca7fa494c5926fc106658598a813edee9970a1ee4978449"}
Jul 29 01:02:23.241564 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:23.241529    2563 generic.go:356] "Generic (PLEG): container finished" podID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerID="2033401272b43b376ca7fa494c5926fc106658598a813edee9970a1ee4978449" exitCode=0
Jul 29 01:02:23.241958 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:23.241615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerDied","Data":"2033401272b43b376ca7fa494c5926fc106658598a813edee9970a1ee4978449"}
Jul 29 01:02:25.500392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.500357    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7g7nn"]
Jul 29 01:02:25.503636 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.503616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.506171 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.506141    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 01:02:25.506315 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.506186    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f54tl\""
Jul 29 01:02:25.506315 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.506284    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 01:02:25.514112 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.514086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7g7nn"]
Jul 29 01:02:25.611508 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.611481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlbzg\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-kube-api-access-xlbzg\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.611643 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.611517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.712919 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.712883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlbzg\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-kube-api-access-xlbzg\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.713097 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.712936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.722454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.722429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.722599 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.722520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlbzg\" (UniqueName: \"kubernetes.io/projected/13db783d-b072-49fb-84ee-7a370e2ea2d2-kube-api-access-xlbzg\") pod \"cert-manager-55b4d954d5-7g7nn\" (UID: \"13db783d-b072-49fb-84ee-7a370e2ea2d2\") " pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.815251 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.815171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7g7nn"
Jul 29 01:02:25.945831 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:25.945802    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7g7nn"]
Jul 29 01:02:26.556816 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:02:26.556763    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13db783d_b072_49fb_84ee_7a370e2ea2d2.slice/crio-6eec6d6935f0d30120a3fdc563dd51c2b3f2c12c8b999a4d7c52922a6eae71b5 WatchSource:0}: Error finding container 6eec6d6935f0d30120a3fdc563dd51c2b3f2c12c8b999a4d7c52922a6eae71b5: Status 404 returned error can't find the container with id 6eec6d6935f0d30120a3fdc563dd51c2b3f2c12c8b999a4d7c52922a6eae71b5
Jul 29 01:02:27.261143 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:27.261099    2563 generic.go:356] "Generic (PLEG): container finished" podID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerID="66c4cc3509679d6597da1c5386f04c01e8ed4a24a45bb78ef571788a4bfaaf27" exitCode=0
Jul 29 01:02:27.261336 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:27.261181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerDied","Data":"66c4cc3509679d6597da1c5386f04c01e8ed4a24a45bb78ef571788a4bfaaf27"}
Jul 29 01:02:27.262492 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:27.262463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7g7nn" event={"ID":"13db783d-b072-49fb-84ee-7a370e2ea2d2","Type":"ContainerStarted","Data":"6eec6d6935f0d30120a3fdc563dd51c2b3f2c12c8b999a4d7c52922a6eae71b5"}
Jul 29 01:02:34.301069 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:34.300982    2563 generic.go:356] "Generic (PLEG): container finished" podID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerID="f5bd73ed8c08a772f38eeb9f2a13d24ae1cb82ca1b6d0dc804aa779b1390dd6f" exitCode=0
Jul 29 01:02:34.301473 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:34.301075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerDied","Data":"f5bd73ed8c08a772f38eeb9f2a13d24ae1cb82ca1b6d0dc804aa779b1390dd6f"}
Jul 29 01:02:34.302644 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:34.302616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7g7nn" event={"ID":"13db783d-b072-49fb-84ee-7a370e2ea2d2","Type":"ContainerStarted","Data":"594adcde537922fa8d71ba6c8b56132513d1bb39045d1dedac007d76f1238724"}
Jul 29 01:02:34.332889 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:34.332846    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7g7nn" podStartSLOduration=1.918269206 podStartE2EDuration="9.332830716s" totalImagesPullingTime="7.41456151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:02:34.331143586 +0000 UTC m=+489.409130584" watchObservedRunningTime="2026-07-29 01:02:34.332830716 +0000 UTC m=+489.410817714"
Jul 29 01:02:35.451952 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.451927    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:35.599123 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.599094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util\") pod \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") "
Jul 29 01:02:35.599312 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.599139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle\") pod \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") "
Jul 29 01:02:35.599312 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.599188    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw7sl\" (UniqueName: \"kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl\") pod \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\" (UID: \"b5fe7e9f-da03-4a29-9adf-3015cbedeb96\") "
Jul 29 01:02:35.599554 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.599526    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle" (OuterVolumeSpecName: "bundle") pod "b5fe7e9f-da03-4a29-9adf-3015cbedeb96" (UID: "b5fe7e9f-da03-4a29-9adf-3015cbedeb96"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:02:35.601563 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.601541    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl" (OuterVolumeSpecName: "kube-api-access-pw7sl") pod "b5fe7e9f-da03-4a29-9adf-3015cbedeb96" (UID: "b5fe7e9f-da03-4a29-9adf-3015cbedeb96"). InnerVolumeSpecName "kube-api-access-pw7sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:02:35.603019 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.602998    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util" (OuterVolumeSpecName: "util") pod "b5fe7e9f-da03-4a29-9adf-3015cbedeb96" (UID: "b5fe7e9f-da03-4a29-9adf-3015cbedeb96"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:02:35.699938 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.699908    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-util\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:02:35.699938 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.699937    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:02:35.699938 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:35.699946    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw7sl\" (UniqueName: \"kubernetes.io/projected/b5fe7e9f-da03-4a29-9adf-3015cbedeb96-kube-api-access-pw7sl\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:02:36.311974 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:36.311945    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc"
Jul 29 01:02:36.311974 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:36.311966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftl4vc" event={"ID":"b5fe7e9f-da03-4a29-9adf-3015cbedeb96","Type":"ContainerDied","Data":"2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df"}
Jul 29 01:02:36.312167 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:36.312012    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f8a5dccab952c74e116cd14d6ae337721a644f0a2a4f70fd93f0a569a53b3df"
Jul 29 01:02:39.861832 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.861799    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"]
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862116    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="pull"
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862128    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="pull"
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862145    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="util"
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862151    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="util"
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862158    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="extract"
Jul 29 01:02:39.862192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862164    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="extract"
Jul 29 01:02:39.862394 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.862225    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5fe7e9f-da03-4a29-9adf-3015cbedeb96" containerName="extract"
Jul 29 01:02:39.866935 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.866912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:39.869790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.869769    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 01:02:39.871137 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.871117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 01:02:39.871246 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.871180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-r7gv4\""
Jul 29 01:02:39.874480 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.874458    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"]
Jul 29 01:02:39.934585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.934551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75bmr\" (UniqueName: \"kubernetes.io/projected/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-kube-api-access-75bmr\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:39.934585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:39.934589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.035407 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.035371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75bmr\" (UniqueName: \"kubernetes.io/projected/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-kube-api-access-75bmr\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.035628 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.035426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.035821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.035802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.043753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.043728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75bmr\" (UniqueName: \"kubernetes.io/projected/a9b16b44-5eb8-433a-8e8e-d50e9cbe1323-kube-api-access-75bmr\") pod \"openshift-lws-operator-fd8ccff4c-ftm69\" (UID: \"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.176320 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.176215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"
Jul 29 01:02:40.305064 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.305005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69"]
Jul 29 01:02:40.307176 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:02:40.307148    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9b16b44_5eb8_433a_8e8e_d50e9cbe1323.slice/crio-2ea9714586aa4b2f31c8fbf66a3694980f3cfa3141e555a5ecb8aedde7567756 WatchSource:0}: Error finding container 2ea9714586aa4b2f31c8fbf66a3694980f3cfa3141e555a5ecb8aedde7567756: Status 404 returned error can't find the container with id 2ea9714586aa4b2f31c8fbf66a3694980f3cfa3141e555a5ecb8aedde7567756
Jul 29 01:02:40.329314 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:40.329287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69" event={"ID":"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323","Type":"ContainerStarted","Data":"2ea9714586aa4b2f31c8fbf66a3694980f3cfa3141e555a5ecb8aedde7567756"}
Jul 29 01:02:44.349367 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:44.349334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69" event={"ID":"a9b16b44-5eb8-433a-8e8e-d50e9cbe1323","Type":"ContainerStarted","Data":"b5e0a10feed06fc43e4f82edce99dc0f879ff863c1fa413d691e93a10737bb11"}
Jul 29 01:02:44.366348 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:44.366303    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftm69" podStartSLOduration=2.269629126 podStartE2EDuration="5.366288994s" totalImagesPullingTime="3.096659868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:02:44.364036237 +0000 UTC m=+499.442023266" watchObservedRunningTime="2026-07-29 01:02:44.366288994 +0000 UTC m=+499.444275992"
Jul 29 01:02:59.406939 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.406902    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"]
Jul 29 01:02:59.409245 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.409229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.413548 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.413523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rzm24\""
Jul 29 01:02:59.413677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.413560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 01:02:59.413677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.413523    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 01:02:59.413677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.413644    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 01:02:59.421760 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.421733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"]
Jul 29 01:02:59.593545 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.593508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.593545 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.593547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-metrics-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.593768 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.593637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkkx\" (UniqueName: \"kubernetes.io/projected/c51e8358-1435-435c-9e54-2052b2a67539-kube-api-access-8dkkx\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.593768 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.593701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c51e8358-1435-435c-9e54-2052b2a67539-manager-config\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.695068 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.694987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkkx\" (UniqueName: \"kubernetes.io/projected/c51e8358-1435-435c-9e54-2052b2a67539-kube-api-access-8dkkx\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.695068 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.695028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c51e8358-1435-435c-9e54-2052b2a67539-manager-config\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.695068 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.695061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.695335 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.695079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-metrics-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.695709 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.695683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c51e8358-1435-435c-9e54-2052b2a67539-manager-config\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.697553 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.697525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-metrics-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.697761 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.697738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c51e8358-1435-435c-9e54-2052b2a67539-cert\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.707223 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.707200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkkx\" (UniqueName: \"kubernetes.io/projected/c51e8358-1435-435c-9e54-2052b2a67539-kube-api-access-8dkkx\") pod \"lws-controller-manager-6c7bdf789-lpl7c\" (UID: \"c51e8358-1435-435c-9e54-2052b2a67539\") " pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.720033 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.720008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:02:59.848716 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:02:59.848691    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"]
Jul 29 01:02:59.850843 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:02:59.850822    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51e8358_1435_435c_9e54_2052b2a67539.slice/crio-2dfcff29b743a8a393066a9a88ffc8ffe91d4adadef86e484b8d205afb5709af WatchSource:0}: Error finding container 2dfcff29b743a8a393066a9a88ffc8ffe91d4adadef86e484b8d205afb5709af: Status 404 returned error can't find the container with id 2dfcff29b743a8a393066a9a88ffc8ffe91d4adadef86e484b8d205afb5709af
Jul 29 01:03:00.420305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.420252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c" event={"ID":"c51e8358-1435-435c-9e54-2052b2a67539","Type":"ContainerStarted","Data":"2dfcff29b743a8a393066a9a88ffc8ffe91d4adadef86e484b8d205afb5709af"}
Jul 29 01:03:00.467130 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.467096    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"]
Jul 29 01:03:00.470171 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.470145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.473380 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.473356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 01:03:00.473475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.473426    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-88xjl\""
Jul 29 01:03:00.473475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.473430    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 01:03:00.473563 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.473513    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 01:03:00.473647 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.473630    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 01:03:00.489815 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.489795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"]
Jul 29 01:03:00.601573 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.601546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.601705 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.601586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.601705 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.601607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.601705 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.601685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.601811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.601721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6plbl\" (UniqueName: \"kubernetes.io/projected/822573e2-adf3-4665-a524-9ef642d7ef06-kube-api-access-6plbl\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.702856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.702768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.702856 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.702832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.703075 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.702861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.703075 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.702902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.703075 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.702981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6plbl\" (UniqueName: \"kubernetes.io/projected/822573e2-adf3-4665-a524-9ef642d7ef06-kube-api-access-6plbl\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.703374 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.703328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.705770 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.705745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/822573e2-adf3-4665-a524-9ef642d7ef06-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.705942 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.705912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.706129 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.706106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/822573e2-adf3-4665-a524-9ef642d7ef06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.717526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.717504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6plbl\" (UniqueName: \"kubernetes.io/projected/822573e2-adf3-4665-a524-9ef642d7ef06-kube-api-access-6plbl\") pod \"opendatahub-operator-controller-manager-795cff746b-qk9b8\" (UID: \"822573e2-adf3-4665-a524-9ef642d7ef06\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.780990 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.780958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:00.947337 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:00.944685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"]
Jul 29 01:03:00.947337 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:03:00.946732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822573e2_adf3_4665_a524_9ef642d7ef06.slice/crio-0e8871ec92ab0a3b7e22c6784d42c684d79b2692b6094f96da228130ef41d525 WatchSource:0}: Error finding container 0e8871ec92ab0a3b7e22c6784d42c684d79b2692b6094f96da228130ef41d525: Status 404 returned error can't find the container with id 0e8871ec92ab0a3b7e22c6784d42c684d79b2692b6094f96da228130ef41d525
Jul 29 01:03:01.426912 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:01.426871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8" event={"ID":"822573e2-adf3-4665-a524-9ef642d7ef06","Type":"ContainerStarted","Data":"0e8871ec92ab0a3b7e22c6784d42c684d79b2692b6094f96da228130ef41d525"}
Jul 29 01:03:02.433339 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:02.433301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c" event={"ID":"c51e8358-1435-435c-9e54-2052b2a67539","Type":"ContainerStarted","Data":"4de6c6a97c34211cf052a62c65e5aa114b1432c5a77891b1c736bd50557ed70b"}
Jul 29 01:03:02.433738 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:02.433412    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:03:02.453373 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:02.453328    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c" podStartSLOduration=1.546208124 podStartE2EDuration="3.453313223s" totalImagesPullingTime="1.907105099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:02.452840732 +0000 UTC m=+517.530827750" watchObservedRunningTime="2026-07-29 01:03:02.453313223 +0000 UTC m=+517.531300222"
Jul 29 01:03:05.449948 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:05.449850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8" event={"ID":"822573e2-adf3-4665-a524-9ef642d7ef06","Type":"ContainerStarted","Data":"c123ec08b59a384f2ea7b6a32712aa1be2a6b70835a551bebb0ad1cc1245fcb8"}
Jul 29 01:03:06.456200 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:06.456168    2563 generic.go:356] "Generic (PLEG): container finished" podID="822573e2-adf3-4665-a524-9ef642d7ef06" containerID="c123ec08b59a384f2ea7b6a32712aa1be2a6b70835a551bebb0ad1cc1245fcb8" exitCode=0
Jul 29 01:03:06.456597 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:06.456271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8" event={"ID":"822573e2-adf3-4665-a524-9ef642d7ef06","Type":"ContainerDied","Data":"c123ec08b59a384f2ea7b6a32712aa1be2a6b70835a551bebb0ad1cc1245fcb8"}
Jul 29 01:03:07.463172 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:07.463135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8" event={"ID":"822573e2-adf3-4665-a524-9ef642d7ef06","Type":"ContainerStarted","Data":"47593a045f4103764a655294ffb66130bdd08bc80f1246fbf1c742f07f631224"}
Jul 29 01:03:07.463620 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:07.463222    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:07.486360 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:07.486313    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8" podStartSLOduration=1.651035329 podStartE2EDuration="7.486298153s" totalImagesPullingTime="4.751441705s" totalInitContainerRuntime="1.083821119s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:07.484004302 +0000 UTC m=+522.561991304" watchObservedRunningTime="2026-07-29 01:03:07.486298153 +0000 UTC m=+522.564285152"
Jul 29 01:03:13.441753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:13.441718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c7bdf789-lpl7c"
Jul 29 01:03:18.469743 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:18.469664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-qk9b8"
Jul 29 01:03:21.394484 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.394452    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"]
Jul 29 01:03:21.396710 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.396689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.401176 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.401156    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 01:03:21.402400 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.402376    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-769kg\""
Jul 29 01:03:21.402514 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.402414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 01:03:21.402514 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.402437    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 01:03:21.403442 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.403285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 01:03:21.413152 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.413125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"]
Jul 29 01:03:21.453655 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.453624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqkc\" (UniqueName: \"kubernetes.io/projected/a0e30713-021f-4145-a2c0-88185779230c-kube-api-access-djqkc\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.453806 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.453665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e30713-021f-4145-a2c0-88185779230c-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.453806 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.453731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e30713-021f-4145-a2c0-88185779230c-tmp\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.554462 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.554412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e30713-021f-4145-a2c0-88185779230c-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.554627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.554508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e30713-021f-4145-a2c0-88185779230c-tmp\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.554627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.554561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqkc\" (UniqueName: \"kubernetes.io/projected/a0e30713-021f-4145-a2c0-88185779230c-kube-api-access-djqkc\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.556804 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.556770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e30713-021f-4145-a2c0-88185779230c-tmp\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.557051 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.557030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e30713-021f-4145-a2c0-88185779230c-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.564117 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.564096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqkc\" (UniqueName: \"kubernetes.io/projected/a0e30713-021f-4145-a2c0-88185779230c-kube-api-access-djqkc\") pod \"kube-auth-proxy-7b44fbcd-6nsqc\" (UID: \"a0e30713-021f-4145-a2c0-88185779230c\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.706464 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.706387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"
Jul 29 01:03:21.851366 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:03:21.851337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e30713_021f_4145_a2c0_88185779230c.slice/crio-1679d160b79714f1f6bdbaf2ea055ad7a63570524efb8837d3a3960c6b2b06cf WatchSource:0}: Error finding container 1679d160b79714f1f6bdbaf2ea055ad7a63570524efb8837d3a3960c6b2b06cf: Status 404 returned error can't find the container with id 1679d160b79714f1f6bdbaf2ea055ad7a63570524efb8837d3a3960c6b2b06cf
Jul 29 01:03:21.863814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:21.863788    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc"]
Jul 29 01:03:22.531616 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:22.531580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc" event={"ID":"a0e30713-021f-4145-a2c0-88185779230c","Type":"ContainerStarted","Data":"1679d160b79714f1f6bdbaf2ea055ad7a63570524efb8837d3a3960c6b2b06cf"}
Jul 29 01:03:24.545527 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:24.545489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc" event={"ID":"a0e30713-021f-4145-a2c0-88185779230c","Type":"ContainerStarted","Data":"969d1c574cff8ac6de58c4fe8fa6fe0ec6158233b4d2c68c205014e521579354"}
Jul 29 01:03:24.562952 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:03:24.562905    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-6nsqc" podStartSLOduration=0.979854566 podStartE2EDuration="3.562888797s" totalImagesPullingTime="2.583034231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:24.561479315 +0000 UTC m=+539.639466316" watchObservedRunningTime="2026-07-29 01:03:24.562888797 +0000 UTC m=+539.640875803"
Jul 29 01:04:34.730015 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.729977    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"]
Jul 29 01:04:34.733251 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.733234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.735861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.735840    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 01:04:34.735993 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.735941    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r7kqd\""
Jul 29 01:04:34.737129 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.737113    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 01:04:34.741461 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.741439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"]
Jul 29 01:04:34.854162 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.854125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.854340 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.854223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b84b\" (UniqueName: \"kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.854340 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.854304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.955584 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.955551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.955728 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.955610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b84b\" (UniqueName: \"kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.955728 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.955651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.956070 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.956048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.956111 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.956048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:34.964638 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:34.964609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b84b\" (UniqueName: \"kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:35.043327 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.043214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:35.169363 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.169324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"]
Jul 29 01:04:35.171816 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:04:35.171779    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acb9197_9efd_4d3c_9f65_c48d852e20cb.slice/crio-a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8 WatchSource:0}: Error finding container a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8: Status 404 returned error can't find the container with id a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8
Jul 29 01:04:35.327397 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.327317    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"]
Jul 29 01:04:35.331041 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.331019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.339932 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.339902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"]
Jul 29 01:04:35.460174 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.460140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.460355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.460247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hndg\" (UniqueName: \"kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.460355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.460309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.561627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.561595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hndg\" (UniqueName: \"kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.561812 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.561647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.561812 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.561704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.562064 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.562041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.562241 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.562224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.572134 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.572108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hndg\" (UniqueName: \"kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.640099 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.640069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:35.770315 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.770281    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"]
Jul 29 01:04:35.772365 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:04:35.772337    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f02d06_8362_4e3b_9d89_d2bc2084dcaa.slice/crio-469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a WatchSource:0}: Error finding container 469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a: Status 404 returned error can't find the container with id 469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a
Jul 29 01:04:35.863149 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.863099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerStarted","Data":"97b8c272486739b0303f43ce536d01cd12f09ebe89348f8a6ee9727200edcdef"}
Jul 29 01:04:35.863318 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.863177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerStarted","Data":"469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a"}
Jul 29 01:04:35.864925 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.864896    2563 generic.go:356] "Generic (PLEG): container finished" podID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerID="ea1143265d8932da649acc497a2f9197a37cafc69c946fb9777c969d1de45589" exitCode=0
Jul 29 01:04:35.865037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.864970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6" event={"ID":"3acb9197-9efd-4d3c-9f65-c48d852e20cb","Type":"ContainerDied","Data":"ea1143265d8932da649acc497a2f9197a37cafc69c946fb9777c969d1de45589"}
Jul 29 01:04:35.865037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.865015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6" event={"ID":"3acb9197-9efd-4d3c-9f65-c48d852e20cb","Type":"ContainerStarted","Data":"a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8"}
Jul 29 01:04:35.929174 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.929108    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"]
Jul 29 01:04:35.932602 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.932586    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:35.940603 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:35.940581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"]
Jul 29 01:04:36.065242 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.065211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.065418 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.065251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxwq4\" (UniqueName: \"kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.065418 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.065349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.165955 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.165903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.166117 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.165973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxwq4\" (UniqueName: \"kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.166117 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.166012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.166278 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.166239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.166325 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.166310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.174583 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.174560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxwq4\" (UniqueName: \"kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.242451 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.242365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:36.372482 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.372461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"]
Jul 29 01:04:36.374556 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:04:36.374529    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d746b96_312c_495e_9f56_0451d2577427.slice/crio-134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84 WatchSource:0}: Error finding container 134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84: Status 404 returned error can't find the container with id 134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84
Jul 29 01:04:36.531542 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.531467    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"]
Jul 29 01:04:36.534971 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.534943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.545225 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.545199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"]
Jul 29 01:04:36.670023 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.669996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.670137 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.670029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97jgh\" (UniqueName: \"kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.670137 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.670118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.771334 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.771242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.771626 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.771373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.771626 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.771394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97jgh\" (UniqueName: \"kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.771700 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.771627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.771700 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.771665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.780319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.780298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97jgh\" (UniqueName: \"kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.847225 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.847198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:36.873624 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.873592    2563 generic.go:356] "Generic (PLEG): container finished" podID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerID="97b8c272486739b0303f43ce536d01cd12f09ebe89348f8a6ee9727200edcdef" exitCode=0
Jul 29 01:04:36.873737 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.873680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerDied","Data":"97b8c272486739b0303f43ce536d01cd12f09ebe89348f8a6ee9727200edcdef"}
Jul 29 01:04:36.874292 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.874246    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:04:36.875291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.875243    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d746b96-312c-495e-9f56-0451d2577427" containerID="0ac6d7658a7e2991214f91fc349be5e5164d80dea3141e3bb3b5515f217bf637" exitCode=0
Jul 29 01:04:36.875410 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.875295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn" event={"ID":"2d746b96-312c-495e-9f56-0451d2577427","Type":"ContainerDied","Data":"0ac6d7658a7e2991214f91fc349be5e5164d80dea3141e3bb3b5515f217bf637"}
Jul 29 01:04:36.875410 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.875379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn" event={"ID":"2d746b96-312c-495e-9f56-0451d2577427","Type":"ContainerStarted","Data":"134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84"}
Jul 29 01:04:36.877728 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.877700    2563 generic.go:356] "Generic (PLEG): container finished" podID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerID="59bf9777018d11211479ad6aab04116850dbfb1315fc26cbe0b124152a4ec17f" exitCode=0
Jul 29 01:04:36.877808 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.877793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6" event={"ID":"3acb9197-9efd-4d3c-9f65-c48d852e20cb","Type":"ContainerDied","Data":"59bf9777018d11211479ad6aab04116850dbfb1315fc26cbe0b124152a4ec17f"}
Jul 29 01:04:36.987646 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:36.987551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"]
Jul 29 01:04:36.989757 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:04:36.989727    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e88861e_b7a1_4a2c_9283_dd4a5a983986.slice/crio-d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb WatchSource:0}: Error finding container d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb: Status 404 returned error can't find the container with id d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb
Jul 29 01:04:37.884857 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.884815    2563 generic.go:356] "Generic (PLEG): container finished" podID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerID="eebda9ea83be171cda46f3db926c6980a23c1e5e835cbc53771c3e7fc3f7c456" exitCode=0
Jul 29 01:04:37.885277 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.884894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerDied","Data":"eebda9ea83be171cda46f3db926c6980a23c1e5e835cbc53771c3e7fc3f7c456"}
Jul 29 01:04:37.887191 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.887164    2563 generic.go:356] "Generic (PLEG): container finished" podID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerID="2eab7b615bb583f28883c8742707e746606ea7728d0769460b9461496bd9d788" exitCode=0
Jul 29 01:04:37.887309 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.887241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6" event={"ID":"3acb9197-9efd-4d3c-9f65-c48d852e20cb","Type":"ContainerDied","Data":"2eab7b615bb583f28883c8742707e746606ea7728d0769460b9461496bd9d788"}
Jul 29 01:04:37.888641 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.888621    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerID="85bfd07a496e20eebc414e896d297f5e8f28f50cb9ce1311e72dd7df20749b97" exitCode=0
Jul 29 01:04:37.888729 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.888710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp" event={"ID":"8e88861e-b7a1-4a2c-9283-dd4a5a983986","Type":"ContainerDied","Data":"85bfd07a496e20eebc414e896d297f5e8f28f50cb9ce1311e72dd7df20749b97"}
Jul 29 01:04:37.888798 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:37.888760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp" event={"ID":"8e88861e-b7a1-4a2c-9283-dd4a5a983986","Type":"ContainerStarted","Data":"d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb"}
Jul 29 01:04:38.895956 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.895880    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerID="43aa1adc6dd4423b1c2b2384c3717fc2e0611a3bf59ee6b8006c02f14a2d7a14" exitCode=0
Jul 29 01:04:38.896304 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.895962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp" event={"ID":"8e88861e-b7a1-4a2c-9283-dd4a5a983986","Type":"ContainerDied","Data":"43aa1adc6dd4423b1c2b2384c3717fc2e0611a3bf59ee6b8006c02f14a2d7a14"}
Jul 29 01:04:38.898234 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.898211    2563 generic.go:356] "Generic (PLEG): container finished" podID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerID="3c2f7b3e51f088eed1fa3059637fcbef0882c5b5193b797c636b45d5269119ec" exitCode=0
Jul 29 01:04:38.898349 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.898301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerDied","Data":"3c2f7b3e51f088eed1fa3059637fcbef0882c5b5193b797c636b45d5269119ec"}
Jul 29 01:04:38.900039 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.900019    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d746b96-312c-495e-9f56-0451d2577427" containerID="4370386cb3e3b746d41a229103ad44b7fbaf9c369975d672802e3f2ca358d0ca" exitCode=0
Jul 29 01:04:38.900120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:38.900078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn" event={"ID":"2d746b96-312c-495e-9f56-0451d2577427","Type":"ContainerDied","Data":"4370386cb3e3b746d41a229103ad44b7fbaf9c369975d672802e3f2ca358d0ca"}
Jul 29 01:04:39.037959 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.037929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:39.193564 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.193481    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util\") pod \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") "
Jul 29 01:04:39.193564 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.193527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle\") pod \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") "
Jul 29 01:04:39.193769 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.193634    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b84b\" (UniqueName: \"kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b\") pod \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\" (UID: \"3acb9197-9efd-4d3c-9f65-c48d852e20cb\") "
Jul 29 01:04:39.194064 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.194039    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle" (OuterVolumeSpecName: "bundle") pod "3acb9197-9efd-4d3c-9f65-c48d852e20cb" (UID: "3acb9197-9efd-4d3c-9f65-c48d852e20cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:39.195777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.195754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b" (OuterVolumeSpecName: "kube-api-access-2b84b") pod "3acb9197-9efd-4d3c-9f65-c48d852e20cb" (UID: "3acb9197-9efd-4d3c-9f65-c48d852e20cb"). InnerVolumeSpecName "kube-api-access-2b84b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:04:39.198915 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.198894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util" (OuterVolumeSpecName: "util") pod "3acb9197-9efd-4d3c-9f65-c48d852e20cb" (UID: "3acb9197-9efd-4d3c-9f65-c48d852e20cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:39.294653 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.294614    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b84b\" (UniqueName: \"kubernetes.io/projected/3acb9197-9efd-4d3c-9f65-c48d852e20cb-kube-api-access-2b84b\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:39.294653 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.294645    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-util\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:39.294653 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.294658    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3acb9197-9efd-4d3c-9f65-c48d852e20cb-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:39.456300 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456267    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c9fbc6f8c-x5hfl"]
Jul 29 01:04:39.456680 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456661    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="pull"
Jul 29 01:04:39.456759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456683    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="pull"
Jul 29 01:04:39.456759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456696    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="extract"
Jul 29 01:04:39.456759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456704    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="extract"
Jul 29 01:04:39.456759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456717    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="util"
Jul 29 01:04:39.456759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="util"
Jul 29 01:04:39.456994 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.456845    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3acb9197-9efd-4d3c-9f65-c48d852e20cb" containerName="extract"
Jul 29 01:04:39.460012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.459992    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.479219 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.479188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c9fbc6f8c-x5hfl"]
Jul 29 01:04:39.597193 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvjth\" (UniqueName: \"kubernetes.io/projected/6b488a67-bead-4a91-9e85-3b36452512e8-kube-api-access-tvjth\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-oauth-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-trusted-ca-bundle\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-console-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597525 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-oauth-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.597525 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.597504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-service-ca\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698637 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-trusted-ca-bundle\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698637 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-console-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-oauth-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-service-ca\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvjth\" (UniqueName: \"kubernetes.io/projected/6b488a67-bead-4a91-9e85-3b36452512e8-kube-api-access-tvjth\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.698861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.698778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-oauth-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.699518 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.699490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-console-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.699622 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.699491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-service-ca\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.699622 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.699501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-trusted-ca-bundle\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.699704 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.699633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b488a67-bead-4a91-9e85-3b36452512e8-oauth-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.701112 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.701085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-serving-cert\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.701185 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.701118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b488a67-bead-4a91-9e85-3b36452512e8-console-oauth-config\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.708250 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.708230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvjth\" (UniqueName: \"kubernetes.io/projected/6b488a67-bead-4a91-9e85-3b36452512e8-kube-api-access-tvjth\") pod \"console-7c9fbc6f8c-x5hfl\" (UID: \"6b488a67-bead-4a91-9e85-3b36452512e8\") " pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.769584 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.769546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:39.900194 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.900167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c9fbc6f8c-x5hfl"]
Jul 29 01:04:39.902490 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:04:39.902465    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b488a67_bead_4a91_9e85_3b36452512e8.slice/crio-4918ded56e69aa739b0a873397c8ca99606d3d00f32ffdd768107a27648ca0ba WatchSource:0}: Error finding container 4918ded56e69aa739b0a873397c8ca99606d3d00f32ffdd768107a27648ca0ba: Status 404 returned error can't find the container with id 4918ded56e69aa739b0a873397c8ca99606d3d00f32ffdd768107a27648ca0ba
Jul 29 01:04:39.907190 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.907165    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d746b96-312c-495e-9f56-0451d2577427" containerID="ba179bdd1b78cf75a907918548092ad78fd582300d7e10fdc7a777191e6feb43" exitCode=0
Jul 29 01:04:39.907297 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.907269    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn" event={"ID":"2d746b96-312c-495e-9f56-0451d2577427","Type":"ContainerDied","Data":"ba179bdd1b78cf75a907918548092ad78fd582300d7e10fdc7a777191e6feb43"}
Jul 29 01:04:39.908973 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.908954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6" event={"ID":"3acb9197-9efd-4d3c-9f65-c48d852e20cb","Type":"ContainerDied","Data":"a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8"}
Jul 29 01:04:39.909061 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.908991    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a04dd5513fd90b788e6e64f8808e899b0e6a0ff78249416907cc03adfc4d40e8"
Jul 29 01:04:39.909061 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.908961    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6"
Jul 29 01:04:39.911297 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.911253    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerID="f36e8433b05baf1a359e3141bfed0dd1133beb54ae3326cd56fd083356919ff7" exitCode=0
Jul 29 01:04:39.911445 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:39.911388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp" event={"ID":"8e88861e-b7a1-4a2c-9283-dd4a5a983986","Type":"ContainerDied","Data":"f36e8433b05baf1a359e3141bfed0dd1133beb54ae3326cd56fd083356919ff7"}
Jul 29 01:04:40.052756 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.052731    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:40.203750 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.203715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle\") pod \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") "
Jul 29 01:04:40.203925 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.203784    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util\") pod \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") "
Jul 29 01:04:40.203925 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.203814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hndg\" (UniqueName: \"kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg\") pod \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\" (UID: \"71f02d06-8362-4e3b-9d89-d2bc2084dcaa\") "
Jul 29 01:04:40.204412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.204384    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle" (OuterVolumeSpecName: "bundle") pod "71f02d06-8362-4e3b-9d89-d2bc2084dcaa" (UID: "71f02d06-8362-4e3b-9d89-d2bc2084dcaa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:40.205991 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.205969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg" (OuterVolumeSpecName: "kube-api-access-6hndg") pod "71f02d06-8362-4e3b-9d89-d2bc2084dcaa" (UID: "71f02d06-8362-4e3b-9d89-d2bc2084dcaa"). InnerVolumeSpecName "kube-api-access-6hndg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:04:40.209317 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.209244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util" (OuterVolumeSpecName: "util") pod "71f02d06-8362-4e3b-9d89-d2bc2084dcaa" (UID: "71f02d06-8362-4e3b-9d89-d2bc2084dcaa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:40.304958 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.304926    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:40.304958 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.304955    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-util\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:40.304958 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.304965    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hndg\" (UniqueName: \"kubernetes.io/projected/71f02d06-8362-4e3b-9d89-d2bc2084dcaa-kube-api-access-6hndg\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:40.917610 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.917583    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv"
Jul 29 01:04:40.918017 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.917612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv" event={"ID":"71f02d06-8362-4e3b-9d89-d2bc2084dcaa","Type":"ContainerDied","Data":"469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a"}
Jul 29 01:04:40.918017 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.917666    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="469010cc41343470e98ae2ced5aab0d28b08fe0b4af184bf27bb5bdd6e43ea9a"
Jul 29 01:04:40.919118 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.919091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c9fbc6f8c-x5hfl" event={"ID":"6b488a67-bead-4a91-9e85-3b36452512e8","Type":"ContainerStarted","Data":"476e57904252f4775cedc269ffbc27e0d053ceb48d47434c7325feda8b9791cc"}
Jul 29 01:04:40.919218 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.919136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c9fbc6f8c-x5hfl" event={"ID":"6b488a67-bead-4a91-9e85-3b36452512e8","Type":"ContainerStarted","Data":"4918ded56e69aa739b0a873397c8ca99606d3d00f32ffdd768107a27648ca0ba"}
Jul 29 01:04:40.939969 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:40.939919    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c9fbc6f8c-x5hfl" podStartSLOduration=1.939903663 podStartE2EDuration="1.939903663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:40.937503106 +0000 UTC m=+616.015490125" watchObservedRunningTime="2026-07-29 01:04:40.939903663 +0000 UTC m=+616.017890663"
Jul 29 01:04:41.079253 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.079225    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:41.081452 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.081430    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:41.215154 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle\") pod \"2d746b96-312c-495e-9f56-0451d2577427\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") "
Jul 29 01:04:41.215154 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215114    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97jgh\" (UniqueName: \"kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh\") pod \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") "
Jul 29 01:04:41.215387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215187    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util\") pod \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") "
Jul 29 01:04:41.215387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxwq4\" (UniqueName: \"kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4\") pod \"2d746b96-312c-495e-9f56-0451d2577427\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") "
Jul 29 01:04:41.215387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215282    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle\") pod \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\" (UID: \"8e88861e-b7a1-4a2c-9283-dd4a5a983986\") "
Jul 29 01:04:41.215387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215332    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util\") pod \"2d746b96-312c-495e-9f56-0451d2577427\" (UID: \"2d746b96-312c-495e-9f56-0451d2577427\") "
Jul 29 01:04:41.215775 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle" (OuterVolumeSpecName: "bundle") pod "2d746b96-312c-495e-9f56-0451d2577427" (UID: "2d746b96-312c-495e-9f56-0451d2577427"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:41.216024 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.215998    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle" (OuterVolumeSpecName: "bundle") pod "8e88861e-b7a1-4a2c-9283-dd4a5a983986" (UID: "8e88861e-b7a1-4a2c-9283-dd4a5a983986"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:41.217325 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.217294    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4" (OuterVolumeSpecName: "kube-api-access-bxwq4") pod "2d746b96-312c-495e-9f56-0451d2577427" (UID: "2d746b96-312c-495e-9f56-0451d2577427"). InnerVolumeSpecName "kube-api-access-bxwq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:04:41.217414 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.217342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh" (OuterVolumeSpecName: "kube-api-access-97jgh") pod "8e88861e-b7a1-4a2c-9283-dd4a5a983986" (UID: "8e88861e-b7a1-4a2c-9283-dd4a5a983986"). InnerVolumeSpecName "kube-api-access-97jgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:04:41.220672 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.220640    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util" (OuterVolumeSpecName: "util") pod "2d746b96-312c-495e-9f56-0451d2577427" (UID: "2d746b96-312c-495e-9f56-0451d2577427"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:41.223303 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.223280    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util" (OuterVolumeSpecName: "util") pod "8e88861e-b7a1-4a2c-9283-dd4a5a983986" (UID: "8e88861e-b7a1-4a2c-9283-dd4a5a983986"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:04:41.316669 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316635    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-util\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.316814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316673    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxwq4\" (UniqueName: \"kubernetes.io/projected/2d746b96-312c-495e-9f56-0451d2577427-kube-api-access-bxwq4\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.316814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316688    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e88861e-b7a1-4a2c-9283-dd4a5a983986-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.316814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316699    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-util\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.316814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316711    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d746b96-312c-495e-9f56-0451d2577427-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.316814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.316722    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97jgh\" (UniqueName: \"kubernetes.io/projected/8e88861e-b7a1-4a2c-9283-dd4a5a983986-kube-api-access-97jgh\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:04:41.926184 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.926154    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp"
Jul 29 01:04:41.926680 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.926178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp" event={"ID":"8e88861e-b7a1-4a2c-9283-dd4a5a983986","Type":"ContainerDied","Data":"d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb"}
Jul 29 01:04:41.926680 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.926227    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0cdf970a9142ca5d733861c98c4630f6ff001a0c402a6c9943671ba91ba19fb"
Jul 29 01:04:41.928103 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.928075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn" event={"ID":"2d746b96-312c-495e-9f56-0451d2577427","Type":"ContainerDied","Data":"134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84"}
Jul 29 01:04:41.928221 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.928105    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn"
Jul 29 01:04:41.928221 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:41.928125    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="134c38a7265024e548fbabfbabe34fa1a8c7abf32f50d5b190c68119fc46df84"
Jul 29 01:04:49.770475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:49.770384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:49.770475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:49.770464    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:49.775327 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:49.775306    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:49.970427 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:49.970402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c9fbc6f8c-x5hfl"
Jul 29 01:04:50.017500 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:04:50.017465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:05:09.380425 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380383    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"]
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380862    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="util"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380882    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="util"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380905    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380928    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="util"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380937    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="util"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380948    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380956    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380967    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="extract"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380975    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="extract"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380991    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.380998    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="pull"
Jul 29 01:05:09.381003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381010    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381018    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381033    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381041    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381056    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="util"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381063    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="util"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381147    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d746b96-312c-495e-9f56-0451d2577427" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381159    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f02d06-8362-4e3b-9d89-d2bc2084dcaa" containerName="extract"
Jul 29 01:05:09.381685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.381171    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e88861e-b7a1-4a2c-9283-dd4a5a983986" containerName="extract"
Jul 29 01:05:09.384568 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.384546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.387197 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.387161    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 01:05:09.387197 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.387191    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 01:05:09.387389 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.387224    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r7kqd\""
Jul 29 01:05:09.388475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.388457    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 01:05:09.388563 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.388481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 01:05:09.396278 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.396245    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"]
Jul 29 01:05:09.449846 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.449813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ccxl\" (UniqueName: \"kubernetes.io/projected/7bc759c8-006e-42cd-aa6b-d01003176ee9-kube-api-access-7ccxl\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.450003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.449855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.450003 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.449876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bc759c8-006e-42cd-aa6b-d01003176ee9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.551189 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.551151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ccxl\" (UniqueName: \"kubernetes.io/projected/7bc759c8-006e-42cd-aa6b-d01003176ee9-kube-api-access-7ccxl\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.551398 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.551199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.551398 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.551219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bc759c8-006e-42cd-aa6b-d01003176ee9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.551398 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:09.551335    2563 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 01:05:09.551556 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:09.551406    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert podName:7bc759c8-006e-42cd-aa6b-d01003176ee9 nodeName:}" failed. No retries permitted until 2026-07-29 01:05:10.051385205 +0000 UTC m=+645.129372190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-cx4mr" (UID: "7bc759c8-006e-42cd-aa6b-d01003176ee9") : secret "plugin-serving-cert" not found
Jul 29 01:05:09.551860 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.551839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bc759c8-006e-42cd-aa6b-d01003176ee9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:09.562473 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:09.562450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ccxl\" (UniqueName: \"kubernetes.io/projected/7bc759c8-006e-42cd-aa6b-d01003176ee9-kube-api-access-7ccxl\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:10.057006 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:10.056973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:10.059639 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:10.059614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bc759c8-006e-42cd-aa6b-d01003176ee9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cx4mr\" (UID: \"7bc759c8-006e-42cd-aa6b-d01003176ee9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:10.294326 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:10.294289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"
Jul 29 01:05:10.428393 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:10.428366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr"]
Jul 29 01:05:10.430423 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:05:10.430395    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bc759c8_006e_42cd_aa6b_d01003176ee9.slice/crio-9fdc3ecda40377d83920d17076829e9bb20dd2b475d98f87eb723863e3927855 WatchSource:0}: Error finding container 9fdc3ecda40377d83920d17076829e9bb20dd2b475d98f87eb723863e3927855: Status 404 returned error can't find the container with id 9fdc3ecda40377d83920d17076829e9bb20dd2b475d98f87eb723863e3927855
Jul 29 01:05:11.067478 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:11.067442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr" event={"ID":"7bc759c8-006e-42cd-aa6b-d01003176ee9","Type":"ContainerStarted","Data":"9fdc3ecda40377d83920d17076829e9bb20dd2b475d98f87eb723863e3927855"}
Jul 29 01:05:15.036956 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.036885    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b5897785d-hvrng" podUID="695df1b3-ed81-44b3-a333-c1d84fdd4547" containerName="console" containerID="cri-o://1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d" gracePeriod=15
Jul 29 01:05:15.391897 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.391866    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:05:15.508210 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508177    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr2fh\" (UniqueName: \"kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.508694 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508647    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.508899    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert\") pod \"695df1b3-ed81-44b3-a333-c1d84fdd4547\" (UID: \"695df1b3-ed81-44b3-a333-c1d84fdd4547\") "
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca" (OuterVolumeSpecName: "service-ca") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509008    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config" (OuterVolumeSpecName: "console-config") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509206    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509224    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-trusted-ca-bundle\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.509291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509240    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.509657 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.509452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:05:15.510869 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.510839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh" (OuterVolumeSpecName: "kube-api-access-zr2fh") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "kube-api-access-zr2fh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:15.510995 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.510969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:05:15.511454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.511413    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "695df1b3-ed81-44b3-a333-c1d84fdd4547" (UID: "695df1b3-ed81-44b3-a333-c1d84fdd4547"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:05:15.610299 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.610244    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.610299 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.610294    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/695df1b3-ed81-44b3-a333-c1d84fdd4547-console-oauth-config\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.610543 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.610310    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr2fh\" (UniqueName: \"kubernetes.io/projected/695df1b3-ed81-44b3-a333-c1d84fdd4547-kube-api-access-zr2fh\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:15.610543 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:15.610324    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/695df1b3-ed81-44b3-a333-c1d84fdd4547-oauth-serving-cert\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:16.098897 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.098862    2563 generic.go:356] "Generic (PLEG): container finished" podID="695df1b3-ed81-44b3-a333-c1d84fdd4547" containerID="1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d" exitCode=0
Jul 29 01:05:16.099384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.098925    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b5897785d-hvrng"
Jul 29 01:05:16.099384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.098958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5897785d-hvrng" event={"ID":"695df1b3-ed81-44b3-a333-c1d84fdd4547","Type":"ContainerDied","Data":"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"}
Jul 29 01:05:16.099384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.099025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b5897785d-hvrng" event={"ID":"695df1b3-ed81-44b3-a333-c1d84fdd4547","Type":"ContainerDied","Data":"f5a131621e802b59be15e9ba52200983ee62c9fbce5a437ba1c59c4f4ff5c006"}
Jul 29 01:05:16.099384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.099055    2563 scope.go:117] "RemoveContainer" containerID="1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"
Jul 29 01:05:16.110063 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.110045    2563 scope.go:117] "RemoveContainer" containerID="1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"
Jul 29 01:05:16.110442 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:16.110417    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d\": container with ID starting with 1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d not found: ID does not exist" containerID="1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"
Jul 29 01:05:16.110538 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.110454    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d"} err="failed to get container status \"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d\": rpc error: code = NotFound desc = could not find container \"1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d\": container with ID starting with 1b66e293f530f4bec9cca0ab340b126ea35478aa3f7807d5605524fc4e25661d not found: ID does not exist"
Jul 29 01:05:16.123289 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.123239    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:05:16.125409 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:16.125385    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b5897785d-hvrng"]
Jul 29 01:05:17.481884 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:17.481849    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="695df1b3-ed81-44b3-a333-c1d84fdd4547" path="/var/lib/kubelet/pods/695df1b3-ed81-44b3-a333-c1d84fdd4547/volumes"
Jul 29 01:05:19.638685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.638652    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"]
Jul 29 01:05:19.639147 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.639077    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="695df1b3-ed81-44b3-a333-c1d84fdd4547" containerName="console"
Jul 29 01:05:19.639147 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.639094    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="695df1b3-ed81-44b3-a333-c1d84fdd4547" containerName="console"
Jul 29 01:05:19.639273 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.639182    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="695df1b3-ed81-44b3-a333-c1d84fdd4547" containerName="console"
Jul 29 01:05:19.649757 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.649736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.652862 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.652838    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-75x59\""
Jul 29 01:05:19.657348 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.657323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"]
Jul 29 01:05:19.748839 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.748794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b687m\" (UniqueName: \"kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.749018 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.748930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.850467 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.850415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.850653 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.850522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b687m\" (UniqueName: \"kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.850892 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.850866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.859859 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.859829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b687m\" (UniqueName: \"kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m\") pod \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:19.963536 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:19.963452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:20.053651 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.053616    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"]
Jul 29 01:05:20.055989 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.055956    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"]
Jul 29 01:05:20.080195 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.080163    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:05:20.086877 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.086847    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.099697 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.099673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:05:20.152971 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.152940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.153162 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.152994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5frj6\" (UniqueName: \"kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.253942 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.253854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.253942 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.253896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5frj6\" (UniqueName: \"kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.254291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.254250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.273529 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.273500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5frj6\" (UniqueName: \"kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6\") pod \"kuadrant-operator-controller-manager-55467d9f7-5rpjb\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:20.399699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:20.399662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:31.230481 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:31.230452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:05:31.231025 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:05:31.230994    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d64cb6_348c_4101_b188_5c3a31ed6b11.slice/crio-09dd62b7be6e1fe6398a84d4215909003f0ec866894ca778274b855c08a767b9 WatchSource:0}: Error finding container 09dd62b7be6e1fe6398a84d4215909003f0ec866894ca778274b855c08a767b9: Status 404 returned error can't find the container with id 09dd62b7be6e1fe6398a84d4215909003f0ec866894ca778274b855c08a767b9
Jul 29 01:05:32.200245 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:32.200208    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr" event={"ID":"7bc759c8-006e-42cd-aa6b-d01003176ee9","Type":"ContainerStarted","Data":"15d7c182032f1243ea4514cb6240a69b90fb4c2caacf8f351c21736f7f7e8cdf"}
Jul 29 01:05:32.201898 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:32.201866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" event={"ID":"34d64cb6-348c-4101-b188-5c3a31ed6b11","Type":"ContainerStarted","Data":"09dd62b7be6e1fe6398a84d4215909003f0ec866894ca778274b855c08a767b9"}
Jul 29 01:05:32.218755 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:32.218660    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cx4mr" podStartSLOduration=2.482709481 podStartE2EDuration="23.218641585s" totalImagesPullingTime="20.735932104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:32.21633389 +0000 UTC m=+667.294320912" watchObservedRunningTime="2026-07-29 01:05:32.218641585 +0000 UTC m=+667.296628589"
Jul 29 01:05:33.632401 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:33.632361    2563 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hfc95_kuadrant-system_81a7669c-d134-40a2-9448-2a09af18f8af_0(7a61707a48e9d1cd379529b9b23f2e11d8c3af182e37ea9a8f8364dbced24aae): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hfc95 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 29 01:05:33.632756 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:33.632425    2563 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hfc95_kuadrant-system_81a7669c-d134-40a2-9448-2a09af18f8af_0(7a61707a48e9d1cd379529b9b23f2e11d8c3af182e37ea9a8f8364dbced24aae): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hfc95 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:34.218468 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.218435    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:34.221003 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:34.220971    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:34.249469 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.249449    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:34.251919 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:34.251894    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:34.389796 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.389746    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b687m\" (UniqueName: \"kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m\") pod \"81a7669c-d134-40a2-9448-2a09af18f8af\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") "
Jul 29 01:05:34.389965 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.389842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume\") pod \"81a7669c-d134-40a2-9448-2a09af18f8af\" (UID: \"81a7669c-d134-40a2-9448-2a09af18f8af\") "
Jul 29 01:05:34.390182 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.390151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81a7669c-d134-40a2-9448-2a09af18f8af" (UID: "81a7669c-d134-40a2-9448-2a09af18f8af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:05:34.392076 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.392052    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m" (OuterVolumeSpecName: "kube-api-access-b687m") pod "81a7669c-d134-40a2-9448-2a09af18f8af" (UID: "81a7669c-d134-40a2-9448-2a09af18f8af"). InnerVolumeSpecName "kube-api-access-b687m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:34.490573 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.490497    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b687m\" (UniqueName: \"kubernetes.io/projected/81a7669c-d134-40a2-9448-2a09af18f8af-kube-api-access-b687m\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:34.490573 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:34.490526    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81a7669c-d134-40a2-9448-2a09af18f8af-extensions-socket-volume\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:05:35.223804 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:35.223773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:35.226729 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:35.226690    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:35.232829 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:35.232799    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:35.474239 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:05:35.474166    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hfc95\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hfc95"
Jul 29 01:05:35.475045 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:35.475022    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a7669c-d134-40a2-9448-2a09af18f8af" path="/var/lib/kubelet/pods/81a7669c-d134-40a2-9448-2a09af18f8af/volumes"
Jul 29 01:05:36.230450 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:36.230415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" event={"ID":"34d64cb6-348c-4101-b188-5c3a31ed6b11","Type":"ContainerStarted","Data":"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"}
Jul 29 01:05:36.230865 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:36.230541    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:36.254796 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:36.254748    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" podStartSLOduration=11.813912422 podStartE2EDuration="16.254733661s" totalImagesPullingTime="4.440821239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:36.252624609 +0000 UTC m=+671.330611629" watchObservedRunningTime="2026-07-29 01:05:36.254733661 +0000 UTC m=+671.332720659"
Jul 29 01:05:47.237206 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:47.237168    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:05:59.749037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:59.749000    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:05:59.749494 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:59.749199    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" podUID="34d64cb6-348c-4101-b188-5c3a31ed6b11" containerName="manager" containerID="cri-o://b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f" gracePeriod=10
Jul 29 01:05:59.998104 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:05:59.998080    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:06:00.119071 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.119038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume\") pod \"34d64cb6-348c-4101-b188-5c3a31ed6b11\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") "
Jul 29 01:06:00.119251 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.119100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5frj6\" (UniqueName: \"kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6\") pod \"34d64cb6-348c-4101-b188-5c3a31ed6b11\" (UID: \"34d64cb6-348c-4101-b188-5c3a31ed6b11\") "
Jul 29 01:06:00.119585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.119559    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34d64cb6-348c-4101-b188-5c3a31ed6b11" (UID: "34d64cb6-348c-4101-b188-5c3a31ed6b11"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:06:00.121202 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.121168    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6" (OuterVolumeSpecName: "kube-api-access-5frj6") pod "34d64cb6-348c-4101-b188-5c3a31ed6b11" (UID: "34d64cb6-348c-4101-b188-5c3a31ed6b11"). InnerVolumeSpecName "kube-api-access-5frj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:00.220077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.220041    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34d64cb6-348c-4101-b188-5c3a31ed6b11-extensions-socket-volume\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:06:00.220077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.220071    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5frj6\" (UniqueName: \"kubernetes.io/projected/34d64cb6-348c-4101-b188-5c3a31ed6b11-kube-api-access-5frj6\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:06:00.346562 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.346521    2563 generic.go:356] "Generic (PLEG): container finished" podID="34d64cb6-348c-4101-b188-5c3a31ed6b11" containerID="b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f" exitCode=0
Jul 29 01:06:00.346720 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.346562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" event={"ID":"34d64cb6-348c-4101-b188-5c3a31ed6b11","Type":"ContainerDied","Data":"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"}
Jul 29 01:06:00.346720 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.346583    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"
Jul 29 01:06:00.346720 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.346647    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb" event={"ID":"34d64cb6-348c-4101-b188-5c3a31ed6b11","Type":"ContainerDied","Data":"09dd62b7be6e1fe6398a84d4215909003f0ec866894ca778274b855c08a767b9"}
Jul 29 01:06:00.346720 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.346672    2563 scope.go:117] "RemoveContainer" containerID="b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"
Jul 29 01:06:00.355885 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.355867    2563 scope.go:117] "RemoveContainer" containerID="b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"
Jul 29 01:06:00.356163 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:06:00.356144    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f\": container with ID starting with b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f not found: ID does not exist" containerID="b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"
Jul 29 01:06:00.356209 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.356172    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f"} err="failed to get container status \"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f\": rpc error: code = NotFound desc = could not find container \"b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f\": container with ID starting with b7d52900318f269a50cff893eb1aedaab53e8d33c6267b0711d452ed67b7118f not found: ID does not exist"
Jul 29 01:06:00.369716 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.369659    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:06:00.373985 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:00.373965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5rpjb"]
Jul 29 01:06:01.474906 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:01.474873    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34d64cb6-348c-4101-b188-5c3a31ed6b11" path="/var/lib/kubelet/pods/34d64cb6-348c-4101-b188-5c3a31ed6b11/volumes"
Jul 29 01:06:06.809504 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.809468    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:06:06.809880 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.809812    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d64cb6-348c-4101-b188-5c3a31ed6b11" containerName="manager"
Jul 29 01:06:06.809880 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.809822    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d64cb6-348c-4101-b188-5c3a31ed6b11" containerName="manager"
Jul 29 01:06:06.809966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.809890    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d64cb6-348c-4101-b188-5c3a31ed6b11" containerName="manager"
Jul 29 01:06:06.842301 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.842248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:06:06.842445 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.842353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:06.844115 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.844066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:06:06.845291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.845249    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 01:06:06.978455 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.978425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fbbf3f3e-f8be-4734-b2bd-b1355c820168-config-file\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:06.978455 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:06.978458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgnh\" (UniqueName: \"kubernetes.io/projected/fbbf3f3e-f8be-4734-b2bd-b1355c820168-kube-api-access-smgnh\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.079765 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.079665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fbbf3f3e-f8be-4734-b2bd-b1355c820168-config-file\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.079765 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.079701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgnh\" (UniqueName: \"kubernetes.io/projected/fbbf3f3e-f8be-4734-b2bd-b1355c820168-kube-api-access-smgnh\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.080375 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.080354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fbbf3f3e-f8be-4734-b2bd-b1355c820168-config-file\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.088616 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.088593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgnh\" (UniqueName: \"kubernetes.io/projected/fbbf3f3e-f8be-4734-b2bd-b1355c820168-kube-api-access-smgnh\") pod \"limitador-limitador-54d6d988d5-nkd8s\" (UID: \"fbbf3f3e-f8be-4734-b2bd-b1355c820168\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.152608 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.152563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:07.290390 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.290364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:06:07.291430 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:06:07.291400    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbf3f3e_f8be_4734_b2bd_b1355c820168.slice/crio-4fc900ab5c534b089249a2c4a08f182680e1ade992d05ea0ff3b765772ebadc9 WatchSource:0}: Error finding container 4fc900ab5c534b089249a2c4a08f182680e1ade992d05ea0ff3b765772ebadc9: Status 404 returned error can't find the container with id 4fc900ab5c534b089249a2c4a08f182680e1ade992d05ea0ff3b765772ebadc9
Jul 29 01:06:07.296719 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.296698    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:07.300181 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.300164    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:07.302968 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.302949    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dtfm8\""
Jul 29 01:06:07.310293 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.310251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:07.382164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.382132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s" event={"ID":"fbbf3f3e-f8be-4734-b2bd-b1355c820168","Type":"ContainerStarted","Data":"4fc900ab5c534b089249a2c4a08f182680e1ade992d05ea0ff3b765772ebadc9"}
Jul 29 01:06:07.382353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.382140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmn5f\" (UniqueName: \"kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f\") pod \"authorino-58cd955675-7fgpj\" (UID: \"04b32b79-df33-420e-9f18-9a99d8519307\") " pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:07.482868 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.482842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmn5f\" (UniqueName: \"kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f\") pod \"authorino-58cd955675-7fgpj\" (UID: \"04b32b79-df33-420e-9f18-9a99d8519307\") " pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:07.497443 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.497417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmn5f\" (UniqueName: \"kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f\") pod \"authorino-58cd955675-7fgpj\" (UID: \"04b32b79-df33-420e-9f18-9a99d8519307\") " pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:07.609932 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.609906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:07.736744 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:07.736721    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:07.739099 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:06:07.739070    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b32b79_df33_420e_9f18_9a99d8519307.slice/crio-efc7340c46dbbd1e6eaf6f92c57d4ed7591b7abfadf3360257e9ff40b6a49a3f WatchSource:0}: Error finding container efc7340c46dbbd1e6eaf6f92c57d4ed7591b7abfadf3360257e9ff40b6a49a3f: Status 404 returned error can't find the container with id efc7340c46dbbd1e6eaf6f92c57d4ed7591b7abfadf3360257e9ff40b6a49a3f
Jul 29 01:06:08.389407 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:08.389370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7fgpj" event={"ID":"04b32b79-df33-420e-9f18-9a99d8519307","Type":"ContainerStarted","Data":"efc7340c46dbbd1e6eaf6f92c57d4ed7591b7abfadf3360257e9ff40b6a49a3f"}
Jul 29 01:06:12.414979 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.414926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s" event={"ID":"fbbf3f3e-f8be-4734-b2bd-b1355c820168","Type":"ContainerStarted","Data":"cec2a94d0135387a2017a20364575e570e4b43f412dccd455835e930bc7f0253"}
Jul 29 01:06:12.415443 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.415125    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:12.416382 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.416357    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7fgpj" event={"ID":"04b32b79-df33-420e-9f18-9a99d8519307","Type":"ContainerStarted","Data":"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"}
Jul 29 01:06:12.434151 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.434108    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s" podStartSLOduration=1.79442988 podStartE2EDuration="6.434095528s" totalImagesPullingTime="4.639665648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:12.432504541 +0000 UTC m=+707.510491539" watchObservedRunningTime="2026-07-29 01:06:12.434095528 +0000 UTC m=+707.512082526"
Jul 29 01:06:12.867878 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.867814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7fgpj" podStartSLOduration=1.674012314 podStartE2EDuration="5.867797973s" totalImagesPullingTime="4.193785659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:12.451756846 +0000 UTC m=+707.529743871" watchObservedRunningTime="2026-07-29 01:06:12.867797973 +0000 UTC m=+707.945784971"
Jul 29 01:06:12.868053 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:12.867931    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:14.425646 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:14.425597    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7fgpj" podUID="04b32b79-df33-420e-9f18-9a99d8519307" containerName="authorino" containerID="cri-o://e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377" gracePeriod=30
Jul 29 01:06:14.665390 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:14.665367    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:14.752776 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:14.752688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmn5f\" (UniqueName: \"kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f\") pod \"04b32b79-df33-420e-9f18-9a99d8519307\" (UID: \"04b32b79-df33-420e-9f18-9a99d8519307\") "
Jul 29 01:06:14.754734 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:14.754705    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f" (OuterVolumeSpecName: "kube-api-access-kmn5f") pod "04b32b79-df33-420e-9f18-9a99d8519307" (UID: "04b32b79-df33-420e-9f18-9a99d8519307"). InnerVolumeSpecName "kube-api-access-kmn5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:14.854340 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:14.854302    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmn5f\" (UniqueName: \"kubernetes.io/projected/04b32b79-df33-420e-9f18-9a99d8519307-kube-api-access-kmn5f\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:06:15.431656 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.431621    2563 generic.go:356] "Generic (PLEG): container finished" podID="04b32b79-df33-420e-9f18-9a99d8519307" containerID="e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377" exitCode=0
Jul 29 01:06:15.432164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.431665    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7fgpj"
Jul 29 01:06:15.432164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.431707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7fgpj" event={"ID":"04b32b79-df33-420e-9f18-9a99d8519307","Type":"ContainerDied","Data":"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"}
Jul 29 01:06:15.432164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.431754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7fgpj" event={"ID":"04b32b79-df33-420e-9f18-9a99d8519307","Type":"ContainerDied","Data":"efc7340c46dbbd1e6eaf6f92c57d4ed7591b7abfadf3360257e9ff40b6a49a3f"}
Jul 29 01:06:15.432164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.431779    2563 scope.go:117] "RemoveContainer" containerID="e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"
Jul 29 01:06:15.440714 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.440698    2563 scope.go:117] "RemoveContainer" containerID="e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"
Jul 29 01:06:15.440972 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:06:15.440953    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377\": container with ID starting with e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377 not found: ID does not exist" containerID="e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"
Jul 29 01:06:15.441020 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.440981    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377"} err="failed to get container status \"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377\": rpc error: code = NotFound desc = could not find container \"e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377\": container with ID starting with e5a7c1742d6db7c926014a073cd7537c0c1b9d27d15f9c6fe5bf5174f0bae377 not found: ID does not exist"
Jul 29 01:06:15.457507 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.457483    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:15.465221 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.465198    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7fgpj"]
Jul 29 01:06:15.474786 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:15.474764    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b32b79-df33-420e-9f18-9a99d8519307" path="/var/lib/kubelet/pods/04b32b79-df33-420e-9f18-9a99d8519307/volumes"
Jul 29 01:06:23.422928 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:23.422843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"
Jul 29 01:06:40.483243 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.483206    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:40.483704 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.483546    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b32b79-df33-420e-9f18-9a99d8519307" containerName="authorino"
Jul 29 01:06:40.483704 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.483557    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b32b79-df33-420e-9f18-9a99d8519307" containerName="authorino"
Jul 29 01:06:40.483704 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.483615    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b32b79-df33-420e-9f18-9a99d8519307" containerName="authorino"
Jul 29 01:06:40.487943 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.487921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:40.492946 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.492926    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dtfm8\""
Jul 29 01:06:40.498254 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.498229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:40.569520 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.569487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjx8w\" (UniqueName: \"kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w\") pod \"authorino-5d4b67c446-6sdmg\" (UID: \"52864280-3e4c-4bf6-ab46-81cd89539e17\") " pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:40.670374 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.670343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjx8w\" (UniqueName: \"kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w\") pod \"authorino-5d4b67c446-6sdmg\" (UID: \"52864280-3e4c-4bf6-ab46-81cd89539e17\") " pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:40.680664 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.680637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjx8w\" (UniqueName: \"kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w\") pod \"authorino-5d4b67c446-6sdmg\" (UID: \"52864280-3e4c-4bf6-ab46-81cd89539e17\") " pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:40.707250 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.707225    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:40.707445 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.707433    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:40.840703 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:40.840674    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:40.842461 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:06:40.842438    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52864280_3e4c_4bf6_ab46_81cd89539e17.slice/crio-86504ee315b7a20b0966d41b7157f02f9760a3c251466389648ba53f61096a64 WatchSource:0}: Error finding container 86504ee315b7a20b0966d41b7157f02f9760a3c251466389648ba53f61096a64: Status 404 returned error can't find the container with id 86504ee315b7a20b0966d41b7157f02f9760a3c251466389648ba53f61096a64
Jul 29 01:06:41.560184 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.560096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" event={"ID":"52864280-3e4c-4bf6-ab46-81cd89539e17","Type":"ContainerStarted","Data":"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"}
Jul 29 01:06:41.560184 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.560145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" event={"ID":"52864280-3e4c-4bf6-ab46-81cd89539e17","Type":"ContainerStarted","Data":"86504ee315b7a20b0966d41b7157f02f9760a3c251466389648ba53f61096a64"}
Jul 29 01:06:41.560658 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.560158    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" podUID="52864280-3e4c-4bf6-ab46-81cd89539e17" containerName="authorino" containerID="cri-o://cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab" gracePeriod=30
Jul 29 01:06:41.577921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.577872    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" podStartSLOduration=1.24094605 podStartE2EDuration="1.57785674s" totalImagesPullingTime="336.91069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:41.575755862 +0000 UTC m=+736.653742883" watchObservedRunningTime="2026-07-29 01:06:41.57785674 +0000 UTC m=+736.655843739"
Jul 29 01:06:41.812356 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.812303    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:41.881580 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.881553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjx8w\" (UniqueName: \"kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w\") pod \"52864280-3e4c-4bf6-ab46-81cd89539e17\" (UID: \"52864280-3e4c-4bf6-ab46-81cd89539e17\") "
Jul 29 01:06:41.883505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.883482    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w" (OuterVolumeSpecName: "kube-api-access-kjx8w") pod "52864280-3e4c-4bf6-ab46-81cd89539e17" (UID: "52864280-3e4c-4bf6-ab46-81cd89539e17"). InnerVolumeSpecName "kube-api-access-kjx8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:41.982134 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:41.982109    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjx8w\" (UniqueName: \"kubernetes.io/projected/52864280-3e4c-4bf6-ab46-81cd89539e17-kube-api-access-kjx8w\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:06:42.565619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.565584    2563 generic.go:356] "Generic (PLEG): container finished" podID="52864280-3e4c-4bf6-ab46-81cd89539e17" containerID="cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab" exitCode=0
Jul 29 01:06:42.565619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.565623    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6sdmg"
Jul 29 01:06:42.566099 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.565669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" event={"ID":"52864280-3e4c-4bf6-ab46-81cd89539e17","Type":"ContainerDied","Data":"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"}
Jul 29 01:06:42.566099 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.565711    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6sdmg" event={"ID":"52864280-3e4c-4bf6-ab46-81cd89539e17","Type":"ContainerDied","Data":"86504ee315b7a20b0966d41b7157f02f9760a3c251466389648ba53f61096a64"}
Jul 29 01:06:42.566099 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.565730    2563 scope.go:117] "RemoveContainer" containerID="cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"
Jul 29 01:06:42.574729 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.574713    2563 scope.go:117] "RemoveContainer" containerID="cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"
Jul 29 01:06:42.574990 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:06:42.574973    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab\": container with ID starting with cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab not found: ID does not exist" containerID="cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"
Jul 29 01:06:42.575034 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.574998    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab"} err="failed to get container status \"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab\": rpc error: code = NotFound desc = could not find container \"cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab\": container with ID starting with cbfb7f42240819576b1b8fb69da10fbf01002b52b837f3b7d4208ae73301a6ab not found: ID does not exist"
Jul 29 01:06:42.587054 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.587035    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:42.590742 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:42.590722    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6sdmg"]
Jul 29 01:06:43.474734 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:06:43.474704    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52864280-3e4c-4bf6-ab46-81cd89539e17" path="/var/lib/kubelet/pods/52864280-3e4c-4bf6-ab46-81cd89539e17/volumes"
Jul 29 01:07:24.813876 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:07:24.813846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:08:00.088459 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.088382    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"]
Jul 29 01:08:00.088927 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.088910    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52864280-3e4c-4bf6-ab46-81cd89539e17" containerName="authorino"
Jul 29 01:08:00.088970 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.088930    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="52864280-3e4c-4bf6-ab46-81cd89539e17" containerName="authorino"
Jul 29 01:08:00.089029 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.089018    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="52864280-3e4c-4bf6-ab46-81cd89539e17" containerName="authorino"
Jul 29 01:08:00.092376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.092342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.096447 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.096420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 01:08:00.096666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.096446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 01:08:00.096666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.096497    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 01:08:00.096666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.096505    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kjskh\""
Jul 29 01:08:00.100387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.100364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"]
Jul 29 01:08:00.263353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnm9k\" (UniqueName: \"kubernetes.io/projected/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-kube-api-access-vnm9k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.263534 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.263534 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.263534 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.263534 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.263534 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.263506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.364885 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.364787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.364885 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.364843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365105 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.364955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365105 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365105 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnm9k\" (UniqueName: \"kubernetes.io/projected/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-kube-api-access-vnm9k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365301 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365301 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365301 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.365440 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.365377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.367175 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.367148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.367519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.367500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.374159 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.374137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnm9k\" (UniqueName: \"kubernetes.io/projected/c4b71f4c-3c33-4e7d-9396-4ba6c008c501-kube-api-access-vnm9k\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9\" (UID: \"c4b71f4c-3c33-4e7d-9396-4ba6c008c501\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.404688 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.404648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:00.534745 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.534719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"]
Jul 29 01:08:00.537304 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:08:00.537277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b71f4c_3c33_4e7d_9396_4ba6c008c501.slice/crio-598078fcfd857ea207cb46eb293ea932fe1ac12b62305f32452e03e3db43f417 WatchSource:0}: Error finding container 598078fcfd857ea207cb46eb293ea932fe1ac12b62305f32452e03e3db43f417: Status 404 returned error can't find the container with id 598078fcfd857ea207cb46eb293ea932fe1ac12b62305f32452e03e3db43f417
Jul 29 01:08:00.581826 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.581794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:08:00.960679 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:00.960638    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9" event={"ID":"c4b71f4c-3c33-4e7d-9396-4ba6c008c501","Type":"ContainerStarted","Data":"598078fcfd857ea207cb46eb293ea932fe1ac12b62305f32452e03e3db43f417"}
Jul 29 01:08:01.967860 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:01.967761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9" event={"ID":"c4b71f4c-3c33-4e7d-9396-4ba6c008c501","Type":"ContainerStarted","Data":"d573690114dbaf4277f3794f9c5d0102134436f275a80a2b94d6c5f902fc4a6e"}
Jul 29 01:08:01.986622 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:01.986576    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9" podStartSLOduration=0.831987936 podStartE2EDuration="1.986561264s" totalImagesPullingTime="1.154573328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:08:01.984237553 +0000 UTC m=+817.062224551" watchObservedRunningTime="2026-07-29 01:08:01.986561264 +0000 UTC m=+817.064548262"
Jul 29 01:08:10.405277 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:10.405220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:10.405773 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:10.405331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:10.417629 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:10.417607    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:11.026215 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:11.026186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9"
Jul 29 01:08:11.386006 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:11.385972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:08:28.095250 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:28.095207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:08:33.693661 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:33.693623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:08:59.983195 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:08:59.983159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:09:16.387324 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.387289    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"]
Jul 29 01:09:16.390589 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.390571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.393375 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.393351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 01:09:16.399031 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.399008    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"]
Jul 29 01:09:16.521949 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.521920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.522067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.521960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.522067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.521985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72d3d73e-ba77-4769-92a9-de28c3400d46-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.522067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.522029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.522169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.522067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.522169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.522087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkcw9\" (UniqueName: \"kubernetes.io/projected/72d3d73e-ba77-4769-92a9-de28c3400d46-kube-api-access-nkcw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623160 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623343 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72d3d73e-ba77-4769-92a9-de28c3400d46-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623343 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623343 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623343 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkcw9\" (UniqueName: \"kubernetes.io/projected/72d3d73e-ba77-4769-92a9-de28c3400d46-kube-api-access-nkcw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623611 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623611 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623723 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.623777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.623734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.626081 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.626048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72d3d73e-ba77-4769-92a9-de28c3400d46-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.626210 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.626187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72d3d73e-ba77-4769-92a9-de28c3400d46-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.631526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.631500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkcw9\" (UniqueName: \"kubernetes.io/projected/72d3d73e-ba77-4769-92a9-de28c3400d46-kube-api-access-nkcw9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb\" (UID: \"72d3d73e-ba77-4769-92a9-de28c3400d46\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.725112 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.725039    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:16.854940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:16.853125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"]
Jul 29 01:09:16.855304 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:09:16.855273    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72d3d73e_ba77_4769_92a9_de28c3400d46.slice/crio-ad6be4fee769cf3b26337b967edec9a3d373c4cb7295cd7254a9894fa4087c4e WatchSource:0}: Error finding container ad6be4fee769cf3b26337b967edec9a3d373c4cb7295cd7254a9894fa4087c4e: Status 404 returned error can't find the container with id ad6be4fee769cf3b26337b967edec9a3d373c4cb7295cd7254a9894fa4087c4e
Jul 29 01:09:17.346447 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:17.346409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb" event={"ID":"72d3d73e-ba77-4769-92a9-de28c3400d46","Type":"ContainerStarted","Data":"dcd3afbaec58bd7999cf9f2c60985cabef1d39653e4437789b5eb25a60099cd6"}
Jul 29 01:09:17.346805 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:17.346455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb" event={"ID":"72d3d73e-ba77-4769-92a9-de28c3400d46","Type":"ContainerStarted","Data":"ad6be4fee769cf3b26337b967edec9a3d373c4cb7295cd7254a9894fa4087c4e"}
Jul 29 01:09:17.382857 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:17.382807    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb" podStartSLOduration=1.1857696309999999 podStartE2EDuration="1.3827935s" totalImagesPullingTime="197.023869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:09:17.381410992 +0000 UTC m=+892.459397990" watchObservedRunningTime="2026-07-29 01:09:17.3827935 +0000 UTC m=+892.460780574"
Jul 29 01:09:18.696908 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:18.696871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:09:26.725678 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:26.725613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:26.726106 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:26.725896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:26.738251 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:26.738226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:09:27.412212 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:09:27.412174    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb"
Jul 29 01:10:13.174695 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.174654    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b9c5dd547-qcbcc"]
Jul 29 01:10:13.177925 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.177906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.181819 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.181797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 01:10:13.181958 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.181800    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dtfm8\""
Jul 29 01:10:13.185436 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.185412    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b9c5dd547-qcbcc"]
Jul 29 01:10:13.310041 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.310008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f73194-c496-41ed-bf3a-3b71c56277c9-tls-cert\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.310041 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.310044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9v4g\" (UniqueName: \"kubernetes.io/projected/80f73194-c496-41ed-bf3a-3b71c56277c9-kube-api-access-b9v4g\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.411001 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.410968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f73194-c496-41ed-bf3a-3b71c56277c9-tls-cert\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.411001 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.411003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9v4g\" (UniqueName: \"kubernetes.io/projected/80f73194-c496-41ed-bf3a-3b71c56277c9-kube-api-access-b9v4g\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.413318 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.413296    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80f73194-c496-41ed-bf3a-3b71c56277c9-tls-cert\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.419654 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.419634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9v4g\" (UniqueName: \"kubernetes.io/projected/80f73194-c496-41ed-bf3a-3b71c56277c9-kube-api-access-b9v4g\") pod \"authorino-b9c5dd547-qcbcc\" (UID: \"80f73194-c496-41ed-bf3a-3b71c56277c9\") " pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.488151 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.488091    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b9c5dd547-qcbcc"
Jul 29 01:10:13.615566 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.615500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b9c5dd547-qcbcc"]
Jul 29 01:10:13.617983 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:10:13.617958    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80f73194_c496_41ed_bf3a_3b71c56277c9.slice/crio-50f7fe8ea35fc33ad3a76a02496c612556a93371102612b76b94745a17555439 WatchSource:0}: Error finding container 50f7fe8ea35fc33ad3a76a02496c612556a93371102612b76b94745a17555439: Status 404 returned error can't find the container with id 50f7fe8ea35fc33ad3a76a02496c612556a93371102612b76b94745a17555439
Jul 29 01:10:13.619709 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.619682    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:10:13.628065 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:13.628041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b9c5dd547-qcbcc" event={"ID":"80f73194-c496-41ed-bf3a-3b71c56277c9","Type":"ContainerStarted","Data":"50f7fe8ea35fc33ad3a76a02496c612556a93371102612b76b94745a17555439"}
Jul 29 01:10:14.635211 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:14.635110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b9c5dd547-qcbcc" event={"ID":"80f73194-c496-41ed-bf3a-3b71c56277c9","Type":"ContainerStarted","Data":"650bfc9f4b44c9293b63e610bb9668d66f5d1fe1401d3856f14e418ea58b721a"}
Jul 29 01:10:14.654211 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:14.654155    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b9c5dd547-qcbcc" podStartSLOduration=0.911881976 podStartE2EDuration="1.654135761s" totalImagesPullingTime="742.253785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:10:14.652572221 +0000 UTC m=+949.730559248" watchObservedRunningTime="2026-07-29 01:10:14.654135761 +0000 UTC m=+949.732122762"
Jul 29 01:10:41.583721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:41.583682    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:10:52.783915 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:10:52.783838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:11:03.093272 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:11:03.093228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:12:41.694742 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:12:41.694695    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:13:21.887125 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:13:21.887081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:13:31.885827 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:13:31.885791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:13:40.592276 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:13:40.592234    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:13:51.695166 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:13:51.695082    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:14:01.690213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:14:01.690173    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:14:13.286121 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:14:13.286089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:14:23.797636 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:14:23.795376    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:14:34.085685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:14:34.085648    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:14:56.186124 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:14:56.186088    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:15:08.188575 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:15:08.188536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:15:30.329853 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:15:30.329772    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:15:43.053370 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:15:43.053332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:15:52.597835 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:15:52.597792    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:16:08.405421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:16:08.405386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:16:17.889713 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:16:17.889675    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:16:27.298493 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:16:27.298444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:16:36.900779 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:16:36.900737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:16:46.000866 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:16:46.000829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:18:34.893777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:18:34.893735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:18:43.922796 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:18:43.922755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:00.996164 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:00.996125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:10.097763 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:10.097722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:21.010188 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:21.010151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:31.922484 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:31.922443    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:41.448887 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:41.448846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:19:50.819490 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:19:50.819413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:02.809012 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:02.808977    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:11.518580 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:11.518538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:22.509822 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:22.509788    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:46.900336 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:46.900301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:55.225621 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:55.225581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:20:56.020319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.020278    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:20:56.023504 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.023481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.031326 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.031306    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5f6d371c-kserve-self-signed-certs\""
Jul 29 01:20:56.073361 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.073325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:20:56.126368 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.126552 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.126552 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cc46\" (UniqueName: \"kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.126552 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.126552 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.126764 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.126633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.227526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.227526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cc46\" (UniqueName: \"kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227687    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.227948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.228140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.228392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.228213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.229946 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.229914    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.230169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.230145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.245110 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.245085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cc46\" (UniqueName: \"kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46\") pod \"e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") " pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.332812 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.332777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:20:56.528510 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.528481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:20:56.530201 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:20:56.530163    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8e38ab6_ab02_44da_9682_56d413085f05.slice/crio-b2739dd2ce816871fe08bb06fb2235fd9221e16749b33b3a5ca7eb7508cf4702 WatchSource:0}: Error finding container b2739dd2ce816871fe08bb06fb2235fd9221e16749b33b3a5ca7eb7508cf4702: Status 404 returned error can't find the container with id b2739dd2ce816871fe08bb06fb2235fd9221e16749b33b3a5ca7eb7508cf4702
Jul 29 01:20:56.921914 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.921814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" event={"ID":"c8e38ab6-ab02-44da-9682-56d413085f05","Type":"ContainerStarted","Data":"c62af694be192b2e17c17b5577385ff698a69b609bdc6d0bfa4a061bf61e3305"}
Jul 29 01:20:56.921914 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:20:56.921859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" event={"ID":"c8e38ab6-ab02-44da-9682-56d413085f05","Type":"ContainerStarted","Data":"b2739dd2ce816871fe08bb06fb2235fd9221e16749b33b3a5ca7eb7508cf4702"}
Jul 29 01:21:06.333309 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:06.333277    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:06.333740 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:06.333335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:06.345707 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:06.345683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:06.373766 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:06.373714    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" podStartSLOduration=11.373699983 podStartE2EDuration="11.373699983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:20:56.972121408 +0000 UTC m=+1592.050108398" watchObservedRunningTime="2026-07-29 01:21:06.373699983 +0000 UTC m=+1601.451686982"
Jul 29 01:21:06.981734 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:06.981699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:08.448183 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:08.448147    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:21:34.984707 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:34.984617    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:21:38.004397 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:38.004361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:21:49.706896 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:49.706852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:21:50.003843 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.003753    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" podUID="c8e38ab6-ab02-44da-9682-56d413085f05" containerName="main" containerID="cri-o://c62af694be192b2e17c17b5577385ff698a69b609bdc6d0bfa4a061bf61e3305" gracePeriod=45
Jul 29 01:21:50.199097 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.199063    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8e38ab6-ab02-44da-9682-56d413085f05" containerID="c62af694be192b2e17c17b5577385ff698a69b609bdc6d0bfa4a061bf61e3305" exitCode=0
Jul 29 01:21:50.199232 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.199134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" event={"ID":"c8e38ab6-ab02-44da-9682-56d413085f05","Type":"ContainerDied","Data":"c62af694be192b2e17c17b5577385ff698a69b609bdc6d0bfa4a061bf61e3305"}
Jul 29 01:21:50.257998 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.257933    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:50.448986 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.448952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449131    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449429 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449183    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cc46\" (UniqueName: \"kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46\") pod \"c8e38ab6-ab02-44da-9682-56d413085f05\" (UID: \"c8e38ab6-ab02-44da-9682-56d413085f05\") "
Jul 29 01:21:50.449429 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449241    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:50.449429 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:50.449584 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home" (OuterVolumeSpecName: "home") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:50.449700 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449683    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:50.449762 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449700    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:50.449762 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.449709    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:50.451386 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.451363    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm" (OuterVolumeSpecName: "dshm") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:50.451502 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.451456    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:21:50.451502 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.451459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46" (OuterVolumeSpecName: "kube-api-access-7cc46") pod "c8e38ab6-ab02-44da-9682-56d413085f05" (UID: "c8e38ab6-ab02-44da-9682-56d413085f05"). InnerVolumeSpecName "kube-api-access-7cc46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:21:50.550790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.550703    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8e38ab6-ab02-44da-9682-56d413085f05-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:50.550790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.550732    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8e38ab6-ab02-44da-9682-56d413085f05-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:50.550790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:50.550746    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cc46\" (UniqueName: \"kubernetes.io/projected/c8e38ab6-ab02-44da-9682-56d413085f05-kube-api-access-7cc46\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.205852 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.205823    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"
Jul 29 01:21:51.205852 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.205827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv" event={"ID":"c8e38ab6-ab02-44da-9682-56d413085f05","Type":"ContainerDied","Data":"b2739dd2ce816871fe08bb06fb2235fd9221e16749b33b3a5ca7eb7508cf4702"}
Jul 29 01:21:51.206396 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.205877    2563 scope.go:117] "RemoveContainer" containerID="c62af694be192b2e17c17b5577385ff698a69b609bdc6d0bfa4a061bf61e3305"
Jul 29 01:21:51.236834 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.236805    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:21:51.240140 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.240091    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5f6d371c-kserve-55647ccddc-rb9zv"]
Jul 29 01:21:51.475139 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:51.475059    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e38ab6-ab02-44da-9682-56d413085f05" path="/var/lib/kubelet/pods/c8e38ab6-ab02-44da-9682-56d413085f05/volumes"
Jul 29 01:21:56.184882 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:21:56.184845    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:22:12.193877 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:22:12.193835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:22:22.693512 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:22:22.693466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:22:32.189527 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:22:32.189489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:22:42.290881 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:22:42.290839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:22:51.492336 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:22:51.492244    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:01.083332 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:01.083292    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:13.593515 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:13.593474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:22.291454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:22.291417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:31.499924 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:31.499888    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:43.085372 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:43.085328    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:23:55.190049 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:23:55.190012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:04.300565 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:04.300525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:15.888877 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:15.888843    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:27.481136 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:27.481100    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:37.693711 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:37.693673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:46.984008 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:46.983969    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:24:58.292821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:24:58.292782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:25:08.291585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:25:08.291552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:25:18.191135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:25:18.191096    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:25:29.791672 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:25:29.791633    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:26:06.803392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:26:06.803360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:26:17.587695 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:26:17.587662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:26:41.000944 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:26:41.000908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:26:51.802242 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:26:51.802209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:27:14.892185 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:27:14.892151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:27:25.685681 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:27:25.685588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:27:49.987291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:27:49.987239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:27:59.394519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:27:59.394478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:29:10.888497 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:29:10.888464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:29:26.485931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:29:26.485894    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:29:34.894921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:29:34.894890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:29:45.094135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:29:45.094098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:30:25.694234 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:30:25.694156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:30:56.086902 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:30:56.086860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:31:20.569825 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.569791    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:31:20.570195 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.570160    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8e38ab6-ab02-44da-9682-56d413085f05" containerName="main"
Jul 29 01:31:20.570195 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.570172    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e38ab6-ab02-44da-9682-56d413085f05" containerName="main"
Jul 29 01:31:20.570287 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.570234    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8e38ab6-ab02-44da-9682-56d413085f05" containerName="main"
Jul 29 01:31:20.573101 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.573078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.577585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.577558    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f46b2363\"/\"e2e-lifecycle-model-f46b2363-kserve-self-signed-certs\""
Jul 29 01:31:20.577731 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.577558    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f46b2363\"/\"openshift-service-ca.crt\""
Jul 29 01:31:20.577731 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.577605    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f46b2363\"/\"default-dockercfg-wdntk\""
Jul 29 01:31:20.577731 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.577680    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f46b2363\"/\"kube-root-ca.crt\""
Jul 29 01:31:20.584225 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.584200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:31:20.674142 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.674142 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnph9\" (UniqueName: \"kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.674437 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.674437 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.674437 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.674437 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.674386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775595 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775595 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775840 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775840 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775840 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.775840 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.775712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnph9\" (UniqueName: \"kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.776149 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.776125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.776228 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.776187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.776228 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.776217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.777963 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.777933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.778108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.778088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.784399 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.784376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnph9\" (UniqueName: \"kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9\") pod \"e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") " pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:20.883580 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:20.883537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:21.020493 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:21.020377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:31:21.022802 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:31:21.022771    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9d49ace_ab98_4751_a987_53318ad5f3d8.slice/crio-11e5f08e32471ce1a62f694d2594c6fd883935794ca59cdbff5a3aeecf98363f WatchSource:0}: Error finding container 11e5f08e32471ce1a62f694d2594c6fd883935794ca59cdbff5a3aeecf98363f: Status 404 returned error can't find the container with id 11e5f08e32471ce1a62f694d2594c6fd883935794ca59cdbff5a3aeecf98363f
Jul 29 01:31:21.110579 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:21.109908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" event={"ID":"e9d49ace-ab98-4751-a987-53318ad5f3d8","Type":"ContainerStarted","Data":"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"}
Jul 29 01:31:21.110579 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:21.109969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" event={"ID":"e9d49ace-ab98-4751-a987-53318ad5f3d8","Type":"ContainerStarted","Data":"11e5f08e32471ce1a62f694d2594c6fd883935794ca59cdbff5a3aeecf98363f"}
Jul 29 01:31:21.129191 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:21.129143    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" podStartSLOduration=1.129129062 podStartE2EDuration="1.129129062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:31:21.12734647 +0000 UTC m=+2216.205333467" watchObservedRunningTime="2026-07-29 01:31:21.129129062 +0000 UTC m=+2216.207116060"
Jul 29 01:31:23.681861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:23.681824    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:31:30.884531 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:30.884491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:30.884960 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:30.884547    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:30.896834 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:30.896805    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:31.172222 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:31.172131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:31:50.891188 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:31:50.891101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:32:09.493314 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:09.493277    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:32:18.784332 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:18.784290    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:32:19.163446 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:19.163409    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:32:34.182508 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.182435    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" podUID="e9d49ace-ab98-4751-a987-53318ad5f3d8" containerName="main" containerID="cri-o://16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205" gracePeriod=45
Jul 29 01:32:34.432440 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.432404    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:32:34.486539 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.486454    2563 generic.go:356] "Generic (PLEG): container finished" podID="e9d49ace-ab98-4751-a987-53318ad5f3d8" containerID="16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205" exitCode=0
Jul 29 01:32:34.486539 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.486490    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"
Jul 29 01:32:34.486539 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.486513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" event={"ID":"e9d49ace-ab98-4751-a987-53318ad5f3d8","Type":"ContainerDied","Data":"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"}
Jul 29 01:32:34.486785 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.486555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll" event={"ID":"e9d49ace-ab98-4751-a987-53318ad5f3d8","Type":"ContainerDied","Data":"11e5f08e32471ce1a62f694d2594c6fd883935794ca59cdbff5a3aeecf98363f"}
Jul 29 01:32:34.486785 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.486570    2563 scope.go:117] "RemoveContainer" containerID="16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"
Jul 29 01:32:34.495254 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.495235    2563 scope.go:117] "RemoveContainer" containerID="16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"
Jul 29 01:32:34.495577 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:32:34.495557    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205\": container with ID starting with 16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205 not found: ID does not exist" containerID="16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"
Jul 29 01:32:34.495629 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.495585    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205"} err="failed to get container status \"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205\": rpc error: code = NotFound desc = could not find container \"16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205\": container with ID starting with 16ef2d1a7332fba66cecbcdb0a1ad46488a590742d18845567bcf3c7af5dd205 not found: ID does not exist"
Jul 29 01:32:34.556024 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.555985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556205 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556205 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnph9\" (UniqueName: \"kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556205 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556205 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556187    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556450 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556245    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm\") pod \"e9d49ace-ab98-4751-a987-53318ad5f3d8\" (UID: \"e9d49ace-ab98-4751-a987-53318ad5f3d8\") "
Jul 29 01:32:34.556450 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:32:34.556450 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556410    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:32:34.556610 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556515    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home" (OuterVolumeSpecName: "home") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:32:34.556667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556630    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.556667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556649    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.556667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.556664    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.558542 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.558521    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9" (OuterVolumeSpecName: "kube-api-access-pnph9") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "kube-api-access-pnph9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:32:34.558802 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.558779    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:32:34.558802 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.558791    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm" (OuterVolumeSpecName: "dshm") pod "e9d49ace-ab98-4751-a987-53318ad5f3d8" (UID: "e9d49ace-ab98-4751-a987-53318ad5f3d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:32:34.658135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.658082    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnph9\" (UniqueName: \"kubernetes.io/projected/e9d49ace-ab98-4751-a987-53318ad5f3d8-kube-api-access-pnph9\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.658135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.658129    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9d49ace-ab98-4751-a987-53318ad5f3d8-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.658135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.658140    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9d49ace-ab98-4751-a987-53318ad5f3d8-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:32:34.808143 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.808110    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:32:34.811413 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:34.811386    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f46b2363/e2e-lifecycle-model-f46b2363-kserve-56f686bc55-gf7ll"]
Jul 29 01:32:35.476350 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:32:35.476317    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9d49ace-ab98-4751-a987-53318ad5f3d8" path="/var/lib/kubelet/pods/e9d49ace-ab98-4751-a987-53318ad5f3d8/volumes"
Jul 29 01:33:05.782041 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:33:05.782003    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:33:18.293121 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:33:18.293018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:34:05.694162 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:05.694129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:34:35.907066 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:35.907035    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:34:57.693872 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.693798    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:34:57.694322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.694178    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9d49ace-ab98-4751-a987-53318ad5f3d8" containerName="main"
Jul 29 01:34:57.694322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.694189    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9d49ace-ab98-4751-a987-53318ad5f3d8" containerName="main"
Jul 29 01:34:57.694322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.694291    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9d49ace-ab98-4751-a987-53318ad5f3d8" containerName="main"
Jul 29 01:34:57.697291 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.697274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.701138 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.701119    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6afea2\"/\"test-model-6afea2-kserve-self-signed-certs\""
Jul 29 01:34:57.702367 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.702346    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6afea2\"/\"openshift-service-ca.crt\""
Jul 29 01:34:57.702367 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.702363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6afea2\"/\"default-dockercfg-2cm96\""
Jul 29 01:34:57.702475 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.702372    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6afea2\"/\"kube-root-ca.crt\""
Jul 29 01:34:57.712125 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.712101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:34:57.730658 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.730753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.730753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.730753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5tqd\" (UniqueName: \"kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.730904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.730904 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.730782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831631 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5tqd\" (UniqueName: \"kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.831821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.831821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.832095 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.832068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.832161 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.832081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.832218 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.832192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.834041 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.834019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.834329 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.834312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:57.841381 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:57.841356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5tqd\" (UniqueName: \"kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd\") pod \"test-model-6afea2-kserve-85f854bd4c-lvtmc\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") " pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:58.006514 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:58.006427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:34:58.141578 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:58.141548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:34:58.143382 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:34:58.143355    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac7b71ab_4bee_404c_a886_a49d45c1cda0.slice/crio-6627ba4a3fed0cd999355094d285b1df95eff339b776eedcbf80c6c2a20b89b3 WatchSource:0}: Error finding container 6627ba4a3fed0cd999355094d285b1df95eff339b776eedcbf80c6c2a20b89b3: Status 404 returned error can't find the container with id 6627ba4a3fed0cd999355094d285b1df95eff339b776eedcbf80c6c2a20b89b3
Jul 29 01:34:58.217619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:58.217592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" event={"ID":"ac7b71ab-4bee-404c-a886-a49d45c1cda0","Type":"ContainerStarted","Data":"5c76ec742b49cc57062f8e4c30215cd72f95d9ddef40df2c3c8aaf3d3ea03615"}
Jul 29 01:34:58.217761 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:58.217630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" event={"ID":"ac7b71ab-4bee-404c-a886-a49d45c1cda0","Type":"ContainerStarted","Data":"6627ba4a3fed0cd999355094d285b1df95eff339b776eedcbf80c6c2a20b89b3"}
Jul 29 01:34:58.236989 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:34:58.236938    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" podStartSLOduration=1.236921978 podStartE2EDuration="1.236921978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:58.234459977 +0000 UTC m=+2433.312446978" watchObservedRunningTime="2026-07-29 01:34:58.236921978 +0000 UTC m=+2433.314908977"
Jul 29 01:35:02.085077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:02.085041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:35:08.007652 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:08.007611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:35:08.008185 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:08.007667    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:35:08.020254 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:08.020227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:35:08.280368 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:08.280284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:35:23.290204 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.290169    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:35:23.293962 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.293934    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.299924 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.299901    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f5ec11\"/\"kube-root-ca.crt\""
Jul 29 01:35:23.300093 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.299926    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f5ec11\"/\"test-model-f5ec11-kserve-self-signed-certs\""
Jul 29 01:35:23.300093 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.299926    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f5ec11\"/\"default-dockercfg-2tn8t\""
Jul 29 01:35:23.300093 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.299964    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f5ec11\"/\"openshift-service-ca.crt\""
Jul 29 01:35:23.318881 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.318847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:35:23.476270 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcmb9\" (UniqueName: \"kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.476270 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.476491 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.476491 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.476491 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.476491 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.476413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577160 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcmb9\" (UniqueName: \"kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577160 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577160 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577805 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.577805 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.577796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.579561 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.579541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.579846 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.579826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.596540 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.596511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcmb9\" (UniqueName: \"kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9\") pod \"test-model-f5ec11-kserve-b87bb45dc-gxz78\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") " pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.603429 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.603405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:23.749561 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:23.749525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:35:23.750293 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:35:23.750248    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b60f111_c43e_4201_88a8_860dd2ba56be.slice/crio-e2196789e2e80f6295435cfbd32e29e3e1c57a373e148a4df9b4f55aed1785d9 WatchSource:0}: Error finding container e2196789e2e80f6295435cfbd32e29e3e1c57a373e148a4df9b4f55aed1785d9: Status 404 returned error can't find the container with id e2196789e2e80f6295435cfbd32e29e3e1c57a373e148a4df9b4f55aed1785d9
Jul 29 01:35:24.350519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:24.350479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" event={"ID":"3b60f111-c43e-4201-88a8-860dd2ba56be","Type":"ContainerStarted","Data":"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"}
Jul 29 01:35:24.350519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:24.350529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" event={"ID":"3b60f111-c43e-4201-88a8-860dd2ba56be","Type":"ContainerStarted","Data":"e2196789e2e80f6295435cfbd32e29e3e1c57a373e148a4df9b4f55aed1785d9"}
Jul 29 01:35:24.374108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:24.374062    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" podStartSLOduration=1.3740466709999999 podStartE2EDuration="1.374046671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:35:24.373905139 +0000 UTC m=+2459.451892138" watchObservedRunningTime="2026-07-29 01:35:24.374046671 +0000 UTC m=+2459.452033670"
Jul 29 01:35:28.310965 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:28.310928    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:35:33.603687 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:33.603645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:33.604206 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:33.603703    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:33.616322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:33.616298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:35:34.414324 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:35:34.414295    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:36:07.513698 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:07.513666    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:36:08.600645 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:08.600611    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:36:12.896274 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:12.896234    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:36:22.532492 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.532391    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" podUID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" containerName="main" containerID="cri-o://5c76ec742b49cc57062f8e4c30215cd72f95d9ddef40df2c3c8aaf3d3ea03615" gracePeriod=45
Jul 29 01:36:22.654276 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.649774    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" containerID="5c76ec742b49cc57062f8e4c30215cd72f95d9ddef40df2c3c8aaf3d3ea03615" exitCode=0
Jul 29 01:36:22.654276 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.649952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" event={"ID":"ac7b71ab-4bee-404c-a886-a49d45c1cda0","Type":"ContainerDied","Data":"5c76ec742b49cc57062f8e4c30215cd72f95d9ddef40df2c3c8aaf3d3ea03615"}
Jul 29 01:36:22.786048 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.785988    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:36:22.830572 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830535    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5tqd\" (UniqueName: \"kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.830730 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830585    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.830730 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.830730 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830633    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.830730 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830689    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.830932 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache\") pod \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\" (UID: \"ac7b71ab-4bee-404c-a886-a49d45c1cda0\") "
Jul 29 01:36:22.831013 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830974    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home" (OuterVolumeSpecName: "home") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:22.831077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.830990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:22.831077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.831060    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:22.831177 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.831055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:22.831177 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.831075    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:22.832808 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.832773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:36:22.833077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.833059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm" (OuterVolumeSpecName: "dshm") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:22.833178 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.833083    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd" (OuterVolumeSpecName: "kube-api-access-p5tqd") pod "ac7b71ab-4bee-404c-a886-a49d45c1cda0" (UID: "ac7b71ab-4bee-404c-a886-a49d45c1cda0"). InnerVolumeSpecName "kube-api-access-p5tqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:22.932482 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.932447    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:22.932482 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.932477    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5tqd\" (UniqueName: \"kubernetes.io/projected/ac7b71ab-4bee-404c-a886-a49d45c1cda0-kube-api-access-p5tqd\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:22.932686 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.932492    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac7b71ab-4bee-404c-a886-a49d45c1cda0-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:22.932686 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:22.932504    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac7b71ab-4bee-404c-a886-a49d45c1cda0-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:23.619449 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.619412    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" podUID="3b60f111-c43e-4201-88a8-860dd2ba56be" containerName="main" containerID="cri-o://fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e" gracePeriod=45
Jul 29 01:36:23.656702 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.656675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc" event={"ID":"ac7b71ab-4bee-404c-a886-a49d45c1cda0","Type":"ContainerDied","Data":"6627ba4a3fed0cd999355094d285b1df95eff339b776eedcbf80c6c2a20b89b3"}
Jul 29 01:36:23.656822 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.656706    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"
Jul 29 01:36:23.656822 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.656729    2563 scope.go:117] "RemoveContainer" containerID="5c76ec742b49cc57062f8e4c30215cd72f95d9ddef40df2c3c8aaf3d3ea03615"
Jul 29 01:36:23.699557 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.699519    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:36:23.705616 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.705589    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6afea2/test-model-6afea2-kserve-85f854bd4c-lvtmc"]
Jul 29 01:36:23.874127 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.874099    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:36:23.941135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941150    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcmb9\" (UniqueName: \"kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941238    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941291    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941553 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm\") pod \"3b60f111-c43e-4201-88a8-860dd2ba56be\" (UID: \"3b60f111-c43e-4201-88a8-860dd2ba56be\") "
Jul 29 01:36:23.941553 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:23.941553 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home" (OuterVolumeSpecName: "home") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:23.941553 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.941543    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:23.943588 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.943555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:36:23.943711 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.943601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm" (OuterVolumeSpecName: "dshm") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:36:23.943711 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:23.943615    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9" (OuterVolumeSpecName: "kube-api-access-tcmb9") pod "3b60f111-c43e-4201-88a8-860dd2ba56be" (UID: "3b60f111-c43e-4201-88a8-860dd2ba56be"). InnerVolumeSpecName "kube-api-access-tcmb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042244    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b60f111-c43e-4201-88a8-860dd2ba56be-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042316    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcmb9\" (UniqueName: \"kubernetes.io/projected/3b60f111-c43e-4201-88a8-860dd2ba56be-kube-api-access-tcmb9\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042327    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042337    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042345    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.042376 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.042352    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b60f111-c43e-4201-88a8-860dd2ba56be-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:24.663910 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.663877    2563 generic.go:356] "Generic (PLEG): container finished" podID="3b60f111-c43e-4201-88a8-860dd2ba56be" containerID="fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e" exitCode=0
Jul 29 01:36:24.664417 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.663925    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"
Jul 29 01:36:24.664417 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.663974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" event={"ID":"3b60f111-c43e-4201-88a8-860dd2ba56be","Type":"ContainerDied","Data":"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"}
Jul 29 01:36:24.664417 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.664028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78" event={"ID":"3b60f111-c43e-4201-88a8-860dd2ba56be","Type":"ContainerDied","Data":"e2196789e2e80f6295435cfbd32e29e3e1c57a373e148a4df9b4f55aed1785d9"}
Jul 29 01:36:24.664417 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.664053    2563 scope.go:117] "RemoveContainer" containerID="fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"
Jul 29 01:36:24.673662 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.673646    2563 scope.go:117] "RemoveContainer" containerID="fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"
Jul 29 01:36:24.673903 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:36:24.673883    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e\": container with ID starting with fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e not found: ID does not exist" containerID="fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"
Jul 29 01:36:24.673957 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.673911    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e"} err="failed to get container status \"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e\": rpc error: code = NotFound desc = could not find container \"fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e\": container with ID starting with fc43aa176a5645c63bbd697503a2c764209fa78f55bd40d7bf3630d7282e0c2e not found: ID does not exist"
Jul 29 01:36:24.686756 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.686733    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:36:24.695801 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:24.695776    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f5ec11/test-model-f5ec11-kserve-b87bb45dc-gxz78"]
Jul 29 01:36:25.475199 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:25.475160    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b60f111-c43e-4201-88a8-860dd2ba56be" path="/var/lib/kubelet/pods/3b60f111-c43e-4201-88a8-860dd2ba56be/volumes"
Jul 29 01:36:25.475587 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:25.475572    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" path="/var/lib/kubelet/pods/ac7b71ab-4bee-404c-a886-a49d45c1cda0/volumes"
Jul 29 01:36:26.326556 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326526    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"]
Jul 29 01:36:26.326931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326863    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" containerName="main"
Jul 29 01:36:26.326931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326874    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" containerName="main"
Jul 29 01:36:26.326931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326900    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b60f111-c43e-4201-88a8-860dd2ba56be" containerName="main"
Jul 29 01:36:26.326931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326905    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b60f111-c43e-4201-88a8-860dd2ba56be" containerName="main"
Jul 29 01:36:26.327067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326961    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac7b71ab-4bee-404c-a886-a49d45c1cda0" containerName="main"
Jul 29 01:36:26.327067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.326969    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b60f111-c43e-4201-88a8-860dd2ba56be" containerName="main"
Jul 29 01:36:26.331108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.331089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:26.335494 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.335473    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:36:26.336807 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.336776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:36:26.348495 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.348468    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"]
Jul 29 01:36:26.463735 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.463693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td\" (UID: \"de27b7c2-5e69-4a4a-8c43-ea11a83a8134\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:26.565151 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.565109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td\" (UID: \"de27b7c2-5e69-4a4a-8c43-ea11a83a8134\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:26.565820 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.565801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td\" (UID: \"de27b7c2-5e69-4a4a-8c43-ea11a83a8134\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:26.642784 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.642736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:26.775861 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.775753    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"]
Jul 29 01:36:26.778081 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:36:26.778053    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde27b7c2_5e69_4a4a_8c43_ea11a83a8134.slice/crio-d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2 WatchSource:0}: Error finding container d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2: Status 404 returned error can't find the container with id d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2
Jul 29 01:36:26.779898 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:26.779879    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:36:27.683629 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:27.683594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td" event={"ID":"de27b7c2-5e69-4a4a-8c43-ea11a83a8134","Type":"ContainerStarted","Data":"d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2"}
Jul 29 01:36:28.491054 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:28.491017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:36:29.697385 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:29.697289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td" event={"ID":"de27b7c2-5e69-4a4a-8c43-ea11a83a8134","Type":"ContainerStarted","Data":"7f737220184c52bb27b1fdc46cfce44b95aefcd19b1fba2ab01bdb345ba66828"}
Jul 29 01:36:29.715008 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:29.714964    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td" podStartSLOduration=1.372860886 podStartE2EDuration="3.714950454s" totalImagesPullingTime="2.342089568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:36:29.713233387 +0000 UTC m=+2524.791220385" watchObservedRunningTime="2026-07-29 01:36:29.714950454 +0000 UTC m=+2524.792937452"
Jul 29 01:36:30.704334 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:30.704300    2563 generic.go:356] "Generic (PLEG): container finished" podID="de27b7c2-5e69-4a4a-8c43-ea11a83a8134" containerID="7f737220184c52bb27b1fdc46cfce44b95aefcd19b1fba2ab01bdb345ba66828" exitCode=0
Jul 29 01:36:30.704769 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:30.704388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td" event={"ID":"de27b7c2-5e69-4a4a-8c43-ea11a83a8134","Type":"ContainerDied","Data":"7f737220184c52bb27b1fdc46cfce44b95aefcd19b1fba2ab01bdb345ba66828"}
Jul 29 01:36:31.842679 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:31.842653    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:31.915405 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:31.915365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca\") pod \"de27b7c2-5e69-4a4a-8c43-ea11a83a8134\" (UID: \"de27b7c2-5e69-4a4a-8c43-ea11a83a8134\") "
Jul 29 01:36:31.915763 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:31.915739    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de27b7c2-5e69-4a4a-8c43-ea11a83a8134" (UID: "de27b7c2-5e69-4a4a-8c43-ea11a83a8134"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:32.016703 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:32.016625    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de27b7c2-5e69-4a4a-8c43-ea11a83a8134-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:36:32.714903 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:32.714876    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"
Jul 29 01:36:32.715084 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:32.714919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td" event={"ID":"de27b7c2-5e69-4a4a-8c43-ea11a83a8134","Type":"ContainerDied","Data":"d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2"}
Jul 29 01:36:32.715084 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:32.714967    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5c08ed4903d8bbe47e744d5e724ed369be382ee4e94f49803e0883278847cd2"
Jul 29 01:36:37.074120 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:37.074082    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"]
Jul 29 01:36:37.076734 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:37.076712    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f5ec11-x87td"]
Jul 29 01:36:37.476307 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:37.476253    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de27b7c2-5e69-4a4a-8c43-ea11a83a8134" path="/var/lib/kubelet/pods/de27b7c2-5e69-4a4a-8c43-ea11a83a8134/volumes"
Jul 29 01:36:40.088680 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:40.088633    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:36:56.709506 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:36:56.709466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:37:26.191372 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:37:26.191329    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:38:10.301921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:38:10.301875    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:38:31.102778 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:38:31.102732    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:38:42.391186 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:38:42.391150    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:39:13.394169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:13.394130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:39:35.791316 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.791280    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:39:35.791693 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.791673    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de27b7c2-5e69-4a4a-8c43-ea11a83a8134" containerName="revoke-keys"
Jul 29 01:39:35.791693 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.791685    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de27b7c2-5e69-4a4a-8c43-ea11a83a8134" containerName="revoke-keys"
Jul 29 01:39:35.791770 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.791740    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de27b7c2-5e69-4a4a-8c43-ea11a83a8134" containerName="revoke-keys"
Jul 29 01:39:35.794699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.794681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.797823 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.797799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"default-dockercfg-7f6mj\""
Jul 29 01:39:35.799596 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.799560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"auth-test-model-5ea1e0-kserve-self-signed-certs\""
Jul 29 01:39:35.799780 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.799596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"openshift-service-ca.crt\""
Jul 29 01:39:35.799780 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.799635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"kube-root-ca.crt\""
Jul 29 01:39:35.803432 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.803402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:39:35.845864 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.845838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.845990 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.845892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdk4s\" (UniqueName: \"kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.846034 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.845981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.846034 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.846020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.846109 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.846038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.846109 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.846097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdk4s\" (UniqueName: \"kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.947811 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.947724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.948091 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.948064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.948155 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.948086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.948214 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.948188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.950094 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.950066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.950422 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.950406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:35.957188 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:35.957158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdk4s\" (UniqueName: \"kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s\") pod \"auth-test-model-5ea1e0-kserve-665fd964ff-dn52b\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") " pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:36.105726 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:36.105692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:36.243905 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:36.243875    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:39:36.245362 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:39:36.245333    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3a00ba1_9bbe_4cff_b40b_1152a56f7b00.slice/crio-2a3160cb0538d61de97b34a34cd96605a818d19a409b935d8e222700374b481f WatchSource:0}: Error finding container 2a3160cb0538d61de97b34a34cd96605a818d19a409b935d8e222700374b481f: Status 404 returned error can't find the container with id 2a3160cb0538d61de97b34a34cd96605a818d19a409b935d8e222700374b481f
Jul 29 01:39:36.682358 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:36.682313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" event={"ID":"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00","Type":"ContainerStarted","Data":"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"}
Jul 29 01:39:36.682358 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:36.682362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" event={"ID":"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00","Type":"ContainerStarted","Data":"2a3160cb0538d61de97b34a34cd96605a818d19a409b935d8e222700374b481f"}
Jul 29 01:39:36.702310 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:36.702232    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" podStartSLOduration=1.702213463 podStartE2EDuration="1.702213463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:39:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:39:36.699933971 +0000 UTC m=+2711.777920993" watchObservedRunningTime="2026-07-29 01:39:36.702213463 +0000 UTC m=+2711.780200453"
Jul 29 01:39:39.288733 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:39.288695    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:39:46.105830 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:46.105786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:46.105830 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:46.105841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:46.119362 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:46.119335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:39:46.753864 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:39:46.753822    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:40:05.390799 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:40:05.390754    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:40:46.186992 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:40:46.186954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:41:07.195784 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:41:07.195747    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:41:24.499930 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:41:24.499890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:41:46.286128 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:41:46.286096    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:41:59.293864 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:41:59.293826    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:42:26.689989 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:42:26.689952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:42:31.408790 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:42:31.408764    2563 scope.go:117] "RemoveContainer" containerID="7f737220184c52bb27b1fdc46cfce44b95aefcd19b1fba2ab01bdb345ba66828"
Jul 29 01:42:44.493394 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:42:44.493357    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:43:06.084513 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:43:06.084477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:43:23.797635 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:43:23.797596    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:43:45.889948 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:43:45.889901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:44:03.797414 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:03.797335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:44:30.586343 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:30.586301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:44:35.902474 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:35.902440    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:44:47.594608 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:47.594572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:44:50.921561 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:50.921516    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" podUID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" containerName="main" containerID="cri-o://1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae" gracePeriod=45
Jul 29 01:44:51.171712 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.171649    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:44:51.269719 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269693    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.269907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdk4s\" (UniqueName: \"kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.269907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.269907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.269907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.269907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.269841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs\") pod \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\" (UID: \"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00\") "
Jul 29 01:44:51.270204 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.270061    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:51.270204 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.270079    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home" (OuterVolumeSpecName: "home") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:51.270204 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.270095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache" (OuterVolumeSpecName: "model-cache") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:51.271976 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.271946    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm" (OuterVolumeSpecName: "dshm") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:51.272087 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.271981    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s" (OuterVolumeSpecName: "kube-api-access-mdk4s") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "kube-api-access-mdk4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:44:51.272087 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.271980    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" (UID: "c3a00ba1-9bbe-4cff-b40b-1152a56f7b00"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:44:51.289825 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.289802    2563 generic.go:356] "Generic (PLEG): container finished" podID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" containerID="1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae" exitCode=0
Jul 29 01:44:51.289929 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.289848    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"
Jul 29 01:44:51.289929 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.289884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" event={"ID":"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00","Type":"ContainerDied","Data":"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"}
Jul 29 01:44:51.290034 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.289931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b" event={"ID":"c3a00ba1-9bbe-4cff-b40b-1152a56f7b00","Type":"ContainerDied","Data":"2a3160cb0538d61de97b34a34cd96605a818d19a409b935d8e222700374b481f"}
Jul 29 01:44:51.290034 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.289960    2563 scope.go:117] "RemoveContainer" containerID="1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"
Jul 29 01:44:51.299442 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.299423    2563 scope.go:117] "RemoveContainer" containerID="1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"
Jul 29 01:44:51.299698 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:44:51.299680    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae\": container with ID starting with 1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae not found: ID does not exist" containerID="1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"
Jul 29 01:44:51.299759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.299704    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae"} err="failed to get container status \"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae\": rpc error: code = NotFound desc = could not find container \"1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae\": container with ID starting with 1d501edcc6235d93b565bf2ea8bb7d438bd96ce3a2568f3b5317ddc151679bae not found: ID does not exist"
Jul 29 01:44:51.310804 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.310783    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:44:51.314938 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.314917    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/auth-test-model-5ea1e0-kserve-665fd964ff-dn52b"]
Jul 29 01:44:51.371266 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371243    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdk4s\" (UniqueName: \"kubernetes.io/projected/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-kube-api-access-mdk4s\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.371379 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371285    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.371379 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371303    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.371379 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371330    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.371379 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371338    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.371379 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.371347    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:44:51.476131 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:51.476051    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" path="/var/lib/kubelet/pods/c3a00ba1-9bbe-4cff-b40b-1152a56f7b00/volumes"
Jul 29 01:44:53.190869 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.190837    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:44:53.191219 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.191173    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" containerName="main"
Jul 29 01:44:53.191219 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.191183    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" containerName="main"
Jul 29 01:44:53.191309 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.191252    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a00ba1-9bbe-4cff-b40b-1152a56f7b00" containerName="main"
Jul 29 01:44:53.195465 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.195447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.200386 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.200283    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"default-dockercfg-7f6mj\""
Jul 29 01:44:53.200386 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.200291    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"kube-root-ca.crt\""
Jul 29 01:44:53.200386 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.200307    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"openshift-service-ca.crt\""
Jul 29 01:44:53.200664 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.200466    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"sub-test-model-5ea1e0-kserve-self-signed-certs\""
Jul 29 01:44:53.204516 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.204496    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:44:53.287276 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.287387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.287387 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.287459 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.287459 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287429    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.287526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.287490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j87f\" (UniqueName: \"kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388064 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j87f\" (UniqueName: \"kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388562 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388542    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388606 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.388730 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.388711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.390405 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.390382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.390675 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.390655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.397962 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.397932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j87f\" (UniqueName: \"kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f\") pod \"sub-test-model-5ea1e0-kserve-b46c76fcc-74f59\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") " pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.505862 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.505786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:44:53.636354 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:53.636327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:44:53.637822 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:44:53.637797    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda32498dc_158b_483c_bc2f_361765fd9bfb.slice/crio-21b81280a246874ec8facfdc640802d15e49dcb54011d6e4a3a33e822abe9b5e WatchSource:0}: Error finding container 21b81280a246874ec8facfdc640802d15e49dcb54011d6e4a3a33e822abe9b5e: Status 404 returned error can't find the container with id 21b81280a246874ec8facfdc640802d15e49dcb54011d6e4a3a33e822abe9b5e
Jul 29 01:44:54.317189 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:54.317152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" event={"ID":"a32498dc-158b-483c-bc2f-361765fd9bfb","Type":"ContainerStarted","Data":"6ca12cb93b1a0886519b1ac7d94cd4ff02397bd74b6ccae6e11faf2b4a2bb26b"}
Jul 29 01:44:54.317594 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:54.317201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" event={"ID":"a32498dc-158b-483c-bc2f-361765fd9bfb","Type":"ContainerStarted","Data":"21b81280a246874ec8facfdc640802d15e49dcb54011d6e4a3a33e822abe9b5e"}
Jul 29 01:44:54.335020 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:54.334974    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" podStartSLOduration=1.334961637 podStartE2EDuration="1.334961637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:44:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:44:54.333709024 +0000 UTC m=+3029.411696039" watchObservedRunningTime="2026-07-29 01:44:54.334961637 +0000 UTC m=+3029.412948636"
Jul 29 01:44:56.987158 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:44:56.987119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:45:03.506073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:03.506037    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:45:03.506576 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:03.506094    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:45:03.518325 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:03.518300    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:45:04.377211 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:04.377173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:45:24.398716 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.398628    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:45:24.402588 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.402569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.407372 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.407349    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"sub-test-model-da474e-kserve-self-signed-certs\""
Jul 29 01:45:24.407372 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.407365    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"kube-root-ca.crt\""
Jul 29 01:45:24.407536 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.407372    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"default-dockercfg-vhcjn\""
Jul 29 01:45:24.407536 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.407353    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"openshift-service-ca.crt\""
Jul 29 01:45:24.413418 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.413395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:45:24.547209 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w77x8\" (UniqueName: \"kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.547390 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.547390 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.547390 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.547568 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.547568 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.547463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.648814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.648744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.648814 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.648783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.648970 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.648905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w77x8\" (UniqueName: \"kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.648970 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.648945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.649038 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.648994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.649038 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.649014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.649153 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.649134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.649373 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.649353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.649373 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.649366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.651098 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.651078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.651186 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.651168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.657093 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.657068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w77x8\" (UniqueName: \"kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8\") pod \"sub-test-model-da474e-kserve-644977c7bf-wsmpl\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") " pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.712538 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.712517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:24.846586 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:24.846560    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:45:24.848427 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:45:24.848397    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47e60bac_977f_4e78_84e0_ffd3a2bffbad.slice/crio-98357380b9b0c16fe894326a9e64e9b3e2eb03d4d70a4dbc0420bb782ea4d78f WatchSource:0}: Error finding container 98357380b9b0c16fe894326a9e64e9b3e2eb03d4d70a4dbc0420bb782ea4d78f: Status 404 returned error can't find the container with id 98357380b9b0c16fe894326a9e64e9b3e2eb03d4d70a4dbc0420bb782ea4d78f
Jul 29 01:45:25.481450 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:25.481416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" event={"ID":"47e60bac-977f-4e78-84e0-ffd3a2bffbad","Type":"ContainerStarted","Data":"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"}
Jul 29 01:45:25.481806 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:25.481461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" event={"ID":"47e60bac-977f-4e78-84e0-ffd3a2bffbad","Type":"ContainerStarted","Data":"98357380b9b0c16fe894326a9e64e9b3e2eb03d4d70a4dbc0420bb782ea4d78f"}
Jul 29 01:45:25.494896 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:25.494850    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" podStartSLOduration=1.4948369160000001 podStartE2EDuration="1.494836916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:45:25.493590289 +0000 UTC m=+3060.571577288" watchObservedRunningTime="2026-07-29 01:45:25.494836916 +0000 UTC m=+3060.572823947"
Jul 29 01:45:28.092274 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:28.092219    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:45:34.712755 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:34.712721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:34.713189 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:34.712771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:34.724891 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:34.724868    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:45:35.532297 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:45:35.532270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:46:00.796272 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:00.796234    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:46:17.589135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:17.589097    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:46:28.715075 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:28.715034    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:46:29.875556 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:29.875517    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:46:32.586835 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:32.586800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:46:33.793191 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.793155    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:46:33.796951 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.796928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.799764 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.799737    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5ea1e0\"/\"rate-test-model-5ea1e0-kserve-self-signed-certs\""
Jul 29 01:46:33.806351 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.806325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:46:33.845836 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6msd4\" (UniqueName: \"kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.845956 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.845956 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.846037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.846037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.846037 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.845988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947104 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6msd4\" (UniqueName: \"kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947104 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947472 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947642 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.947721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.947708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.949481 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.949460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.949633 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.949614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:33.958454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:33.958433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6msd4\" (UniqueName: \"kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4\") pod \"rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") " pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:34.106208 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:34.106176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:34.237468 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:34.237439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:46:34.239309 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:46:34.239283    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61c0fbcb_56e1_4411_abdb_cbe52f8ef989.slice/crio-068dc93ca0e17d46bf0592a568875c0b7511a2139bb34e603201f763be02cc72 WatchSource:0}: Error finding container 068dc93ca0e17d46bf0592a568875c0b7511a2139bb34e603201f763be02cc72: Status 404 returned error can't find the container with id 068dc93ca0e17d46bf0592a568875c0b7511a2139bb34e603201f763be02cc72
Jul 29 01:46:34.831877 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:34.831825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" event={"ID":"61c0fbcb-56e1-4411-abdb-cbe52f8ef989","Type":"ContainerStarted","Data":"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"}
Jul 29 01:46:34.831877 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:34.831876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" event={"ID":"61c0fbcb-56e1-4411-abdb-cbe52f8ef989","Type":"ContainerStarted","Data":"068dc93ca0e17d46bf0592a568875c0b7511a2139bb34e603201f763be02cc72"}
Jul 29 01:46:34.848488 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:34.848438    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" podStartSLOduration=1.84841893 podStartE2EDuration="1.84841893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:46:34.848366145 +0000 UTC m=+3129.926353145" watchObservedRunningTime="2026-07-29 01:46:34.84841893 +0000 UTC m=+3129.926405929"
Jul 29 01:46:43.733310 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:43.733241    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" podUID="a32498dc-158b-483c-bc2f-361765fd9bfb" containerName="main" containerID="cri-o://6ca12cb93b1a0886519b1ac7d94cd4ff02397bd74b6ccae6e11faf2b4a2bb26b" gracePeriod=45
Jul 29 01:46:43.881863 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:43.881833    2563 generic.go:356] "Generic (PLEG): container finished" podID="a32498dc-158b-483c-bc2f-361765fd9bfb" containerID="6ca12cb93b1a0886519b1ac7d94cd4ff02397bd74b6ccae6e11faf2b4a2bb26b" exitCode=0
Jul 29 01:46:43.882032 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:43.881908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" event={"ID":"a32498dc-158b-483c-bc2f-361765fd9bfb","Type":"ContainerDied","Data":"6ca12cb93b1a0886519b1ac7d94cd4ff02397bd74b6ccae6e11faf2b4a2bb26b"}
Jul 29 01:46:43.983230 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:43.983207    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:46:44.035682 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035611    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.035682 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035657    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.035682 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j87f\" (UniqueName: \"kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.035919 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035694    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.035919 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035730    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.035919 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache\") pod \"a32498dc-158b-483c-bc2f-361765fd9bfb\" (UID: \"a32498dc-158b-483c-bc2f-361765fd9bfb\") "
Jul 29 01:46:44.036084 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035917    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:44.036084 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035995    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.036084 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.035979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home" (OuterVolumeSpecName: "home") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:44.036212 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.036094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache" (OuterVolumeSpecName: "model-cache") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:44.037735 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.037709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm" (OuterVolumeSpecName: "dshm") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:44.037844 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.037765    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f" (OuterVolumeSpecName: "kube-api-access-5j87f") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "kube-api-access-5j87f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:46:44.037893 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.037847    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a32498dc-158b-483c-bc2f-361765fd9bfb" (UID: "a32498dc-158b-483c-bc2f-361765fd9bfb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:46:44.107053 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.107010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:44.107170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.107071    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:44.119400 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.119382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:44.137245 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.137218    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a32498dc-158b-483c-bc2f-361765fd9bfb-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.137245 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.137238    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.137245 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.137248    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.137479 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.137278    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j87f\" (UniqueName: \"kubernetes.io/projected/a32498dc-158b-483c-bc2f-361765fd9bfb-kube-api-access-5j87f\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.137479 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.137292    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a32498dc-158b-483c-bc2f-361765fd9bfb-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:44.891319 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.891288    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"
Jul 29 01:46:44.891752 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.891286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59" event={"ID":"a32498dc-158b-483c-bc2f-361765fd9bfb","Type":"ContainerDied","Data":"21b81280a246874ec8facfdc640802d15e49dcb54011d6e4a3a33e822abe9b5e"}
Jul 29 01:46:44.891752 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.891473    2563 scope.go:117] "RemoveContainer" containerID="6ca12cb93b1a0886519b1ac7d94cd4ff02397bd74b6ccae6e11faf2b4a2bb26b"
Jul 29 01:46:44.894114 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.894083    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" podUID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" containerName="main" containerID="cri-o://7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5" gracePeriod=45
Jul 29 01:46:44.903648 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.903631    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:46:44.914915 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.914895    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:46:44.922732 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:44.922711    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/sub-test-model-5ea1e0-kserve-b46c76fcc-74f59"]
Jul 29 01:46:45.135326 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.135303    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:46:45.245519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245447    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245519 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w77x8\" (UniqueName: \"kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245739 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245739 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245739 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245739 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home\") pod \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\" (UID: \"47e60bac-977f-4e78-84e0-ffd3a2bffbad\") "
Jul 29 01:46:45.245940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache" (OuterVolumeSpecName: "model-cache") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:45.245940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:45.245940 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.245871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home" (OuterVolumeSpecName: "home") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:45.247803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.247782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:46:45.247893 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.247781    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8" (OuterVolumeSpecName: "kube-api-access-w77x8") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "kube-api-access-w77x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:46:45.247893 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.247820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm" (OuterVolumeSpecName: "dshm") pod "47e60bac-977f-4e78-84e0-ffd3a2bffbad" (UID: "47e60bac-977f-4e78-84e0-ffd3a2bffbad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:46:45.346772 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346752    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.346772 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346771    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.346903 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346781    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w77x8\" (UniqueName: \"kubernetes.io/projected/47e60bac-977f-4e78-84e0-ffd3a2bffbad-kube-api-access-w77x8\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.346903 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346789    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.346903 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346798    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.346903 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.346805    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47e60bac-977f-4e78-84e0-ffd3a2bffbad-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:46:45.475685 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.475655    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32498dc-158b-483c-bc2f-361765fd9bfb" path="/var/lib/kubelet/pods/a32498dc-158b-483c-bc2f-361765fd9bfb/volumes"
Jul 29 01:46:45.897779 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.897746    2563 generic.go:356] "Generic (PLEG): container finished" podID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" containerID="7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5" exitCode=0
Jul 29 01:46:45.898223 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.897790    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"
Jul 29 01:46:45.898223 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.897833    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" event={"ID":"47e60bac-977f-4e78-84e0-ffd3a2bffbad","Type":"ContainerDied","Data":"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"}
Jul 29 01:46:45.898223 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.897875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl" event={"ID":"47e60bac-977f-4e78-84e0-ffd3a2bffbad","Type":"ContainerDied","Data":"98357380b9b0c16fe894326a9e64e9b3e2eb03d4d70a4dbc0420bb782ea4d78f"}
Jul 29 01:46:45.898223 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.897898    2563 scope.go:117] "RemoveContainer" containerID="7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"
Jul 29 01:46:45.906454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.906435    2563 scope.go:117] "RemoveContainer" containerID="7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"
Jul 29 01:46:45.906704 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:46:45.906686    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5\": container with ID starting with 7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5 not found: ID does not exist" containerID="7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"
Jul 29 01:46:45.906754 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.906714    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5"} err="failed to get container status \"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5\": rpc error: code = NotFound desc = could not find container \"7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5\": container with ID starting with 7081821c364b3c0f8608fd615098cb191595df43e467eda10e95173b25a5c5f5 not found: ID does not exist"
Jul 29 01:46:45.914823 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.914798    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:46:45.918293 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:45.918271    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/sub-test-model-da474e-kserve-644977c7bf-wsmpl"]
Jul 29 01:46:46.685115 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:46.685070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:46:47.475499 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:47.475404    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" path="/var/lib/kubelet/pods/47e60bac-977f-4e78-84e0-ffd3a2bffbad/volumes"
Jul 29 01:46:54.889871 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:46:54.889837    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:47:27.591516 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:47:27.591478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:47:41.292082 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:47:41.292045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:47:57.685429 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:47:57.685395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:48:01.555699 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:01.555669    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:48:03.283498 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283461    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599323440"]
Jul 29 01:48:03.283867 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283842    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32498dc-158b-483c-bc2f-361765fd9bfb" containerName="main"
Jul 29 01:48:03.283867 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283854    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32498dc-158b-483c-bc2f-361765fd9bfb" containerName="main"
Jul 29 01:48:03.283867 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283866    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" containerName="main"
Jul 29 01:48:03.283977 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283871    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" containerName="main"
Jul 29 01:48:03.283977 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283933    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e60bac-977f-4e78-84e0-ffd3a2bffbad" containerName="main"
Jul 29 01:48:03.283977 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.283942    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32498dc-158b-483c-bc2f-361765fd9bfb" containerName="main"
Jul 29 01:48:03.287050 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.287033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:03.290506 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.290487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-96m7w\""
Jul 29 01:48:03.295349 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.295326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599323440"]
Jul 29 01:48:03.427234 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.427203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf4j8\" (UniqueName: \"kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8\") pod \"test-curl-7573-140388599323440\" (UID: \"73146e79-502d-48b8-892f-6b3726a059d4\") " pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:03.528240 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.528211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf4j8\" (UniqueName: \"kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8\") pod \"test-curl-7573-140388599323440\" (UID: \"73146e79-502d-48b8-892f-6b3726a059d4\") " pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:03.536607 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.536547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf4j8\" (UniqueName: \"kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8\") pod \"test-curl-7573-140388599323440\" (UID: \"73146e79-502d-48b8-892f-6b3726a059d4\") " pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:03.597066 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.597045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:03.724601 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.724575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599323440"]
Jul 29 01:48:03.726379 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:03.726357    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73146e79_502d_48b8_892f_6b3726a059d4.slice/crio-cfec6efe6a7a148ccb09d62ba331c55941a4d2f8704e34b97acf4f7e659c29ec WatchSource:0}: Error finding container cfec6efe6a7a148ccb09d62ba331c55941a4d2f8704e34b97acf4f7e659c29ec: Status 404 returned error can't find the container with id cfec6efe6a7a148ccb09d62ba331c55941a4d2f8704e34b97acf4f7e659c29ec
Jul 29 01:48:03.727652 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:03.727635    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:48:04.307759 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:04.307722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599323440" event={"ID":"73146e79-502d-48b8-892f-6b3726a059d4","Type":"ContainerStarted","Data":"cfec6efe6a7a148ccb09d62ba331c55941a4d2f8704e34b97acf4f7e659c29ec"}
Jul 29 01:48:05.314917 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:05.314881    2563 generic.go:356] "Generic (PLEG): container finished" podID="73146e79-502d-48b8-892f-6b3726a059d4" containerID="a86cd84dd88e5b5282c24e5677d8f162c77b0be9b97eb4389daab345b2bbc419" exitCode=0
Jul 29 01:48:05.315324 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:05.314969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599323440" event={"ID":"73146e79-502d-48b8-892f-6b3726a059d4","Type":"ContainerDied","Data":"a86cd84dd88e5b5282c24e5677d8f162c77b0be9b97eb4389daab345b2bbc419"}
Jul 29 01:48:06.450550 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.450528    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:06.484872 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.484846    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599323440_73146e79-502d-48b8-892f-6b3726a059d4/test-curl-7573-140388599323440/0.log"
Jul 29 01:48:06.505649 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.505621    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599323440"]
Jul 29 01:48:06.510229 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.510203    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599323440"]
Jul 29 01:48:06.556522 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.556499    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf4j8\" (UniqueName: \"kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8\") pod \"73146e79-502d-48b8-892f-6b3726a059d4\" (UID: \"73146e79-502d-48b8-892f-6b3726a059d4\") "
Jul 29 01:48:06.558586 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.558563    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8" (OuterVolumeSpecName: "kube-api-access-pf4j8") pod "73146e79-502d-48b8-892f-6b3726a059d4" (UID: "73146e79-502d-48b8-892f-6b3726a059d4"). InnerVolumeSpecName "kube-api-access-pf4j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:06.627672 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.627646    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599322032"]
Jul 29 01:48:06.627996 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.627983    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73146e79-502d-48b8-892f-6b3726a059d4" containerName="test-curl-7573-140388599323440"
Jul 29 01:48:06.628040 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.627997    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="73146e79-502d-48b8-892f-6b3726a059d4" containerName="test-curl-7573-140388599323440"
Jul 29 01:48:06.628073 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.628056    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="73146e79-502d-48b8-892f-6b3726a059d4" containerName="test-curl-7573-140388599323440"
Jul 29 01:48:06.630890 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.630876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:06.635949 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.635929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599322032"]
Jul 29 01:48:06.657556 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.657532    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf4j8\" (UniqueName: \"kubernetes.io/projected/73146e79-502d-48b8-892f-6b3726a059d4-kube-api-access-pf4j8\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:06.758007 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.757986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r9q4\" (UniqueName: \"kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4\") pod \"test-curl-7573-140388599322032\" (UID: \"1eb2c2ca-2a26-4a87-a3a1-848c639586d3\") " pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:06.858846 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.858812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r9q4\" (UniqueName: \"kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4\") pod \"test-curl-7573-140388599322032\" (UID: \"1eb2c2ca-2a26-4a87-a3a1-848c639586d3\") " pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:06.868217 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.868191    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r9q4\" (UniqueName: \"kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4\") pod \"test-curl-7573-140388599322032\" (UID: \"1eb2c2ca-2a26-4a87-a3a1-848c639586d3\") " pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:06.941231 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:06.941173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:07.071329 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:07.071304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599322032"]
Jul 29 01:48:07.075189 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:07.075145    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eb2c2ca_2a26_4a87_a3a1_848c639586d3.slice/crio-82891f65e45fbe6ee8de95cad439df0e6e4b52d87d695ba12a9989c8c859d8b8 WatchSource:0}: Error finding container 82891f65e45fbe6ee8de95cad439df0e6e4b52d87d695ba12a9989c8c859d8b8: Status 404 returned error can't find the container with id 82891f65e45fbe6ee8de95cad439df0e6e4b52d87d695ba12a9989c8c859d8b8
Jul 29 01:48:07.331931 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:07.331910    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:07.332077 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:07.331933    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfec6efe6a7a148ccb09d62ba331c55941a4d2f8704e34b97acf4f7e659c29ec"
Jul 29 01:48:07.333133 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:07.333107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599322032" event={"ID":"1eb2c2ca-2a26-4a87-a3a1-848c639586d3","Type":"ContainerStarted","Data":"82891f65e45fbe6ee8de95cad439df0e6e4b52d87d695ba12a9989c8c859d8b8"}
Jul 29 01:48:07.343246 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:07.343223    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599323440\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="73146e79-502d-48b8-892f-6b3726a059d4" pod="opendatahub/test-curl-7573-140388599323440"
Jul 29 01:48:07.476283 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:07.476239    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73146e79-502d-48b8-892f-6b3726a059d4" path="/var/lib/kubelet/pods/73146e79-502d-48b8-892f-6b3726a059d4/volumes"
Jul 29 01:48:08.339411 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:08.339374    2563 generic.go:356] "Generic (PLEG): container finished" podID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" containerID="b19c8661a3a24a763c0fa4f681308e361a1f417b461784959454e3130d178ae8" exitCode=0
Jul 29 01:48:08.339596 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:08.339426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599322032" event={"ID":"1eb2c2ca-2a26-4a87-a3a1-848c639586d3","Type":"ContainerDied","Data":"b19c8661a3a24a763c0fa4f681308e361a1f417b461784959454e3130d178ae8"}
Jul 29 01:48:09.476284 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.476245    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:09.500170 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.500141    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599322032_1eb2c2ca-2a26-4a87-a3a1-848c639586d3/test-curl-7573-140388599322032/0.log"
Jul 29 01:48:09.522436 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.522406    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599322032"]
Jul 29 01:48:09.526476 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.526451    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599322032"]
Jul 29 01:48:09.582354 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.582320    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r9q4\" (UniqueName: \"kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4\") pod \"1eb2c2ca-2a26-4a87-a3a1-848c639586d3\" (UID: \"1eb2c2ca-2a26-4a87-a3a1-848c639586d3\") "
Jul 29 01:48:09.584368 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.584341    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4" (OuterVolumeSpecName: "kube-api-access-2r9q4") pod "1eb2c2ca-2a26-4a87-a3a1-848c639586d3" (UID: "1eb2c2ca-2a26-4a87-a3a1-848c639586d3"). InnerVolumeSpecName "kube-api-access-2r9q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:09.679850 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.679772    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:09.680142 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.680129    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" containerName="test-curl-7573-140388599322032"
Jul 29 01:48:09.680185 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.680143    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" containerName="test-curl-7573-140388599322032"
Jul 29 01:48:09.680220 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.680198    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" containerName="test-curl-7573-140388599322032"
Jul 29 01:48:09.683088 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.683068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:09.683458 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.683437    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r9q4\" (UniqueName: \"kubernetes.io/projected/1eb2c2ca-2a26-4a87-a3a1-848c639586d3-kube-api-access-2r9q4\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:09.691743 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.691717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:09.784774 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.784739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xszg\" (UniqueName: \"kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg\") pod \"test-curl-7573-140388599339024\" (UID: \"8ce3c5b6-32f8-4c3d-a411-16dcaac03598\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:09.886191 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.886152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xszg\" (UniqueName: \"kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg\") pod \"test-curl-7573-140388599339024\" (UID: \"8ce3c5b6-32f8-4c3d-a411-16dcaac03598\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:09.895560 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.895534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xszg\" (UniqueName: \"kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg\") pod \"test-curl-7573-140388599339024\" (UID: \"8ce3c5b6-32f8-4c3d-a411-16dcaac03598\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:09.993242 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:09.993155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:10.132895 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:10.132871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:10.135127 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:10.135099    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ce3c5b6_32f8_4c3d_a411_16dcaac03598.slice/crio-9cdcb91520a539ed356b02a76ae56c4d556bba7d787be62d55f18a808db16feb WatchSource:0}: Error finding container 9cdcb91520a539ed356b02a76ae56c4d556bba7d787be62d55f18a808db16feb: Status 404 returned error can't find the container with id 9cdcb91520a539ed356b02a76ae56c4d556bba7d787be62d55f18a808db16feb
Jul 29 01:48:10.349981 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:10.349931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599339024" event={"ID":"8ce3c5b6-32f8-4c3d-a411-16dcaac03598","Type":"ContainerStarted","Data":"9cdcb91520a539ed356b02a76ae56c4d556bba7d787be62d55f18a808db16feb"}
Jul 29 01:48:10.351136 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:10.351115    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:10.351242 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:10.351142    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82891f65e45fbe6ee8de95cad439df0e6e4b52d87d695ba12a9989c8c859d8b8"
Jul 29 01:48:10.361933 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:10.361907    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599322032\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:11.358488 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:11.358441    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" containerID="1d05f7bc0003a90ea3e8119ec28352d25784a8eea37511c540ab17fb871acced" exitCode=0
Jul 29 01:48:11.358863 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:11.358530    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599339024" event={"ID":"8ce3c5b6-32f8-4c3d-a411-16dcaac03598","Type":"ContainerDied","Data":"1d05f7bc0003a90ea3e8119ec28352d25784a8eea37511c540ab17fb871acced"}
Jul 29 01:48:11.375073 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:11.375040    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599322032\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" pod="opendatahub/test-curl-7573-140388599322032"
Jul 29 01:48:11.475122 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:11.475092    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eb2c2ca-2a26-4a87-a3a1-848c639586d3" path="/var/lib/kubelet/pods/1eb2c2ca-2a26-4a87-a3a1-848c639586d3/volumes"
Jul 29 01:48:12.494677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.494648    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:12.515233 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.515204    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599339024_8ce3c5b6-32f8-4c3d-a411-16dcaac03598/test-curl-7573-140388599339024/0.log"
Jul 29 01:48:12.542518 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.537691    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:12.545018 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.544981    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:12.610433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.610397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xszg\" (UniqueName: \"kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg\") pod \"8ce3c5b6-32f8-4c3d-a411-16dcaac03598\" (UID: \"8ce3c5b6-32f8-4c3d-a411-16dcaac03598\") "
Jul 29 01:48:12.612497 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.612459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg" (OuterVolumeSpecName: "kube-api-access-8xszg") pod "8ce3c5b6-32f8-4c3d-a411-16dcaac03598" (UID: "8ce3c5b6-32f8-4c3d-a411-16dcaac03598"). InnerVolumeSpecName "kube-api-access-8xszg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:12.656849 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.656814    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:12.657214 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.657200    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:12.657289 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.657216    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:12.657334 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.657289    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:12.660303 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.660286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:12.666742 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.666719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:12.711929 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.711890    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xszg\" (UniqueName: \"kubernetes.io/projected/8ce3c5b6-32f8-4c3d-a411-16dcaac03598-kube-api-access-8xszg\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:12.812736 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.812700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs8dg\" (UniqueName: \"kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg\") pod \"test-curl-7573-140388599339024\" (UID: \"8034e50b-72ce-40df-b445-90d36c963f41\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:12.913666 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.913578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs8dg\" (UniqueName: \"kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg\") pod \"test-curl-7573-140388599339024\" (UID: \"8034e50b-72ce-40df-b445-90d36c963f41\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:12.922230 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.922204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs8dg\" (UniqueName: \"kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg\") pod \"test-curl-7573-140388599339024\" (UID: \"8034e50b-72ce-40df-b445-90d36c963f41\") " pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:12.969758 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:12.969731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:13.098050 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.098022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:13.103307 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:13.103276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8034e50b_72ce_40df_b445_90d36c963f41.slice/crio-33ddedfd3de4e79b23fee1667977ae5b6c2bcf62d208f065a03cc11b5d5c2511 WatchSource:0}: Error finding container 33ddedfd3de4e79b23fee1667977ae5b6c2bcf62d208f065a03cc11b5d5c2511: Status 404 returned error can't find the container with id 33ddedfd3de4e79b23fee1667977ae5b6c2bcf62d208f065a03cc11b5d5c2511
Jul 29 01:48:13.369840 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.369803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599339024" event={"ID":"8034e50b-72ce-40df-b445-90d36c963f41","Type":"ContainerStarted","Data":"33ddedfd3de4e79b23fee1667977ae5b6c2bcf62d208f065a03cc11b5d5c2511"}
Jul 29 01:48:13.370900 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.370880    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cdcb91520a539ed356b02a76ae56c4d556bba7d787be62d55f18a808db16feb"
Jul 29 01:48:13.370994 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.370903    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:13.382821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.382786    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7573-140388599339024" oldPodUID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" podUID="8034e50b-72ce-40df-b445-90d36c963f41"
Jul 29 01:48:13.475355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.475321    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce3c5b6-32f8-4c3d-a411-16dcaac03598" path="/var/lib/kubelet/pods/8ce3c5b6-32f8-4c3d-a411-16dcaac03598/volumes"
Jul 29 01:48:13.788199 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:13.788117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:48:14.377114 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:14.377081    2563 generic.go:356] "Generic (PLEG): container finished" podID="8034e50b-72ce-40df-b445-90d36c963f41" containerID="ceb20caa0b2d70fa5e5a7808e5987b91f86b87c9d4ad20dce16f17cf0914355b" exitCode=0
Jul 29 01:48:14.377317 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:14.377179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599339024" event={"ID":"8034e50b-72ce-40df-b445-90d36c963f41","Type":"ContainerDied","Data":"ceb20caa0b2d70fa5e5a7808e5987b91f86b87c9d4ad20dce16f17cf0914355b"}
Jul 29 01:48:15.515313 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.515289    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:15.533397 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.533374    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599339024_8034e50b-72ce-40df-b445-90d36c963f41/test-curl-7573-140388599339024/0.log"
Jul 29 01:48:15.535619 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.535595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs8dg\" (UniqueName: \"kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg\") pod \"8034e50b-72ce-40df-b445-90d36c963f41\" (UID: \"8034e50b-72ce-40df-b445-90d36c963f41\") "
Jul 29 01:48:15.537568 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.537537    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg" (OuterVolumeSpecName: "kube-api-access-bs8dg") pod "8034e50b-72ce-40df-b445-90d36c963f41" (UID: "8034e50b-72ce-40df-b445-90d36c963f41"). InnerVolumeSpecName "kube-api-access-bs8dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:15.553405 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.553382    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:15.555403 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.555384    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599339024"]
Jul 29 01:48:15.636505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.636447    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs8dg\" (UniqueName: \"kubernetes.io/projected/8034e50b-72ce-40df-b445-90d36c963f41-kube-api-access-bs8dg\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:15.681871 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.681837    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:15.682292 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.682249    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8034e50b-72ce-40df-b445-90d36c963f41" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:15.682292 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.682283    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8034e50b-72ce-40df-b445-90d36c963f41" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:15.682447 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.682394    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8034e50b-72ce-40df-b445-90d36c963f41" containerName="test-curl-7573-140388599339024"
Jul 29 01:48:15.685464 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.685444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:15.691581 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.691553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:15.737655 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.737628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhmpz\" (UniqueName: \"kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz\") pod \"test-curl-7573-140388599340032\" (UID: \"9cce34fa-607a-4dab-9d02-b0098636c3dc\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:15.838202 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.838168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhmpz\" (UniqueName: \"kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz\") pod \"test-curl-7573-140388599340032\" (UID: \"9cce34fa-607a-4dab-9d02-b0098636c3dc\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:15.846860 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.846840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhmpz\" (UniqueName: \"kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz\") pod \"test-curl-7573-140388599340032\" (UID: \"9cce34fa-607a-4dab-9d02-b0098636c3dc\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:15.996326 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:15.996218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:16.127392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.127366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:16.129240 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:16.129216    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cce34fa_607a_4dab_9d02_b0098636c3dc.slice/crio-458c7876853073e5af63ba0d9da8d9d32e473c03ed26b3d982df757ec5baaab8 WatchSource:0}: Error finding container 458c7876853073e5af63ba0d9da8d9d32e473c03ed26b3d982df757ec5baaab8: Status 404 returned error can't find the container with id 458c7876853073e5af63ba0d9da8d9d32e473c03ed26b3d982df757ec5baaab8
Jul 29 01:48:16.388283 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.388225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599340032" event={"ID":"9cce34fa-607a-4dab-9d02-b0098636c3dc","Type":"ContainerStarted","Data":"458c7876853073e5af63ba0d9da8d9d32e473c03ed26b3d982df757ec5baaab8"}
Jul 29 01:48:16.396049 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.396014    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33ddedfd3de4e79b23fee1667977ae5b6c2bcf62d208f065a03cc11b5d5c2511"
Jul 29 01:48:16.396192 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.396086    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:16.407048 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:16.407022    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599339024\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="8034e50b-72ce-40df-b445-90d36c963f41" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:16.573929 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.573895    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" podUID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" containerName="main" containerID="cri-o://99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd" gracePeriod=45
Jul 29 01:48:16.825365 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.825341    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:48:16.846174 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846150    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846296 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846219    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6msd4\" (UniqueName: \"kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846296 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846296 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846436 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846306    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846436 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846391    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home\") pod \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\" (UID: \"61c0fbcb-56e1-4411-abdb-cbe52f8ef989\") "
Jul 29 01:48:16.846777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846553    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache" (OuterVolumeSpecName: "model-cache") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:16.846777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846578    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:16.846777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846705    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home" (OuterVolumeSpecName: "home") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:16.846777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846713    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:16.846777 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.846773    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:16.848302 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.848278    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:48:16.848370 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.848331    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm" (OuterVolumeSpecName: "dshm") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:16.848370 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.848339    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4" (OuterVolumeSpecName: "kube-api-access-6msd4") pod "61c0fbcb-56e1-4411-abdb-cbe52f8ef989" (UID: "61c0fbcb-56e1-4411-abdb-cbe52f8ef989"). InnerVolumeSpecName "kube-api-access-6msd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:16.865100 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:16.865068    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599339024\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="8034e50b-72ce-40df-b445-90d36c963f41" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:16.947884 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.947794    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:16.947884 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.947828    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:16.947884 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.947839    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6msd4\" (UniqueName: \"kubernetes.io/projected/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-kube-api-access-6msd4\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:16.947884 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:16.947847    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61c0fbcb-56e1-4411-abdb-cbe52f8ef989-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.402941 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.402899    2563 generic.go:356] "Generic (PLEG): container finished" podID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" containerID="99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd" exitCode=0
Jul 29 01:48:17.402941 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.402943    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"
Jul 29 01:48:17.403168 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.402980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" event={"ID":"61c0fbcb-56e1-4411-abdb-cbe52f8ef989","Type":"ContainerDied","Data":"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"}
Jul 29 01:48:17.403168 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.403028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx" event={"ID":"61c0fbcb-56e1-4411-abdb-cbe52f8ef989","Type":"ContainerDied","Data":"068dc93ca0e17d46bf0592a568875c0b7511a2139bb34e603201f763be02cc72"}
Jul 29 01:48:17.403168 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.403052    2563 scope.go:117] "RemoveContainer" containerID="99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"
Jul 29 01:48:17.404691 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.404668    2563 generic.go:356] "Generic (PLEG): container finished" podID="9cce34fa-607a-4dab-9d02-b0098636c3dc" containerID="0dd466ae7994308ca8aa5c3f7a9c4dbafb7c41a4af8fb849319061c4b55b7c8e" exitCode=0
Jul 29 01:48:17.404842 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.404814    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599340032" event={"ID":"9cce34fa-607a-4dab-9d02-b0098636c3dc","Type":"ContainerDied","Data":"0dd466ae7994308ca8aa5c3f7a9c4dbafb7c41a4af8fb849319061c4b55b7c8e"}
Jul 29 01:48:17.409631 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:17.409606    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599339024\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="8034e50b-72ce-40df-b445-90d36c963f41" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:17.411976 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:17.411949    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599339024\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="8034e50b-72ce-40df-b445-90d36c963f41" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:17.413899 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.413883    2563 scope.go:117] "RemoveContainer" containerID="99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"
Jul 29 01:48:17.414193 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:17.414172    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd\": container with ID starting with 99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd not found: ID does not exist" containerID="99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"
Jul 29 01:48:17.414246 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.414201    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd"} err="failed to get container status \"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd\": rpc error: code = NotFound desc = could not find container \"99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd\": container with ID starting with 99175e4961f0a730f6ceba68ef7b06693f0f0b2588373d033adfee02ff52f3cd not found: ID does not exist"
Jul 29 01:48:17.439125 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.439096    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:48:17.444634 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.444605    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5ea1e0/rate-test-model-5ea1e0-kserve-59876bbd6c-sdvrx"]
Jul 29 01:48:17.444998 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:17.444967    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599339024\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="8034e50b-72ce-40df-b445-90d36c963f41" pod="opendatahub/test-curl-7573-140388599339024"
Jul 29 01:48:17.475341 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.475317    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" path="/var/lib/kubelet/pods/61c0fbcb-56e1-4411-abdb-cbe52f8ef989/volumes"
Jul 29 01:48:17.475654 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:17.475641    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8034e50b-72ce-40df-b445-90d36c963f41" path="/var/lib/kubelet/pods/8034e50b-72ce-40df-b445-90d36c963f41/volumes"
Jul 29 01:48:18.546524 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.546495    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:18.562735 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.562703    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhmpz\" (UniqueName: \"kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz\") pod \"9cce34fa-607a-4dab-9d02-b0098636c3dc\" (UID: \"9cce34fa-607a-4dab-9d02-b0098636c3dc\") "
Jul 29 01:48:18.565202 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.565175    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599340032_9cce34fa-607a-4dab-9d02-b0098636c3dc/test-curl-7573-140388599340032/0.log"
Jul 29 01:48:18.565384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.565312    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz" (OuterVolumeSpecName: "kube-api-access-dhmpz") pod "9cce34fa-607a-4dab-9d02-b0098636c3dc" (UID: "9cce34fa-607a-4dab-9d02-b0098636c3dc"). InnerVolumeSpecName "kube-api-access-dhmpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:18.585917 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.585888    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:18.590167 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.590145    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:18.663724 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.663648    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhmpz\" (UniqueName: \"kubernetes.io/projected/9cce34fa-607a-4dab-9d02-b0098636c3dc-kube-api-access-dhmpz\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:18.704318 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704285    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:18.704641 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704628    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cce34fa-607a-4dab-9d02-b0098636c3dc" containerName="test-curl-7573-140388599340032"
Jul 29 01:48:18.704702 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704642    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cce34fa-607a-4dab-9d02-b0098636c3dc" containerName="test-curl-7573-140388599340032"
Jul 29 01:48:18.704702 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704666    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" containerName="main"
Jul 29 01:48:18.704702 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704672    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" containerName="main"
Jul 29 01:48:18.704794 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704731    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c0fbcb-56e1-4411-abdb-cbe52f8ef989" containerName="main"
Jul 29 01:48:18.704794 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.704739    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cce34fa-607a-4dab-9d02-b0098636c3dc" containerName="test-curl-7573-140388599340032"
Jul 29 01:48:18.707721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.707704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:18.712018 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.711993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:18.764282 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.764225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stcgr\" (UniqueName: \"kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr\") pod \"test-curl-7573-140388599340032\" (UID: \"9f12f4fb-e9a5-4fa2-a827-0192aec1409d\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:18.864975 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.864931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stcgr\" (UniqueName: \"kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr\") pod \"test-curl-7573-140388599340032\" (UID: \"9f12f4fb-e9a5-4fa2-a827-0192aec1409d\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:18.874509 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:18.874472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stcgr\" (UniqueName: \"kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr\") pod \"test-curl-7573-140388599340032\" (UID: \"9f12f4fb-e9a5-4fa2-a827-0192aec1409d\") " pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:19.017640 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.017554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:19.157135 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.157111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:19.158776 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:48:19.158751    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f12f4fb_e9a5_4fa2_a827_0192aec1409d.slice/crio-b799c03121a225743d344a7c443c61e68f2ee44f99767d0a6562b37f99a04b00 WatchSource:0}: Error finding container b799c03121a225743d344a7c443c61e68f2ee44f99767d0a6562b37f99a04b00: Status 404 returned error can't find the container with id b799c03121a225743d344a7c443c61e68f2ee44f99767d0a6562b37f99a04b00
Jul 29 01:48:19.418210 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.418177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599340032" event={"ID":"9f12f4fb-e9a5-4fa2-a827-0192aec1409d","Type":"ContainerStarted","Data":"b799c03121a225743d344a7c443c61e68f2ee44f99767d0a6562b37f99a04b00"}
Jul 29 01:48:19.419369 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.419347    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:19.419474 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.419370    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="458c7876853073e5af63ba0d9da8d9d32e473c03ed26b3d982df757ec5baaab8"
Jul 29 01:48:19.431271 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.431227    2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7573-140388599340032" oldPodUID="9cce34fa-607a-4dab-9d02-b0098636c3dc" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d"
Jul 29 01:48:19.475176 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:19.475143    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cce34fa-607a-4dab-9d02-b0098636c3dc" path="/var/lib/kubelet/pods/9cce34fa-607a-4dab-9d02-b0098636c3dc/volumes"
Jul 29 01:48:20.428537 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:20.428501    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" containerID="a215882a64d38efa509314e2ee8ba71772eb4bf4af6a3b6b793b140364988f90" exitCode=0
Jul 29 01:48:20.428912 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:20.428554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7573-140388599340032" event={"ID":"9f12f4fb-e9a5-4fa2-a827-0192aec1409d","Type":"ContainerDied","Data":"a215882a64d38efa509314e2ee8ba71772eb4bf4af6a3b6b793b140364988f90"}
Jul 29 01:48:21.569657 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.569630    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:21.589307 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.589278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stcgr\" (UniqueName: \"kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr\") pod \"9f12f4fb-e9a5-4fa2-a827-0192aec1409d\" (UID: \"9f12f4fb-e9a5-4fa2-a827-0192aec1409d\") "
Jul 29 01:48:21.591529 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.591497    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr" (OuterVolumeSpecName: "kube-api-access-stcgr") pod "9f12f4fb-e9a5-4fa2-a827-0192aec1409d" (UID: "9f12f4fb-e9a5-4fa2-a827-0192aec1409d"). InnerVolumeSpecName "kube-api-access-stcgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:21.596939 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.596918    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7573-140388599340032_9f12f4fb-e9a5-4fa2-a827-0192aec1409d/test-curl-7573-140388599340032/0.log"
Jul 29 01:48:21.618566 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.618541    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:21.622049 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.622028    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7573-140388599340032"]
Jul 29 01:48:21.690627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:21.690547    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stcgr\" (UniqueName: \"kubernetes.io/projected/9f12f4fb-e9a5-4fa2-a827-0192aec1409d-kube-api-access-stcgr\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:48:22.441167 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:22.441139    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:22.441350 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:22.441196    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b799c03121a225743d344a7c443c61e68f2ee44f99767d0a6562b37f99a04b00"
Jul 29 01:48:22.451498 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:48:22.451469    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7573-140388599340032\" is forbidden: User \"system:node:ip-10-0-129-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-224.ec2.internal' and this object" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" pod="opendatahub/test-curl-7573-140388599340032"
Jul 29 01:48:23.474969 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:23.474933    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" path="/var/lib/kubelet/pods/9f12f4fb-e9a5-4fa2-a827-0192aec1409d/volumes"
Jul 29 01:48:34.281069 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:48:34.281031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:49:04.498800 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:04.498764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:49:22.050356 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.050307    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:22.050816 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.050660    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" containerName="test-curl-7573-140388599340032"
Jul 29 01:49:22.050816 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.050672    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" containerName="test-curl-7573-140388599340032"
Jul 29 01:49:22.050816 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.050745    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f12f4fb-e9a5-4fa2-a827-0192aec1409d" containerName="test-curl-7573-140388599340032"
Jul 29 01:49:22.053584 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.053566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:22.056413 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.056389    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hzslz\""
Jul 29 01:49:22.061774 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.061752    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:22.113635 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.113604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmr6q\" (UniqueName: \"kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q\") pod \"kuadrant-operator-catalog-qs9fq\" (UID: \"34e98e8c-affb-4164-8e79-13099a72b7c0\") " pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:22.214952 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.214922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmr6q\" (UniqueName: \"kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q\") pod \"kuadrant-operator-catalog-qs9fq\" (UID: \"34e98e8c-affb-4164-8e79-13099a72b7c0\") " pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:22.223015 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.222990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmr6q\" (UniqueName: \"kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q\") pod \"kuadrant-operator-catalog-qs9fq\" (UID: \"34e98e8c-affb-4164-8e79-13099a72b7c0\") " pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:22.363813 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.363779    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:22.497142 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.497118    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:22.499102 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:49:22.499077    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34e98e8c_affb_4164_8e79_13099a72b7c0.slice/crio-d3683012887b1fb9b222995ddff062156f62725d4817d4eec902badf38fa2500 WatchSource:0}: Error finding container d3683012887b1fb9b222995ddff062156f62725d4817d4eec902badf38fa2500: Status 404 returned error can't find the container with id d3683012887b1fb9b222995ddff062156f62725d4817d4eec902badf38fa2500
Jul 29 01:49:22.763966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:22.763887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" event={"ID":"34e98e8c-affb-4164-8e79-13099a72b7c0","Type":"ContainerStarted","Data":"d3683012887b1fb9b222995ddff062156f62725d4817d4eec902badf38fa2500"}
Jul 29 01:49:24.778671 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:24.778629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" event={"ID":"34e98e8c-affb-4164-8e79-13099a72b7c0","Type":"ContainerStarted","Data":"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"}
Jul 29 01:49:24.795785 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:24.795734    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" podStartSLOduration=1.3415931859999999 podStartE2EDuration="2.79571955s" totalImagesPullingTime="1.454126364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:49:24.794143512 +0000 UTC m=+3299.872130512" watchObservedRunningTime="2026-07-29 01:49:24.79571955 +0000 UTC m=+3299.873706548"
Jul 29 01:49:32.364821 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.364781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:32.365314 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.364877    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:32.386647 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.386621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:32.840158 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.840135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:32.873001 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.872964    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:32.984706 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.984669    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:49:32.988385 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.988364    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:32.991117 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.991092    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-295648\"/\"default-dockercfg-lnm7h\""
Jul 29 01:49:32.991226 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.991126    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-295648\"/\"kube-root-ca.crt\""
Jul 29 01:49:32.991312 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.991253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-295648\"/\"ipp-route-295648-kserve-self-signed-certs\""
Jul 29 01:49:32.991312 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.991252    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-295648\"/\"openshift-service-ca.crt\""
Jul 29 01:49:32.999464 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:32.999441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:49:33.111353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.111353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.111353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.111677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.111677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.111677 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.111538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvbf8\" (UniqueName: \"kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212589 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvbf8\" (UniqueName: \"kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.212803 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.212704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.213111 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.213089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.213171 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.213130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.213171 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.213145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.215059 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.215039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.215234 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.215218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.220573 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.220556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvbf8\" (UniqueName: \"kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8\") pod \"ipp-route-295648-kserve-7669ff7b59-z6ztb\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") " pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.297823 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.297792    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:33.440392 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.440364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:49:33.442434 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:49:33.442399    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc06ab9df_852b_4ec1_b6b1_256960275a2e.slice/crio-e056ff904d320938050bb4f7fd71eb833b265075faeb7e9eb0a581f4530f1072 WatchSource:0}: Error finding container e056ff904d320938050bb4f7fd71eb833b265075faeb7e9eb0a581f4530f1072: Status 404 returned error can't find the container with id e056ff904d320938050bb4f7fd71eb833b265075faeb7e9eb0a581f4530f1072
Jul 29 01:49:33.826402 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.826303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" event={"ID":"c06ab9df-852b-4ec1-b6b1-256960275a2e","Type":"ContainerStarted","Data":"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"}
Jul 29 01:49:33.826402 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.826360    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" event={"ID":"c06ab9df-852b-4ec1-b6b1-256960275a2e","Type":"ContainerStarted","Data":"e056ff904d320938050bb4f7fd71eb833b265075faeb7e9eb0a581f4530f1072"}
Jul 29 01:49:33.845384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:33.845338    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" podStartSLOduration=1.8453228529999999 podStartE2EDuration="1.845322853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:49:33.844040072 +0000 UTC m=+3308.922027072" watchObservedRunningTime="2026-07-29 01:49:33.845322853 +0000 UTC m=+3308.923309830"
Jul 29 01:49:34.834370 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:34.834318    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" podUID="34e98e8c-affb-4164-8e79-13099a72b7c0" containerName="registry-server" containerID="cri-o://3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0" gracePeriod=2
Jul 29 01:49:35.085109 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.085044    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:35.234313 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.234279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmr6q\" (UniqueName: \"kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q\") pod \"34e98e8c-affb-4164-8e79-13099a72b7c0\" (UID: \"34e98e8c-affb-4164-8e79-13099a72b7c0\") "
Jul 29 01:49:35.236384 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.236358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q" (OuterVolumeSpecName: "kube-api-access-mmr6q") pod "34e98e8c-affb-4164-8e79-13099a72b7c0" (UID: "34e98e8c-affb-4164-8e79-13099a72b7c0"). InnerVolumeSpecName "kube-api-access-mmr6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:49:35.335201 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.335129    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmr6q\" (UniqueName: \"kubernetes.io/projected/34e98e8c-affb-4164-8e79-13099a72b7c0-kube-api-access-mmr6q\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:49:35.839935 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.839897    2563 generic.go:356] "Generic (PLEG): container finished" podID="34e98e8c-affb-4164-8e79-13099a72b7c0" containerID="3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0" exitCode=0
Jul 29 01:49:35.840355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.839943    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq"
Jul 29 01:49:35.840355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.839981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" event={"ID":"34e98e8c-affb-4164-8e79-13099a72b7c0","Type":"ContainerDied","Data":"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"}
Jul 29 01:49:35.840355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.840035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qs9fq" event={"ID":"34e98e8c-affb-4164-8e79-13099a72b7c0","Type":"ContainerDied","Data":"d3683012887b1fb9b222995ddff062156f62725d4817d4eec902badf38fa2500"}
Jul 29 01:49:35.840355 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.840057    2563 scope.go:117] "RemoveContainer" containerID="3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"
Jul 29 01:49:35.849622 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.849603    2563 scope.go:117] "RemoveContainer" containerID="3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"
Jul 29 01:49:35.849896 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:49:35.849878    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0\": container with ID starting with 3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0 not found: ID does not exist" containerID="3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"
Jul 29 01:49:35.849942 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.849908    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0"} err="failed to get container status \"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0\": rpc error: code = NotFound desc = could not find container \"3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0\": container with ID starting with 3565c0851613dfe9c1ac12a2b38c8aad8a13dd999906e616f9afb0dd1c9f72a0 not found: ID does not exist"
Jul 29 01:49:35.859219 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.859198    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:35.865086 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:35.865066    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qs9fq"]
Jul 29 01:49:37.288722 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:37.288688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:49:37.475583 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:37.475543    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34e98e8c-affb-4164-8e79-13099a72b7c0" path="/var/lib/kubelet/pods/34e98e8c-affb-4164-8e79-13099a72b7c0/volumes"
Jul 29 01:49:43.298443 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:43.298403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:43.298835 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:43.298472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:43.311005 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:43.310979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:49:43.895067 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:49:43.895039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:50:02.080878 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:02.080798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:50:27.289564 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:27.289531    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:50:42.933654 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.933622    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"]
Jul 29 01:50:42.934108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.933973    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34e98e8c-affb-4164-8e79-13099a72b7c0" containerName="registry-server"
Jul 29 01:50:42.934108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.933992    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34e98e8c-affb-4164-8e79-13099a72b7c0" containerName="registry-server"
Jul 29 01:50:42.934108 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.934047    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34e98e8c-affb-4164-8e79-13099a72b7c0" containerName="registry-server"
Jul 29 01:50:42.936723 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.936707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:42.939656 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.939630    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:50:42.939782 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.939662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:50:42.947887 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:42.947853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"]
Jul 29 01:50:43.120825 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:43.120791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd\" (UID: \"5f7ae407-2397-492b-81d5-3ab62691d18c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:43.221859 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:43.221778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd\" (UID: \"5f7ae407-2397-492b-81d5-3ab62691d18c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:43.222421 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:43.222400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd\" (UID: \"5f7ae407-2397-492b-81d5-3ab62691d18c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:43.248658 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:43.248630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:43.588615 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:43.588570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"]
Jul 29 01:50:43.595237 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:50:43.595212    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7ae407_2397_492b_81d5_3ab62691d18c.slice/crio-07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3 WatchSource:0}: Error finding container 07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3: Status 404 returned error can't find the container with id 07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3
Jul 29 01:50:44.210385 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:44.210305    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd" event={"ID":"5f7ae407-2397-492b-81d5-3ab62691d18c","Type":"ContainerStarted","Data":"451b92b54cf17c075d95d4ef38f17e0be222233e7cb9c04f330065cafcc46c1a"}
Jul 29 01:50:44.210385 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:44.210363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd" event={"ID":"5f7ae407-2397-492b-81d5-3ab62691d18c","Type":"ContainerStarted","Data":"07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3"}
Jul 29 01:50:45.216858 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:45.216820    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f7ae407-2397-492b-81d5-3ab62691d18c" containerID="451b92b54cf17c075d95d4ef38f17e0be222233e7cb9c04f330065cafcc46c1a" exitCode=0
Jul 29 01:50:45.217284 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:45.216906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd" event={"ID":"5f7ae407-2397-492b-81d5-3ab62691d18c","Type":"ContainerDied","Data":"451b92b54cf17c075d95d4ef38f17e0be222233e7cb9c04f330065cafcc46c1a"}
Jul 29 01:50:46.353049 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:46.353027    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:46.549438 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:46.549360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca\") pod \"5f7ae407-2397-492b-81d5-3ab62691d18c\" (UID: \"5f7ae407-2397-492b-81d5-3ab62691d18c\") "
Jul 29 01:50:46.549724 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:46.549703    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f7ae407-2397-492b-81d5-3ab62691d18c" (UID: "5f7ae407-2397-492b-81d5-3ab62691d18c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:50:46.649894 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:46.649864    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7ae407-2397-492b-81d5-3ab62691d18c-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:50:47.228461 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:47.228430    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"
Jul 29 01:50:47.228643 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:47.228480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd" event={"ID":"5f7ae407-2397-492b-81d5-3ab62691d18c","Type":"ContainerDied","Data":"07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3"}
Jul 29 01:50:47.228643 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:47.228535    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07cb1b5ced9f221bda40bc26f1c028cfc269511d6fbd18b0375973bdd6672ce3"
Jul 29 01:50:48.441134 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:48.441087    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"]
Jul 29 01:50:48.444289 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:48.444244    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-dc8b9a-vxbjd"]
Jul 29 01:50:49.482412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:49.482377    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f7ae407-2397-492b-81d5-3ab62691d18c" path="/var/lib/kubelet/pods/5f7ae407-2397-492b-81d5-3ab62691d18c/volumes"
Jul 29 01:50:52.483866 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:52.483835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:50:58.468916 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.468877    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"]
Jul 29 01:50:58.469356 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.469328    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7ae407-2397-492b-81d5-3ab62691d18c" containerName="revoke-keys"
Jul 29 01:50:58.469356 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.469344    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7ae407-2397-492b-81d5-3ab62691d18c" containerName="revoke-keys"
Jul 29 01:50:58.469448 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.469414    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7ae407-2397-492b-81d5-3ab62691d18c" containerName="revoke-keys"
Jul 29 01:50:58.472545 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.472528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:50:58.475318 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.475297    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:50:58.475445 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.475329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:50:58.479551 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.479526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"]
Jul 29 01:50:58.556907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.556878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6\" (UID: \"d36aa85f-0860-46eb-875d-504f8ba8e8b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:50:58.657846 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.657814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6\" (UID: \"d36aa85f-0860-46eb-875d-504f8ba8e8b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:50:58.658478 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.658458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6\" (UID: \"d36aa85f-0860-46eb-875d-504f8ba8e8b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:50:58.784769 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.784683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:50:58.920352 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:58.920329    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"]
Jul 29 01:50:58.922695 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:50:58.922653    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36aa85f_0860_46eb_875d_504f8ba8e8b4.slice/crio-f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a WatchSource:0}: Error finding container f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a: Status 404 returned error can't find the container with id f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a
Jul 29 01:50:59.294778 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:59.294742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6" event={"ID":"d36aa85f-0860-46eb-875d-504f8ba8e8b4","Type":"ContainerStarted","Data":"4f96ccb9254c5d261bdf65b6dd268b1828ad2cea87176ea85991a15ccae7b47c"}
Jul 29 01:50:59.294932 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:59.294805    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6" event={"ID":"d36aa85f-0860-46eb-875d-504f8ba8e8b4","Type":"ContainerStarted","Data":"f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a"}
Jul 29 01:50:59.311074 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:50:59.311028    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6" podStartSLOduration=1.311014808 podStartE2EDuration="1.311014808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:50:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:50:59.310454808 +0000 UTC m=+3394.388441807" watchObservedRunningTime="2026-07-29 01:50:59.311014808 +0000 UTC m=+3394.389001806"
Jul 29 01:51:00.301552 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:00.301519    2563 generic.go:356] "Generic (PLEG): container finished" podID="d36aa85f-0860-46eb-875d-504f8ba8e8b4" containerID="4f96ccb9254c5d261bdf65b6dd268b1828ad2cea87176ea85991a15ccae7b47c" exitCode=0
Jul 29 01:51:00.301919 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:00.301610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6" event={"ID":"d36aa85f-0860-46eb-875d-504f8ba8e8b4","Type":"ContainerDied","Data":"4f96ccb9254c5d261bdf65b6dd268b1828ad2cea87176ea85991a15ccae7b47c"}
Jul 29 01:51:01.434627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:01.434602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:51:01.484908 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:01.484883    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca\") pod \"d36aa85f-0860-46eb-875d-504f8ba8e8b4\" (UID: \"d36aa85f-0860-46eb-875d-504f8ba8e8b4\") "
Jul 29 01:51:01.485194 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:01.485174    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d36aa85f-0860-46eb-875d-504f8ba8e8b4" (UID: "d36aa85f-0860-46eb-875d-504f8ba8e8b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:01.586399 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:01.586372    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d36aa85f-0860-46eb-875d-504f8ba8e8b4-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:02.314072 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:02.314047    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"
Jul 29 01:51:02.314072 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:02.314054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6" event={"ID":"d36aa85f-0860-46eb-875d-504f8ba8e8b4","Type":"ContainerDied","Data":"f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a"}
Jul 29 01:51:02.314273 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:02.314099    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f244631e8afeaee5e1671feb3f0deb5b7ee9a4c804cb730c3696a0847495fc9a"
Jul 29 01:51:12.896318 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:12.896280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:51:14.305928 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:14.305892    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"]
Jul 29 01:51:14.311029 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:14.311004    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-295648-zrqn6"]
Jul 29 01:51:15.476098 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:15.476065    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d36aa85f-0860-46eb-875d-504f8ba8e8b4" path="/var/lib/kubelet/pods/d36aa85f-0860-46eb-875d-504f8ba8e8b4/volumes"
Jul 29 01:51:19.637123 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:19.637046    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:51:23.805585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:23.805550    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:51:34.656015 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.655952    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" podUID="c06ab9df-852b-4ec1-b6b1-256960275a2e" containerName="main" containerID="cri-o://d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c" gracePeriod=45
Jul 29 01:51:34.912220 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.912163    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:51:34.979771 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.979921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.979921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979824    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.979921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979865    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvbf8\" (UniqueName: \"kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.979921 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.980187 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.979925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache\") pod \"c06ab9df-852b-4ec1-b6b1-256960275a2e\" (UID: \"c06ab9df-852b-4ec1-b6b1-256960275a2e\") "
Jul 29 01:51:34.980187 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.980094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:34.980187 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.980110    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home" (OuterVolumeSpecName: "home") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:34.980361 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.980197    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-home\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:34.980361 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.980215    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-tmp-dir\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:34.980361 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.980298    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:34.981948 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.981918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm" (OuterVolumeSpecName: "dshm") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:34.981948 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.981940    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:51:34.982129 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:34.982039    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8" (OuterVolumeSpecName: "kube-api-access-wvbf8") pod "c06ab9df-852b-4ec1-b6b1-256960275a2e" (UID: "c06ab9df-852b-4ec1-b6b1-256960275a2e"). InnerVolumeSpecName "kube-api-access-wvbf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:51:35.081080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.081041    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvbf8\" (UniqueName: \"kubernetes.io/projected/c06ab9df-852b-4ec1-b6b1-256960275a2e-kube-api-access-wvbf8\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:35.081080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.081078    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c06ab9df-852b-4ec1-b6b1-256960275a2e-tls-certs\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:35.081080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.081088    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-model-cache\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:35.081288 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.081096    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c06ab9df-852b-4ec1-b6b1-256960275a2e-dshm\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:51:35.488305 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.488273    2563 generic.go:356] "Generic (PLEG): container finished" podID="c06ab9df-852b-4ec1-b6b1-256960275a2e" containerID="d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c" exitCode=0
Jul 29 01:51:35.488420 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.488306    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"
Jul 29 01:51:35.488420 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.488361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" event={"ID":"c06ab9df-852b-4ec1-b6b1-256960275a2e","Type":"ContainerDied","Data":"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"}
Jul 29 01:51:35.488420 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.488399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb" event={"ID":"c06ab9df-852b-4ec1-b6b1-256960275a2e","Type":"ContainerDied","Data":"e056ff904d320938050bb4f7fd71eb833b265075faeb7e9eb0a581f4530f1072"}
Jul 29 01:51:35.488420 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.488416    2563 scope.go:117] "RemoveContainer" containerID="d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"
Jul 29 01:51:35.497353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.497331    2563 scope.go:117] "RemoveContainer" containerID="d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"
Jul 29 01:51:35.497586 ip-10-0-129-224 kubenswrapper[2563]: E0729 01:51:35.497568    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c\": container with ID starting with d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c not found: ID does not exist" containerID="d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"
Jul 29 01:51:35.497627 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.497593    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c"} err="failed to get container status \"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c\": rpc error: code = NotFound desc = could not find container \"d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c\": container with ID starting with d843bc21347187da8e286da7f0b9ce1ba5f51f18de765a57542bc5184472b97c not found: ID does not exist"
Jul 29 01:51:35.505293 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.505271    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:51:35.512284 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:35.511137    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-295648/ipp-route-295648-kserve-7669ff7b59-z6ztb"]
Jul 29 01:51:37.475058 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:51:37.475020    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c06ab9df-852b-4ec1-b6b1-256960275a2e" path="/var/lib/kubelet/pods/c06ab9df-852b-4ec1-b6b1-256960275a2e/volumes"
Jul 29 01:52:03.574251 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574219    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"]
Jul 29 01:52:03.574667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574592    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36aa85f-0860-46eb-875d-504f8ba8e8b4" containerName="revoke-keys"
Jul 29 01:52:03.574667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574603    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36aa85f-0860-46eb-875d-504f8ba8e8b4" containerName="revoke-keys"
Jul 29 01:52:03.574667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574616    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c06ab9df-852b-4ec1-b6b1-256960275a2e" containerName="main"
Jul 29 01:52:03.574667 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574621    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c06ab9df-852b-4ec1-b6b1-256960275a2e" containerName="main"
Jul 29 01:52:03.574797 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574678    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c06ab9df-852b-4ec1-b6b1-256960275a2e" containerName="main"
Jul 29 01:52:03.574797 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.574687    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36aa85f-0860-46eb-875d-504f8ba8e8b4" containerName="revoke-keys"
Jul 29 01:52:03.577766 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.577749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:03.580645 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.580623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:52:03.580912 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.580895    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:52:03.587322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.587300    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"]
Jul 29 01:52:03.715997 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.715971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt\" (UID: \"e081f944-89e2-4f5b-ae71-9acaa58711d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:03.816887 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.816856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt\" (UID: \"e081f944-89e2-4f5b-ae71-9acaa58711d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:03.817621 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.817593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt\" (UID: \"e081f944-89e2-4f5b-ae71-9acaa58711d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:03.888992 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:03.888970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:04.018426 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:04.018367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"]
Jul 29 01:52:04.021091 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:52:04.021063    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode081f944_89e2_4f5b_ae71_9acaa58711d3.slice/crio-ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f WatchSource:0}: Error finding container ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f: Status 404 returned error can't find the container with id ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f
Jul 29 01:52:04.641347 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:04.641318    2563 generic.go:356] "Generic (PLEG): container finished" podID="e081f944-89e2-4f5b-ae71-9acaa58711d3" containerID="7251a2a98a9f27efb9ec752b3d3c153873921bd6dd93d8bd822528b4b455dc99" exitCode=0
Jul 29 01:52:04.641665 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:04.641400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt" event={"ID":"e081f944-89e2-4f5b-ae71-9acaa58711d3","Type":"ContainerDied","Data":"7251a2a98a9f27efb9ec752b3d3c153873921bd6dd93d8bd822528b4b455dc99"}
Jul 29 01:52:04.641665 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:04.641468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt" event={"ID":"e081f944-89e2-4f5b-ae71-9acaa58711d3","Type":"ContainerStarted","Data":"ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f"}
Jul 29 01:52:05.772017 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:05.771992    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:05.935136 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:05.935066    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca\") pod \"e081f944-89e2-4f5b-ae71-9acaa58711d3\" (UID: \"e081f944-89e2-4f5b-ae71-9acaa58711d3\") "
Jul 29 01:52:05.935425 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:05.935405    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e081f944-89e2-4f5b-ae71-9acaa58711d3" (UID: "e081f944-89e2-4f5b-ae71-9acaa58711d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:06.035605 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:06.035580    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081f944-89e2-4f5b-ae71-9acaa58711d3-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:52:06.652419 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:06.652388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"
Jul 29 01:52:06.652612 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:06.652392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt" event={"ID":"e081f944-89e2-4f5b-ae71-9acaa58711d3","Type":"ContainerDied","Data":"ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f"}
Jul 29 01:52:06.652612 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:06.652513    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff1575358de797c744313255bdb9eabfdfb3582e1be82e906a0250e3f3a1fd7f"
Jul 29 01:52:14.366029 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:14.365975    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"]
Jul 29 01:52:14.368739 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:14.368716    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8230fd-m9pmt"]
Jul 29 01:52:15.475369 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:15.475332    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e081f944-89e2-4f5b-ae71-9acaa58711d3" path="/var/lib/kubelet/pods/e081f944-89e2-4f5b-ae71-9acaa58711d3/volumes"
Jul 29 01:52:17.496295 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:17.496252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:52:21.163908 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.163870    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"]
Jul 29 01:52:21.164411 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.164392    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e081f944-89e2-4f5b-ae71-9acaa58711d3" containerName="revoke-keys"
Jul 29 01:52:21.164493 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.164413    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e081f944-89e2-4f5b-ae71-9acaa58711d3" containerName="revoke-keys"
Jul 29 01:52:21.164545 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.164509    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e081f944-89e2-4f5b-ae71-9acaa58711d3" containerName="revoke-keys"
Jul 29 01:52:21.167798 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.167777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:21.170946 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.170922    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:52:21.171068 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.170943    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:52:21.175581 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.175561    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"]
Jul 29 01:52:21.263106 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.263078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv\" (UID: \"c1f9485f-01d9-4a10-a960-1185a53bdaff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:21.363536 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.363510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv\" (UID: \"c1f9485f-01d9-4a10-a960-1185a53bdaff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:21.364076 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.364058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv\" (UID: \"c1f9485f-01d9-4a10-a960-1185a53bdaff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:21.479623 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.479558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:21.616136 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.616108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"]
Jul 29 01:52:21.618046 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:52:21.617991    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f9485f_01d9_4a10_a960_1185a53bdaff.slice/crio-c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4 WatchSource:0}: Error finding container c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4: Status 404 returned error can't find the container with id c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4
Jul 29 01:52:21.741142 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:21.741070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv" event={"ID":"c1f9485f-01d9-4a10-a960-1185a53bdaff","Type":"ContainerStarted","Data":"c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4"}
Jul 29 01:52:22.747908 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:22.747827    2563 generic.go:356] "Generic (PLEG): container finished" podID="c1f9485f-01d9-4a10-a960-1185a53bdaff" containerID="355d6c444d35a4cc1dbd06e0cc0fb0f15adf5a0074cd58c4c452ef511a299412" exitCode=0
Jul 29 01:52:22.748234 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:22.747903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv" event={"ID":"c1f9485f-01d9-4a10-a960-1185a53bdaff","Type":"ContainerDied","Data":"355d6c444d35a4cc1dbd06e0cc0fb0f15adf5a0074cd58c4c452ef511a299412"}
Jul 29 01:52:23.884914 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:23.884890    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:23.986213 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:23.986173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca\") pod \"c1f9485f-01d9-4a10-a960-1185a53bdaff\" (UID: \"c1f9485f-01d9-4a10-a960-1185a53bdaff\") "
Jul 29 01:52:23.986556 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:23.986533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1f9485f-01d9-4a10-a960-1185a53bdaff" (UID: "c1f9485f-01d9-4a10-a960-1185a53bdaff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:24.087537 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:24.087517    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f9485f-01d9-4a10-a960-1185a53bdaff-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:52:24.760331 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:24.760299    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"
Jul 29 01:52:24.760504 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:24.760299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv" event={"ID":"c1f9485f-01d9-4a10-a960-1185a53bdaff","Type":"ContainerDied","Data":"c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4"}
Jul 29 01:52:24.760504 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:24.760453    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c92537ab7671679d200b1e00d30c08c56b201fbff660ba50290796a57f3d74c4"
Jul 29 01:52:31.972886 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:31.972852    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"]
Jul 29 01:52:31.979783 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:31.979757    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-5ea1e0-nx8vv"]
Jul 29 01:52:33.476557 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:33.476511    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f9485f-01d9-4a10-a960-1185a53bdaff" path="/var/lib/kubelet/pods/c1f9485f-01d9-4a10-a960-1185a53bdaff/volumes"
Jul 29 01:52:35.184229 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:35.184189    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:52:38.855368 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.855330    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"]
Jul 29 01:52:38.856697 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.856670    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f9485f-01d9-4a10-a960-1185a53bdaff" containerName="revoke-keys"
Jul 29 01:52:38.856838 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.856827    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f9485f-01d9-4a10-a960-1185a53bdaff" containerName="revoke-keys"
Jul 29 01:52:38.857154 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.857139    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f9485f-01d9-4a10-a960-1185a53bdaff" containerName="revoke-keys"
Jul 29 01:52:38.860989 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.860963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:38.864051 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.864032    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:52:38.864214 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.864199    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:52:38.880080 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.880041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"]
Jul 29 01:52:38.910694 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:38.910658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5\" (UID: \"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:39.011313 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.011281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5\" (UID: \"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:39.011885 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.011866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5\" (UID: \"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:39.174385 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.174302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:39.301526 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.301502    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"]
Jul 29 01:52:39.303367 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:52:39.303328    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860e2d62_ed0b_4e3c_ab05_a90c97ffdcd8.slice/crio-e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a WatchSource:0}: Error finding container e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a: Status 404 returned error can't find the container with id e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a
Jul 29 01:52:39.845718 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.845685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5" event={"ID":"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8","Type":"ContainerStarted","Data":"eedc7e08e03742da674467ef440fef010a701a428840b3d7dcef35b40b606597"}
Jul 29 01:52:39.845892 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.845743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5" event={"ID":"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8","Type":"ContainerStarted","Data":"e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a"}
Jul 29 01:52:39.863786 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:39.863742    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5" podStartSLOduration=1.863728273 podStartE2EDuration="1.863728273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:52:39.861135615 +0000 UTC m=+3494.939122611" watchObservedRunningTime="2026-07-29 01:52:39.863728273 +0000 UTC m=+3494.941715270"
Jul 29 01:52:40.852018 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:40.851979    2563 generic.go:356] "Generic (PLEG): container finished" podID="860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" containerID="eedc7e08e03742da674467ef440fef010a701a428840b3d7dcef35b40b606597" exitCode=0
Jul 29 01:52:40.852173 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:40.852056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5" event={"ID":"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8","Type":"ContainerDied","Data":"eedc7e08e03742da674467ef440fef010a701a428840b3d7dcef35b40b606597"}
Jul 29 01:52:41.986657 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:41.986630    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:42.039928 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.039896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca\") pod \"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8\" (UID: \"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8\") "
Jul 29 01:52:42.040246 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.040223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" (UID: "860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:42.141454 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.141394    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8-maas-api-service-ca\") on node \"ip-10-0-129-224.ec2.internal\" DevicePath \"\""
Jul 29 01:52:42.866907 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.866877    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"
Jul 29 01:52:42.867079 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.866875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5" event={"ID":"860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8","Type":"ContainerDied","Data":"e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a"}
Jul 29 01:52:42.867079 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:42.866997    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3cc10e347cf5c14c1629062b73a932f6243bf636b1148210477cca646bae43a"
Jul 29 01:52:49.751168 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:49.751089    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"]
Jul 29 01:52:49.754078 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:49.754052    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-da474e-7q5s5"]
Jul 29 01:52:51.475099 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:51.475063    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" path="/var/lib/kubelet/pods/860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8/volumes"
Jul 29 01:52:52.884405 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:52.884370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nkd8s"]
Jul 29 01:52:57.009756 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:52:57.009712    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b9c5dd547-qcbcc_80f73194-c496-41ed-bf3a-3b71c56277c9/authorino/0.log"
Jul 29 01:53:02.931370 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:02.931340    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-qk9b8_822573e2-adf3-4665-a524-9ef642d7ef06/copy-manifests/0.log"
Jul 29 01:53:02.937838 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:02.937810    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-qk9b8_822573e2-adf3-4665-a524-9ef642d7ef06/manager/0.log"
Jul 29 01:53:04.435070 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.435043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/extract/0.log"
Jul 29 01:53:04.440634 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.440610    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/util/0.log"
Jul 29 01:53:04.446753 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.446727    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/pull/0.log"
Jul 29 01:53:04.556078 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.556053    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/util/0.log"
Jul 29 01:53:04.561952 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.561927    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/pull/0.log"
Jul 29 01:53:04.567457 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.567439    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/extract/0.log"
Jul 29 01:53:04.675641 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.675610    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/util/0.log"
Jul 29 01:53:04.681477 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.681458    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/pull/0.log"
Jul 29 01:53:04.687001 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.686937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/extract/0.log"
Jul 29 01:53:04.798055 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.798036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/util/0.log"
Jul 29 01:53:04.803232 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.803215    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/pull/0.log"
Jul 29 01:53:04.808043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.808015    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/extract/0.log"
Jul 29 01:53:04.923276 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:04.923229    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b9c5dd547-qcbcc_80f73194-c496-41ed-bf3a-3b71c56277c9/authorino/0.log"
Jul 29 01:53:05.244272 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:05.244231    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cx4mr_7bc759c8-006e-42cd-aa6b-d01003176ee9/kuadrant-console-plugin/0.log"
Jul 29 01:53:05.581593 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:05.581508    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nkd8s_fbbf3f3e-f8be-4734-b2bd-b1355c820168/limitador/0.log"
Jul 29 01:53:06.261650 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:06.261620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-6nsqc_a0e30713-021f-4145-a2c0-88185779230c/kube-auth-proxy/0.log"
Jul 29 01:53:07.168150 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:07.168117    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ln2m9_c4b71f4c-3c33-4e7d-9396-4ba6c008c501/main/0.log"
Jul 29 01:53:07.719426 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:07.719399    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-z2rcb_72d3d73e-ba77-4769-92a9-de28c3400d46/main/0.log"
Jul 29 01:53:16.354019 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:16.353986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7fr2p_87481eaf-e0ac-43d7-87be-b329439e1fe5/global-pull-secret-syncer/0.log"
Jul 29 01:53:16.529776 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:16.529745    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cfhvf_eefea536-19ad-4de7-86c6-9e6a5d58bb17/konnectivity-agent/0.log"
Jul 29 01:53:16.599470 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:16.599440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-224.ec2.internal_ac1805d9fb4faade8c3be0e92a054aab/haproxy/0.log"
Jul 29 01:53:21.853876 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.853837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/extract/0.log"
Jul 29 01:53:21.876712 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.876683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/util/0.log"
Jul 29 01:53:21.899721 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.899671    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599mvq6_3acb9197-9efd-4d3c-9f65-c48d852e20cb/pull/0.log"
Jul 29 01:53:21.935294 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.935251    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/extract/0.log"
Jul 29 01:53:21.957992 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.957965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/util/0.log"
Jul 29 01:53:21.984419 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:21.984395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pkhdn_2d746b96-312c-495e-9f56-0451d2577427/pull/0.log"
Jul 29 01:53:22.025412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.025384    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/extract/0.log"
Jul 29 01:53:22.050283 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.050244    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/util/0.log"
Jul 29 01:53:22.071655 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.071625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73cvxnp_8e88861e-b7a1-4a2c-9283-dd4a5a983986/pull/0.log"
Jul 29 01:53:22.110407 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.110319    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/extract/0.log"
Jul 29 01:53:22.131492 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.131464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/util/0.log"
Jul 29 01:53:22.154005 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.153981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12c2nv_71f02d06-8362-4e3b-9d89-d2bc2084dcaa/pull/0.log"
Jul 29 01:53:22.820420 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.820386    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b9c5dd547-qcbcc_80f73194-c496-41ed-bf3a-3b71c56277c9/authorino/0.log"
Jul 29 01:53:22.923848 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:22.923821    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cx4mr_7bc759c8-006e-42cd-aa6b-d01003176ee9/kuadrant-console-plugin/0.log"
Jul 29 01:53:23.195933 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:23.195899    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nkd8s_fbbf3f3e-f8be-4734-b2bd-b1355c820168/limitador/0.log"
Jul 29 01:53:24.768926 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.768877    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/alertmanager/0.log"
Jul 29 01:53:24.789378 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.789348    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/config-reloader/0.log"
Jul 29 01:53:24.808793 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.808766    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/kube-rbac-proxy-web/0.log"
Jul 29 01:53:24.829289 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.829236    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/kube-rbac-proxy/0.log"
Jul 29 01:53:24.850026 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.849999    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/kube-rbac-proxy-metric/0.log"
Jul 29 01:53:24.872161 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.872133    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/prom-label-proxy/0.log"
Jul 29 01:53:24.891443 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:24.891417    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_855b1250-9f85-4266-900f-bc0755aad237/init-config-reloader/0.log"
Jul 29 01:53:25.163724 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.163693    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dpcrf_bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8/node-exporter/0.log"
Jul 29 01:53:25.183008 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.182980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dpcrf_bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8/kube-rbac-proxy/0.log"
Jul 29 01:53:25.203373 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.203350    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dpcrf_bd6b80a4-f48b-43fd-bcd9-fb8bd36f22d8/init-textfile/0.log"
Jul 29 01:53:25.311899 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.311870    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-59h4n_a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4/kube-rbac-proxy-main/0.log"
Jul 29 01:53:25.332467 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.332443    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-59h4n_a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4/kube-rbac-proxy-self/0.log"
Jul 29 01:53:25.351976 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.351945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-59h4n_a5e416d9-c8aa-4cf8-a7f4-d6a6b79c06d4/openshift-state-metrics/0.log"
Jul 29 01:53:25.630322 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.630295    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58b56ff58c-95x9j_f4c37f3d-8622-4627-97a5-adde485f486a/telemeter-client/0.log"
Jul 29 01:53:25.648873 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.648852    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58b56ff58c-95x9j_f4c37f3d-8622-4627-97a5-adde485f486a/reload/0.log"
Jul 29 01:53:25.667997 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.667970    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58b56ff58c-95x9j_f4c37f3d-8622-4627-97a5-adde485f486a/kube-rbac-proxy/0.log"
Jul 29 01:53:25.693585 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.693564    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/thanos-query/0.log"
Jul 29 01:53:25.717022 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.717003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/kube-rbac-proxy-web/0.log"
Jul 29 01:53:25.741169 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.741148    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/kube-rbac-proxy/0.log"
Jul 29 01:53:25.762412 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.762395    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/prom-label-proxy/0.log"
Jul 29 01:53:25.783042 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.783022    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/kube-rbac-proxy-rules/0.log"
Jul 29 01:53:25.803981 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:25.803964    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c5b5df75f-jlthr_fb5fa64b-6822-4560-b89c-2186b8d7cdec/kube-rbac-proxy-metrics/0.log"
Jul 29 01:53:26.125001 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.124970    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"]
Jul 29 01:53:26.125348 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.125335    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" containerName="revoke-keys"
Jul 29 01:53:26.125418 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.125350    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" containerName="revoke-keys"
Jul 29 01:53:26.125418 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.125409    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="860e2d62-ed0b-4e3c-ab05-a90c97ffdcd8" containerName="revoke-keys"
Jul 29 01:53:26.128686 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.128664    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.131344 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.131323    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4fd22\"/\"kube-root-ca.crt\""
Jul 29 01:53:26.132505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.132482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4fd22\"/\"default-dockercfg-8vnr7\""
Jul 29 01:53:26.132505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.132502    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4fd22\"/\"openshift-service-ca.crt\""
Jul 29 01:53:26.139671 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.139648    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"]
Jul 29 01:53:26.200407 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.200378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-lib-modules\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.200550 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.200424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-proc\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.200550 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.200486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-sys\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.200550 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.200522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hkgx\" (UniqueName: \"kubernetes.io/projected/bbab0354-e62d-46ee-96c4-64bdc8d52aad-kube-api-access-8hkgx\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.200684 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.200555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-podres\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.300899 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.300870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-lib-modules\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.300919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-proc\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.300950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-sys\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.300973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hkgx\" (UniqueName: \"kubernetes.io/projected/bbab0354-e62d-46ee-96c4-64bdc8d52aad-kube-api-access-8hkgx\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.301000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-podres\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301043 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.301034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-lib-modules\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301244 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.301061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-sys\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301244 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.301070    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-proc\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.301244 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.301097    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbab0354-e62d-46ee-96c4-64bdc8d52aad-podres\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.309217 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.309199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hkgx\" (UniqueName: \"kubernetes.io/projected/bbab0354-e62d-46ee-96c4-64bdc8d52aad-kube-api-access-8hkgx\") pod \"perf-node-gather-daemonset-jklvq\" (UID: \"bbab0354-e62d-46ee-96c4-64bdc8d52aad\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.439684 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.439552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:26.570082 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:26.570057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"]
Jul 29 01:53:26.571804 ip-10-0-129-224 kubenswrapper[2563]: W0729 01:53:26.571772    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbbab0354_e62d_46ee_96c4_64bdc8d52aad.slice/crio-e302eb0031fb92f36bf8f93bc2f823c679c10d8a8679c7b8566ee905a294c9ff WatchSource:0}: Error finding container e302eb0031fb92f36bf8f93bc2f823c679c10d8a8679c7b8566ee905a294c9ff: Status 404 returned error can't find the container with id e302eb0031fb92f36bf8f93bc2f823c679c10d8a8679c7b8566ee905a294c9ff
Jul 29 01:53:27.105091 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:27.105054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq" event={"ID":"bbab0354-e62d-46ee-96c4-64bdc8d52aad","Type":"ContainerStarted","Data":"291cbeea8de72d43baecaeab7ae38959775b3ebb162839693d5c4756d69c6a3c"}
Jul 29 01:53:27.105505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:27.105118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq" event={"ID":"bbab0354-e62d-46ee-96c4-64bdc8d52aad","Type":"ContainerStarted","Data":"e302eb0031fb92f36bf8f93bc2f823c679c10d8a8679c7b8566ee905a294c9ff"}
Jul 29 01:53:27.105505 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:27.105157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:27.123647 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:27.123597    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq" podStartSLOduration=1.123582131 podStartE2EDuration="1.123582131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:53:27.121780823 +0000 UTC m=+3542.199767824" watchObservedRunningTime="2026-07-29 01:53:27.123582131 +0000 UTC m=+3542.201569135"
Jul 29 01:53:28.023433 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:28.023393    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c9fbc6f8c-x5hfl_6b488a67-bead-4a91-9e85-3b36452512e8/console/0.log"
Jul 29 01:53:29.789703 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:29.789672    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89gsp_057a7481-05af-465c-acad-ff70ddbef93f/dns/0.log"
Jul 29 01:53:29.810603 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:29.810582    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89gsp_057a7481-05af-465c-acad-ff70ddbef93f/kube-rbac-proxy/0.log"
Jul 29 01:53:29.885088 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:29.885063    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g2jdj_e2e3b46e-2cdd-48e3-8704-dfa897beccda/dns-node-resolver/0.log"
Jul 29 01:53:30.494804 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:30.494768    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kfvrh_df7478cd-6084-4776-b323-38fa6eeaeddd/node-ca/0.log"
Jul 29 01:53:31.781162 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:31.781133    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-6nsqc_a0e30713-021f-4145-a2c0-88185779230c/kube-auth-proxy/0.log"
Jul 29 01:53:32.846673 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:32.846644    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s269z_8d76dbfe-b355-43ae-a6ff-bf58ffae18c4/serve-healthcheck-canary/0.log"
Jul 29 01:53:33.120062 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.119980    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-jklvq"
Jul 29 01:53:33.358489 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.358449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-ql5tv_6add1afe-4f0f-4c2e-b2e5-3ce07696f232/insights-operator/0.log"
Jul 29 01:53:33.359518 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.359497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-ql5tv_6add1afe-4f0f-4c2e-b2e5-3ce07696f232/insights-operator/1.log"
Jul 29 01:53:33.381016 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.380946    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bgqlp_a585056b-05ae-4451-93a3-cccbbdd3107b/kube-rbac-proxy/0.log"
Jul 29 01:53:33.403966 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.403943    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bgqlp_a585056b-05ae-4451-93a3-cccbbdd3107b/exporter/0.log"
Jul 29 01:53:33.431182 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:33.431157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bgqlp_a585056b-05ae-4451-93a3-cccbbdd3107b/extractor/0.log"
Jul 29 01:53:36.845449 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:36.845415    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-qk9b8_822573e2-adf3-4665-a524-9ef642d7ef06/manager/0.log"
Jul 29 01:53:36.865590 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:36.865562    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-qk9b8_822573e2-adf3-4665-a524-9ef642d7ef06/copy-manifests/0.log"
Jul 29 01:53:37.075027 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:37.074995    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-6nsqc_a0e30713-021f-4145-a2c0-88185779230c/kube-auth-proxy/0.log"
Jul 29 01:53:38.193353 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:38.193317    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c7bdf789-lpl7c_c51e8358-1435-435c-9e54-2052b2a67539/manager/0.log"
Jul 29 01:53:38.220822 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:38.220724    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ftm69_a9b16b44-5eb8-433a-8e8e-d50e9cbe1323/openshift-lws-operator/0.log"
Jul 29 01:53:38.932106 ip-10-0-129-224 kubenswrapper[2563]: I0729 01:53:38.932080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-6nsqc_a0e30713-021f-4145-a2c0-88185779230c/kube-auth-proxy/0.log"